LICENSE

THIS FILE IS PURELY INFORMATIONAL NOT FINAL
WE ARE STILL COMPILING LICENSE INFORMATION
LICENSING WILL BE FINALIZED LATER

ETS-contributed files:
Copyright 2021-2022 Educational Testing Service

This work was funded by IES grant R305A210297

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.
A copy of the GNU General Public License is included in the COPYING file. See
also the note on the license in the README.md file.

This program incorporates source files and data from a number of sources,
as noted below. License terms for each of these incorporated works are
listed, along with information on modifications. Original content contributed
to this project by ETS team members was funded by IES Award number R305A210297.

DATASETS:

MOBY HYPHENATOR

    Moby Hyphenator data was released to the public domain as part of the Moby Project.
    See: https://web.archive.org/web/20170930060409/http://icon.shef.ac.uk/Moby/

    We include a copy of the Moby word hyphenation file as awe_workbench/parser/data/mhyph.txt


MORPHOLEX

    The MorphoLex English morphology database is licensed under the Creative Commons Attribution 4.0 International license (See: https://github.com/hugomailhot/MorphoLex-en/blob/master/LICENSE)

    This dataset is described in:
    Sachez-Gutierrez, C.H., Mailhot, H., Deacon, S.H. et al. (2017).
        MorphoLex: A derivational morphological database for 70,000 English words.
        Behavioral Research Methods, 50, 1568-1580. (2018). 
        https://doi.org/10.3758/s13428-017-0981-8

    We include a reformatted copy of this data in the source data directory as awe_workbench/parser/data/MorphoLexWords.txt


SENTIWORD

    The SentiWord sentiment lexicon is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. See: https://hlt-nlp.fbk.eu/technologies/sentiwords

    The following citations are relevant:

    Gatti, L., Guerini, M., & Turchi, M. (2016). SentiWords: Deriving a high precision and high coverage lexicon for sentiment analysis. IEEE Transactions on Affective Computing, 7(4), 409-421. [Preprint]

    Guerini M., Gatti L. & Turchi M. "Sentiment Analysis: How to Derive Prior Polarities from SentiWordNet". In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP'13), pp 1259-1269. Seattle, Washington, USA. 2013. [Preprint]

    The following citation is also relevant, ans it provided training data used in the
    above citations:

    Warriner A. B., Kuperman V. & Brysbaert M. "Norms of valence, arousal, and dominance for 13,915 English lemmas". Behavior research methods, 45(4), 1191-1207. 2013.  [Download page]
        
    We include an edited version of this data (differing primarily by the deletion of words with a polarity of 0) as awe_workbench/parser/data/SentiWord_1.0.csv.

NEW ACADEMIC WORD LIST

    The New Academic Word List is licensed under a  Creative Commons Attribution-ShareAlike 4.0 International License. (See: http://www.newgeneralservicelist.org/nawl-new-academic-word-list/)

    The following citation is relevant:
    Browne, C., Culligan, B. & Phillips, J. (2013). The New General Service List. 
        Retrieved from http://www.newgeneralservicelist.org

    We included an edited version of this data (differing primarily by the deletion of words that are frequent enough not to qualify as Tier 2 vocabulary, or which are sufficiently specialized as not to count as Tier 2 vocabulary) as awe_workbench/parser/data/NAWL.txt).

BNC-COCA WORD WORD FAMILY LIST

    The BNC-Coca Word Lists are freely available on the web, distributed by Paul Nation. They are licensed for use by anyone without restriction (Paul Nation, personal communication). Documentation of the license will be noted here as soon as it is available.

    Nation, I. S. P. (2016). Making and using word lists 
        for language learning and testing. John Benjamins
        Publishing Company.

    Further information at:

    https://www.victoria.ac.nz/__data/assets/pdf_file/0004/1689349/Information-on-the-BNC_COCA-word-family-lists-20180705.pdf
    
We include a copy of the this data in the source data directory as awe_workbench/parser/data/wordfamilies.txt. This file has been reformatted and the content slightly modified by the inclusion of some rare words in specific word families.

THE UNIVERSITY WORD LIST

    The University Word List is freely available on the web, at http://webhome.auburn.edu/~nunnath/engl6240/wlistuni.html It is licensed for use by anyone without restriction (Paul Nation, personal communication). Documentation of the license will be noted here as soon as it is available.

    The following citation is relevant:

    Xue Guoyi and Nation, I.S.P, 1984, A University Word List, Language Learning and Communication 3, 2:215-229

    We include an edited version of this data (differing primarily by the deletion of words that are frequent enough not to qualify as Tier 2 vocabulary, or which are sufficiently specialized as not to count as Tier 2 vocabulary) as awe_workbench/parser/data/UniversityWordList.txt.

ENGLISH ABSTRACTNESS/CONCRETENESS RATINGS

    The concreteness ratings provided here are calculated from the concreteness ratings supplied by:

    Scott GG, Keitel A, Becirspahic M, Yao B, Sereno SC. The Glasgow Norms: Ratings of 5,500 words on nine scales. Behav Res Methods. 2019 Jun;51(3):1258-1270. doi: 10.3758/s13428-018-1099-3. PMID: 30206797; PMCID: PMC6538586.

    This resource contains a list of more than 5,500 words, rated on multiple scales including concreteness, and is released under the Creative Commons Attribution 4.0 International License (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6538586/#). Relevant information extracted from this resource, edited to make the words listed map directly onto the first sense of a relevant WordNet synset, is contained in the data directory as the file Concr_Base.csv. This information is then used to impute concreteness ratings based on WordNet synset relationships, using the program PrepareConcr.py. Concreteness ratings are provided per word and part of speech, rather than per word as in most other concreteness resources.	

COXHEAD ACADEMIC WORD LIST

The Coxhead Academic Word List is freely available on the web, distributed by Avery Coxhead and Victoria University of Wellington. See: https://www.wgtn.ac.nz/lals/resources/academicwordlist

    These are freely available without restriction, per Averil Coxhead, personal communication. We will post the license that applies when this information is posted by the authors.
    
    We include an edited version of this data (differing primarily by the deletion of words that are frequent enough not to qualify as Tier 2 vocabulary, or which are sufficiently specialized as not to count as Tier 2 vocabulary) as awe_workbench/parser/data/awlsublists.txt. 
 
    The following citations are relevant:

    Coxhead, A. (2000). A new academic word list. TESOL quarterly, 34(2), 213-238.

THE ACADEMIC KEYWORD LIST

    The Academic Keyword list is freely available on the web, distributed by Magali Paquot and
    UC Louvain. (See: https://uclouvain.be/en/research-institutes/ilc/cecl/academic-keyword-list.html) It is released under the Creative Commons Attribution 4.0 International License.

    The following citation is relevant:

    Paquot, M. (2010). Academic Vocabulary in Learner Writing: From Extraction to Analysis. 
        London & New-York: Continuum, 29-63.

    We include an edited version of this data (differing primarily by the deletion of words that are frequent enough not to qualify as Tier 2 vocabulary, or which are sufficiently specialized as not to count as Tier 2 vocabulary) as awe_workbench/parser/data/AcademicKeywordList.txt
    
MIDDLE SCHOOL WORD LISTS

    The middle school word lists are freely available at:

        https://lextutor.ca/vp/eng/nat_lists/ms-cat_subs/english.pdf
        https://lextutor.ca/vp/eng/nat_lists/ms-cat_subs/health.pdf
        https://lextutor.ca/vp/eng/nat_lists/ms-cat_subs/math.pdf
        https://lextutor.ca/vp/eng/nat_lists/ms-cat_subs/science.pdf
        https://lextutor.ca/vp/eng/nat_lists/ms-cat_subs/social_and_history.pdf

    Relevant citation:
        Greene, J. W., & Coxhead, A. (2015). Academic vocabulary for middle
            school students. Baltimore: Brookes.

    These are freely available without restriction, per Averil Coxhead, personal communication. We will post the license that applies when this information is posted by the authors. Slightly modified copies of these lists are included in the source data directory, awe_workbench/parser/data/. Edits focus on dropping some domain specific words that are not truly Tier II vocabulary.

TRANSITION WORD LIST
    transition_terms.json: Copyright 2021 Educational Testing Service. This list of transition terms was compiled by Paul Deane as part of ongoing research into student writing, funded by IES grant #xxxx. It is not derived from, and bears no specific relation, to lists of transition words included in other ETS software. Like other ETS-contributed work, this file is distributed under the Gnu Public License (See above).

ARGUMENT AND PERSPECTIVE LEXICON
    argVoc.csv: Copyright 2021 Educational Testing Service. This lexicon of words relevant to characterizing
    argumentation, sentiment, subjectivity, and point of view effects was compiled by Paul Deane as part of ongoing research into student writing, funded by IES grant award number R305A210297. It is not derived from, and bears no specific relation, to other programs or datasets created by ETS for the purpose of analyzing argumentation, sentiment, or text subjectivity.