UREL
Guidelines for UREL Annotation
Introduction
Your task is to rate the degree of semantic relatedness between two uses of a word. For instance, presented with a sentence pair as in the below table, you are asked to rate the semantic relatedness between the two uses of gras...
WSBEST
Instructions for the sense selection task
Introduction
During the course of this experiment, you will be presented with a series of sentences as well as a target word.
You will be provided several descriptions of meanings that may or may not apply to the target word. Your...
TEXT SENTIMENT
Sentiment Annotation
In the Sentiment Annotation task, annotators are shown a single text and are asked to rate the emotional sentiment as "positive", "neutral", and "negative". The non-label ('-') is used when the annotator is unable to make a judgment.
TEXT RANK
Text Rank Annotation Task
In the Text Rank annotation task, the annotators are asked to rank a set of sentences with a target word according to a specified criterion. The number of sentences may vary based on the task and annotation requirements. For example, the property could be someth...
TEXT PAIR
Guidelines for Text Pair Annotation
Introduction
Your task is to rate the degree of semantic relatedness between two uses of a word. For instance, presented with a sentence pair as in the below table, you are asked to rate the semantic relatedness between the two uses of
TEXT LABEL
Instructions for the graded sense annotation task
Adapted from Diana McCarthy's Guidelines (see references)
Introduction
You, the annotator, will view a series of sentences (uses) containing a target word. Each sentence comes with one sentence of preceding and one sentenc...
TEXT FREE SUBSITUTION
Instructions for the Text Free Substitution Task
Please note that there is no right or wrong answer. We just want your opinion for each item on this task.
During the course of this experiment, you will be presented with a series of sentences near the top of your screen. Each sente...