Natural Language Processing (AD-802 (A)) Unit 5 Notes

How unit 5 is examined

This unit covers meaning representation (FOL, description logics), syntax-driven semantics, word senses and their relations, thematic roles, selectional restrictions and word sense disambiguation; no topic was asked in the supplied papers, so each is kept short.

Requirements for representation

Not asked since 2022

Definition. A meaning representation is a formal structure that captures the meaning of a sentence so that a system can reason about it and act on it.

Key points.

Verifiability: the system must be able to compare a claim with a knowledge base to decide whether it is true.
Unambiguous representation: each input has one clear meaning form, even when the surface sentence is ambiguous.
Canonical form: sentences with the same meaning, such as "Maharani serves vegetarian food" and "Vegetarian dishes are served by Maharani", map to one representation.
Inference and variables: the representation must support drawing new conclusions and handle variables, and it must be expressive enough to cover the domain.

First-Order Logic

Definition. First-Order Logic (FOL) represents meaning using constants, predicates, variables, connectives and the quantifiers $\forall$ and $\exists$.

Key points.

Terms name objects: constants (Maharani), functions (LocationOf(x)) and variables (x).
Predicates express relations, for example $Serves(Maharani, VegetarianFood)$.
Connectives $\land, \lor, \neg, \Rightarrow$ combine formulas, and $\forall x$ (all) and $\exists x$ (some) quantify variables.
Example: "A restaurant serves Indian food" becomes $\exists x\, Restaurant(x) \land Serves(x, IndianFood)$.

Description Logics

Definition. Description Logics are a family of decidable fragments of FOL that describe the world using concepts (classes), roles (relations) and individuals.

Key points.

They are the formal basis of ontologies and the semantic web language OWL.
A TBox holds concept definitions and subsumption ($Dog \sqsubseteq Animal$), and an ABox holds facts about individuals.
Main reasoning tasks are subsumption (is one concept more general than another?) and instance checking (does an individual belong to a concept?).
They trade some expressive power for efficient, guaranteed-to-finish reasoning.

Syntax-Driven Semantic analysis

Definition. Syntax-driven semantic analysis builds the meaning of a sentence from the meanings of its parts, guided by the parse tree, following the principle of compositionality.

Key points.

The meaning of the whole is a function of the meanings of its words and the way they are combined syntactically.
The parser first produces the parse tree, and semantic rules then compute meaning bottom-up from the leaves.
Lambda calculus is used to compose meanings, for example $\lambda x.\,Eats(x, Rice)$ applied to $Ram$ gives $Eats(Ram, Rice)$.
Idioms such as "kick the bucket" break strict compositionality.

Semantic attachments

Definition. A semantic attachment is a semantic rule attached to a grammar rule that tells how to compute the meaning of the parent from the meanings of its children.

Key points.

Each CFG rule is written as $A \rightarrow \alpha_1 \dots \alpha_n \{f(\alpha_1.sem, \dots, \alpha_n.sem)\}$.
Example: $NP \rightarrow ProperNoun \{ProperNoun.sem\}$ passes the noun's meaning up unchanged.
Example: $S \rightarrow NP\ VP \{VP.sem(NP.sem)\}$ applies the verb phrase meaning to the subject.
Attachments are evaluated bottom-up during or after parsing to give the final logical form.

Word Senses

Definition. A word sense is one discrete meaning of a word; a word with several senses is polysemous or homonymous.

Key points.

Homonymy means unrelated meanings sharing one form, such as "bank" (river edge) and "bank" (money institution).
Polysemy means related meanings of one word, such as "bank" as a building and as an institution.
Lexical resources like WordNet store the senses of each word as a sense inventory, grouped in synsets.
Choosing the right sense in context is the task of word sense disambiguation.

Relations between Senses

Definition. Sense relations are the semantic links between word senses, such as synonymy, antonymy, hypernymy and hyponymy.

Key points.

Synonymy: two senses with nearly the same meaning, such as "big" and "large".
Antonymy: opposite senses, such as "hot" and "cold".
Hypernymy and hyponymy: "vehicle" is a hypernym (general) of "car", and "car" is a hyponym (specific) of "vehicle".
Meronymy is the part-of relation ("wheel" is part of "car"), and WordNet stores all of these relations.

Thematic Roles

Definition. Thematic roles (semantic roles) describe the role a noun phrase plays in the event named by the verb, independent of its syntactic position.

Key points.

Agent is the doer of the action, and Theme or Patient is the entity affected.
Other roles are Experiencer, Instrument, Source, Goal and Beneficiary.
In "Ram opened the door with a key", Ram is Agent, door is Theme and key is Instrument.
The same roles appear under different syntax: "The door was opened by Ram" keeps Ram as Agent.

selectional restrictions

Definition. A selectional restriction is a semantic constraint that a verb places on the type of argument it accepts for each thematic role.

Key points.

"Eat" requires an edible Theme, so "ate a concrete slab" violates the restriction.
Restrictions are stated with semantic types from a hierarchy such as WordNet hypernyms.
They help disambiguate word senses: in "ate the bass", the fish sense of bass fits the edible constraint.
They fail for metaphor and figurative use, such as "the car drinks petrol".

WSD using Supervised

Definition. Supervised WSD trains a classifier on a corpus in which each ambiguous word is hand-labelled with its correct sense, then predicts the sense of new occurrences.

Key points.

Features come from the context: neighbouring words (collocations), surrounding bag of words, and POS tags.
Common classifiers are Naive Bayes, decision lists and SVMs, and Naive Bayes picks $\hat{s}=\arg\max_s P(s)\prod_j P(f_j\mid s)$.
Sense-tagged corpora such as SemCor provide the training data.
Its main drawback is the cost of manual sense tagging, which limits coverage to few words.

Dictionary and Thesaurus

Definition. Dictionary and thesaurus based WSD, such as the Lesk algorithm, chooses the sense whose dictionary definition overlaps most with the context words.

Key points.

Simplified Lesk counts the words shared between each sense's gloss and the words around the target.
The sense with the highest overlap count is selected.
Thesaurus methods (Walker) use the categories of the context words and pick the sense whose category is most frequent.
These knowledge-based methods need no labelled corpus, but they are less accurate than supervised methods.

Applications of WSD

Definition. Word sense disambiguation is used wherever the correct meaning of a word decides the quality of a language task.

Key points.

Machine translation needs it to pick the right target word, as "bank" translates differently for river and money.
Information retrieval uses it to return documents matching the intended sense of the query.
Question answering, text summarisation and sentiment analysis use it to understand the meaning correctly.
Speech synthesis needs it to pronounce words like "bass" correctly.

Last-minute revision

A meaning representation must be verifiable, unambiguous, in canonical form and support inference.
FOL uses constants, predicates, variables, connectives and the quantifiers $\forall$ and $\exists$.
Description logics are decidable FOL fragments with a TBox (concepts) and an ABox (facts), the base of OWL.
Compositionality: the meaning of the whole is built from the meanings of the parts through the parse tree.
A semantic attachment is a meaning rule written next to a grammar rule.
Homonymy is unrelated meanings; polysemy is related meanings.
Hypernym is the general term, hyponym the specific term, meronym the part.
Thematic roles include Agent, Theme, Instrument, Source, Goal, Experiencer.
A selectional restriction limits the semantic type of a verb's argument.
Supervised WSD needs a sense-tagged corpus; Lesk needs only a dictionary.
Lesk picks the sense with the largest gloss overlap with the context.

Memory hooks

Verifiable, Unambiguous, Canonical, Inference: "VUCI" for the four representation needs.
Hyper = over (general), Hypo = under (specific).
Agent acts, Patient suffers, Instrument helps.
Lesk = overlap: count shared words.
Supervised means a teacher labels the senses; knowledge-based means the dictionary is the teacher.

Coverage checklist

Requirements for representation: no past questions in the supplied papers.
First-Order Logic: no past questions in the supplied papers.
Description Logics: no past questions in the supplied papers.
Syntax-Driven Semantic analysis: no past questions in the supplied papers.
Semantic attachments: no past questions in the supplied papers.
Word Senses: no past questions in the supplied papers.
Relations between Senses: no past questions in the supplied papers.
Thematic Roles: no past questions in the supplied papers.
selectional restrictions: no past questions in the supplied papers.
WSD using Supervised: no past questions in the supplied papers.
Dictionary and Thesaurus: no past questions in the supplied papers.
Applications of WSD: no past questions in the supplied papers.

Account

Natural Language Processing (AD-802 (A)) - Unit 5 Short Notes

How unit 5 is examined

Requirements for representation

First-Order Logic

Description Logics

Syntax-Driven Semantic analysis

Semantic attachments

Word Senses

Relations between Senses

Thematic Roles

selectional restrictions

WSD using Supervised

Dictionary and Thesaurus

Applications of WSD

Last-minute revision

Memory hooks

Coverage checklist