Le TAL statistique

Transcription

Le TAL statistique

Méthodes Statistiques en Traitement des Langues
État des lieux et perspectives
Journée d’étude sur le TAL,
Nancy, le 15 janvier 2013
François Yvon
François Yvon
Méthodes Statistiques en TAL
1 / 34
Le TAL, Keksé ?
Le Traitement Automatique des Langues (Gazdar, 1996)
théorie du calcul linguistique
(...) is the study of the computational, mathematical and statistical properties of natural
languages and systems for processing natural languages.
pyscho-linguistique computationnelle
Computational psycholinguistics involves the construction of psycho-logically motivated
computational models of aspects of human NLP.
outils de traitement pour des applications
Applied NLP involves the construction of intelligent computational artefacts that process
natural languages in ways that are useful to people other than computational linguists.
François Yvon
2 / 34
Le TAL, Keksé ?
Le Traitement Automatique des Langues (Gazdar, 1996)
Méthodes statistiques en TAL
The data intensive approach to language, which is becoming known as ’Text Analysis’ , takes a
pragmatic view that is well suited to meet the recent emphasis on evaluations and concrete
deliverables. Text Analysis focuses on a broad (though superficial) coverage of unrestricted text,
rather than deep analysis of (artificially) restricted domain. (Church & Mercer, 1993)
Exploiter de grands corpus pour traiter de grands corpus
François Yvon
2 / 34
Une approche couronnée de succès
From Peter Norvig (http://norvig.com/chomsky.html)
Search engines: 100% of major players are trained and probabilistic.
Their operation cannot be described by a simple function.
Speech recognition: 100% of major systems are trained and
probabilistic, mostly relying on probabilistic hidden Markov models.
Machine translation: 100% of top competitors in competitions such
as NIST use statistical methods.
Question answering: this application is less well-developed, and
many systems build heavily on the statistical and probabilistic
approach used by search engines.
François Yvon
3 / 34
Now let’s look at some components that are of interest only to
the computational linguist, not to the end user
François Yvon
3 / 34
Word sense disambiguation: 100% of top competitors at the
SemEval-2 competition used statistical techniques; most are
probabilistic; some use a hybrid approach incorporating rules from
sources such as Wordnet.
Coreference resolution: The majority of current systems are
statistical, although we should mention the system of Haghighi and
Klein, which can be described as a hybrid system that is mostly
rule-based rather than trained, and performs on par with top statistical
systems.
Part of speech tagging: Most current systems are statistical. The
Brill tagger stands out as a successful hybrid system: it learns a set
of deterministic rules from statistical data.
Parsing: There are many parsing systems, using multiple
approaches. Almost all of the most successful are statistical, and the
majority are probabilistic (with a substantial minority of deterministic
parsers).
François Yvon
3 / 34
Couronnée de trop de succès ?
Un exercice d’auto-analyse... lexicale
(...) we may have been too successful. Not only have we
succeeded in making room for what we were interested in,
but there is no longer much room for anything else (Church,
2011)
François Yvon
4 / 34
(...) we may have been too successful. Not only have we succeeded in making room
for what we were interested in, but there is no longer much room for anything else
(Church, 2011)
d’après (Hall, Jurafsky, & Manning, 2008)
François Yvon
4 / 34
(...) we may have been too successful. Not only have we succeeded in making room
for what we were interested in, but there is no longer much room for anything else
(Church, 2011)
Analyse de 16 500 articles de aclweb.org/anthology-new
François Yvon
4 / 34
Un contexte favorable
The increasing availability of machine-readable corpora has
suggested new methods for studies in a variety of areas such
as lexical knowledge acquisition, grammar construction, and
machine translation.
Though common in the speech community, the use of
statistical and probabilistic methods to discover and organize
data is relatively new to the field at large. (Armstrong, 1993)
Un terreau fertile
des données [annotées et brutes], des machines pour les traiter
méthodes de traitement [RF, TLP, IR]
arguments théoriques [retour de l’empiricisme]
incitations sonnantes et trébuchantes [les campagnes
d’évaluation du DARPA]
François Yvon
5 / 34
Des modèles simples peuvent avoir une valeur grande d’usage
François Yvon
6 / 34
Reconnaissance vocale: modéliser les séquences
Anytime a linguist leaves the group, the recognition rate goes up (Jelinek, 1985)
Une façon pratique de probabiliser des phrases
P(w1 . . . wL ) =
L
Y
P(wi |wi−n+1 ...wi−1 )
i=n
Modéliser la syntaxe avec des dépendances locales
François Yvon
7 / 34
Reconnaissance vocale: modéliser les séquences
Anytime a linguist leaves the group, the recognition rate goes up (Jelinek, 1985)
Une façon pratique de probabiliser des phrases
P(w1 . . . wL ) =
L
Y
P(wi |wi−n+1 ...wi−1 )
i=n
Modéliser la syntaxe avec des dépendances locales
François Yvon
7 / 34
Recherche d’information: les “ sacs-de-mots”
text indexing systems based on the assignment of appropriately weighted single terms
produce retrieval results that are superior to those obtainable with other more elaborate
text representations. (Salton & Buckley, 1988)
Représentation numérique d’un document
d ∈ R|V| , dT =(θ1 , . . . θ|V| ), avec θi = f (c(wi ∈ d), c(wi ∈ D))
c(wi ∈ d)
ou ...
ld
dist(d, d0 ) ∝ dT d0 ou ||d − d0 ||k ou ...
θi = TF-IDF(wi ) ou
Modéliser la similarité de contenu en ignorant la syntaxe
François Yvon
8 / 34
Recherche d’information: les “ sacs-de-mots”
text indexing systems based on the assignment of appropriately weighted single terms
produce retrieval results that are superior to those obtainable with other more elaborate
text representations. (Salton & Buckley, 1988)
Représentation numérique d’un document
d ∈ R|V| , dT =(θ1 , . . . θ|V| ), avec θi = f (c(wi ∈ d), c(wi ∈ D))
c(wi ∈ d)
ou ...
ld
dist(d, d0 ) ∝ dT d0 ou ||d − d0 ||k ou ...
θi = TF-IDF(wi ) ou
Modéliser la similarité de contenu en ignorant la syntaxe
François Yvon
8 / 34
Lexicographie computationnelle: la puissance des
associations
You shall know a word by the company it keeps (Firth, 1956)
Représentation numérique d’un mot isolé
r(w) ∈ R|V| , r(w)T =(θ1 , . . . θ|V| ) avec θi = f (c(wi , w))
θi ∝ P(w|wi ) ou IM(w, wi ) ou...
dist(w, w0 ) ∝r(w)T r(w0 )
Déduire le comportement d’observations distributionnelles
François Yvon
9 / 34
Lexicographie computationnelle: la puissance des
associations
You shall know a word by the company it keeps (Firth, 1956)
Représentation numérique d’un mot isolé
r(w) ∈ R|V| , r(w)T =(θ1 , . . . θ|V| ) avec θi = f (c(wi , w))
θi ∝ P(w|wi ) ou IM(w, wi ) ou...
dist(w, w0 ) ∝r(w)T r(w0 )
Déduire le comportement d’observations distributionnelles
François Yvon
9 / 34
Let’s go pick some low hanging fruit. Let’s do what we can with
short-distance dependencies (Church, 2011)
François Yvon
10 / 34
Désambiguïser / Classer
Many important natural language inferences can be viewed as problems of resolving
ambiguity, either semantic or syntactic, based on properties of the surrounding context
(Roth, 1998)
npobj
npsuj
det
npobj
ppmod
det
Paul voit un homme avec un parapluie
François Yvon
11 / 34
(Roth, 1998)
npobj
npsuj
det
npobj
ppmod
det
Paul voit un homme avec un téléscope
François Yvon
11 / 34
(Roth, 1998)
npobj
npsuj
det
npobj
ppmod
det
npsuj
det
npobj
det
npobj
ppmod
François Yvon
11 / 34
(Roth, 1998)
npobj
npsuj
det
npobj
ppmod
det
npsuj
det
npobj
det
npobj
ppmod
Adapter les méthodes de la reconnaissance des formes
François Yvon
11 / 34
(Roth, 1998)
Une méthodologie bien rôdée
1
exprimer P comme une décision binaire y ∈ {0, 1}
2
représenter le contexte comme un vecteur x ∈ Rp
3
produire des données annotées D = {(xi , yi ), i = 1 . . . N}
4
apprendre fθ : θ∗ = argmaxθ H(θ, D)
5
calculer un taux d’erreur Ẽ[yi 6= fθ (xi )]
François Yvon
12 / 34
(Roth, 1998)
Désambiguïsateur universel
rattachement prépositionnel : head = V or head = N?
correction "grammaticale” : word = there or word = their ?
désambiguïsation sémantique: bank = B A N K /1 or B A N K /2?
coréférence : (Marie, elle) coréférents ?, il référentiel ?
opinions, sentiments: (text positif ?, utile ?)
implication textuelle: e1 implique e2 ?
etc, etc, etc
François Yvon
12 / 34
(Roth, 1998)
Désambiguïsateur universel
Limites, développements
construction de x
choix de fθ
choix de H
élargissement de codom (f )
François Yvon
12 / 34
Désambiguïser les structures
NLP research can often be distinguished from text retrieval and mining in its focus on
structural representations of text, or of the language the text is written in. (...) The
assumption is that the linguistic structure of the text must be analyzed, inferred, or, in
our terminology, predicted (...) (Smith, 2011).
x= son petit masque la gêne
S
S
NP
Det
son
y1 =
N
Adj
NP
VP
N
petit masque
Pro
V
Det
la
gêne
son
y2 =
VP
N
V
petit masque
NP
Det
N
la
gêne
Probabiliser les modèles de structures discrètes
François Yvon
13 / 34
NLP research can often be distinguished from text retrieval and mining in its focus on
structural representations of text, or of the language the text is written in. (...) The
assumption is that the linguistic structure of the text must be analyzed, inferred, or, in
our terminology, predicted (...) (Smith, 2011).
x= son petit masque la gêne
S
S
NP
Det
son
y1 =
N
Adj
NP
VP
N
petit masque
Pro
V
Det
la
gêne
son
y2 =
VP
N
V
petit masque
NP
Det
N
la
gêne
Probabiliser les modèles de structures discrètes
François Yvon
13 / 34
The assumption is that the linguistic structure of the text must be analyzed, inferred, or,
in our terminology, predicted (Smith, 2011)
Une méthodologie bien rôdée (au XXème siècle)
1
choisir un modèle de génération pour (x, y)
G = (V, T, S, P), avec P = {X
2
→
?
α}.Dérivations: S ⇒y x
probabiliser les dérivations
G = (V, T, S, P), avec P = {p(X
→
α)}.Dérivations: p(x, y) =
Q
y
p(Xi
→
αi )
3
inférer y∗ = argmax Pθ (x, y)
4
5
estimer Pθ (x, y) / θ∗ = argmaxθ `(θ, D)
6
)∗ ]
[changement d’algèbre (Goodman, 1998)]
calculer un taux d’erreur Ẽ[yi 6= (yi
François Yvon
[par décompte]
14 / 34
Probabilisation des structures
séquences: P-FSA
prédiction de mots, modèles de langue
couples de séquences: HMM, P-FST
POS tagging, grapheme-phoneme, correction orthographique, analyse morphologique
arbres: P-CFG, P-LCFG, P-TSG, P-CCG, P-TAG, P-TA, etc.
analyse morphologique, analyse syntaxique en constituants, modèles de langue
dépendances: P-LinkGrammars, P-PDA
analyse syntaxique
couples d’arbres: P-SDTG, P-TIG, P-TT
traduction automatique
etc, etc, etc
François Yvon
14 / 34
Probabilisation des structures
Extensions, limites, développements
étiquetage généralisé: balisage
Isabelle
Tellier
nous
fait
visiter
Vandroeuvre
B-Pers
I-Pers
O
O
O
B-Loc
analyse syntaxique de surface, repérage d’entités
combattre le surapprentissage: lissage
les
I-Loc
Nancy
I-Loc
(Chen & Goodman, 1996)
estimer Pθ à structure cachée: EM pour les CFG
(Lari & Young, 1990)
factorisation de la loi jointe et localité des dépendences
François Yvon
14 / 34
Désambiguïser les structures (alt. take)
in our terminology, predicted(Smith, 2011)
Une méthodologie bien rôdée (au XXIeme siècle)
1
choisir une représentation jointe F pour (x, y)
F : X × Y → Rd ; (x, y) → F(x, y)
2
poser une règle d’inférence: y∗ (x, θ) = argmaxy θT F(x, y)
3
4
choisir une fonction de perte `(y, y0 )
5
estimer θ / θ∗ = argmaxθ ||θ||k + Ẽ[`(y∗ (x, θ), yi )]
6
calculer un taux d’erreur Ẽ[yi 6= (yi )∗ ]
François Yvon
15 / 34
Apprentissage structuré discriminant
séquences:
modèle exponentiel (Rosenfeld, Chen, & Zhu, 2001)
couples de séquences:
CRF (Lafferty, McCallum, & Pereira, 2001; Eisner, 2002),
Perceptron structuré (Collins & Duffy, 2002)
arbres: modèle exponentiel (Rozenknop, 2002; Jousse, Gilleron, Tellier, &
Tommasi, 2006), struct-SVM (Tsochantaridis, Hofmann, Joachims, & Altun, 2004; Taskar,
Klein, Collins, Koller, & Manning, 2004)
graphes de dépendances:
couples d’arbres:
(McDonald, Crammer, & Pereira, 2005)
(Blunsom, Cohn, & Osborne, 2008)
etc, etc, etc
François Yvon
15 / 34
Apprentissage structuré discriminant
Limites, développements
combattre le surapprentissage: régularisation
apprendre avec dérivations cachées
stratégies d’apprentissage et d’optimisation
apprendre avec des pertes structurées
François Yvon
15 / 34
Des fruits moins faciles à atteindre
François Yvon
16 / 34
Apprendre sans supervision
Computational approaches to language processing are almost exclusively supervised,
relying on hand-labeled corpora for training. This reliance is largely due to
unsupervised approaches having repeatedly exhibited discouraging performance. In
particular, the problem of learning syntax (grammar) from completely unannotated text
has received a great deal of attention for well over a decade, with little in the way of
positive results (...) (Klein, 2005)
Apprendre [la grammaire] sans supervision est difficile
inférence non-supervisée de HMM (Merialdo, 1994)
inférence non-supervisée de CFG
Trop de paramètres, pas assez de contraintes
François Yvon
17 / 34
Apprendre sans supervision
Computational approaches to language processing are almost exclusively supervised,
relying on hand-labeled corpora for training. This reliance is largely due to
unsupervised approaches having repeatedly exhibited discouraging performance.
(Klein, 2005)
Apprendre [la grammaire] sans supervision est difficile
Des avancées pratiques, utilisant des représentations “pauvres”
segmenter
I
I
des documents en thèmes: TextTiling (Hearst, 1997)
des mots en “morphèmes” (Goldsmith, 2001)
regrouper
I
I
documents en thèmes (Nigam, McCallum, Thrun, & Mitchell, 2000)
des mots /“morphèmes” en classes syntaxico-sémantiques (Finch &
Chater, 1992; Brown, deSouza, Mercer, Pietra, & Lai, 1992)
reconnaitre
I
I
des collocations, des pré-termes (Dunning, 1993)
des associations bilingues (Gale & Church, 1991)
François Yvon
17 / 34
Apprentissage des représentations
The basic idea is to learn to associate each word in the dictionary with a
continuous-valued vector representation. Each word corresponds to a point in a feature
space. (...) The hope is that functionally similar words get to be closer to each other in
that space, at least along some directions. (...) The advantage of this distributed
representation approach is that it allows the model to generalize well to sequences that
are not in the set of training word sequences, but that are similar in terms of their
features, i.e., their distributed representation. (Bengio, 2008)
Représentations continues
Représenter chaque mot par un vecteur dans Rp de façon que deux
mots proches partagent des propriétés distributionnelles.
La même idée, trois fois
version spectrale (ACM, LSA, et au-delà) (Benzécri, 1975; Dumais et al,
1991)
version probabiliste (PLSA, LDA, H-LDA et au-delà) (Hofmann, 2001;
Blei, Ng, & Jordan, 2002)
version neuronaliste (NNLMs, deep learning et au-delà) (Bengio et al
2003)
François Yvon
18 / 34
Apprentissage des représentations
The basic idea is to learn to associate each word in the dictionary with a
continuous-valued vector representation. Each word corresponds to a point in a feature
space. (...) (Bengio, 2008)
Représentations continues
Représenter chaque mot par un vecteur dans Rp de façon que deux
mots proches partagent des propriétés distributionnelles.
La même idée, trois fois
version spectrale (ACM, LSA, et au-delà) (Benzécri, 1975; Dumais et al,
1991)
version probabiliste (PLSA, LDA, H-LDA et au-delà) (Hofmann, 2001;
Blei et al., 2002)
version neuronaliste (NNLMs, deep learning et au-delà) (Bengio et al
2003)
François Yvon
18 / 34
15 years of picking low hanging fruit has produced a
relatively stable stream of results, and relatively stable
funding. (Church, 2011)
François Yvon
19 / 34
Effectivité de l’appentissage supervisé
Ces réussites en appellent d’autres
Un cadre générique pour l’apprentissage supervisé
des algorithmes pour optimiser, estimer et inférer
des métriques, des pertes, des protocoles, des données annotées
des modules performants aux performances étalonnés
une collaboration bien établie avec les apprentistes
François Yvon
20 / 34
Des “challenges” en veux-tu en voilà
implication textuelle cross-lingue
CLTE addresses textual entailment (TE) recognition under the dimension of
cross-linguality, and within the challenging application scenario of content
synchronization
génération à partir de connaissances
The task for participating teams is to develop systems that map the input
representations provided by the KBGen organisers to sentences
correction grammaticale
A participating system in this shared task is given short English texts written
by non-native speakers of English. The system detects the grammatical errors
present in the input texts, and returns the corrected texts.
François Yvon
20 / 34
Des “challenges” en veux-tu en voilà
Des demandes pressantes ... et solvables
Machine Reading
The Machine Reading program aims to address this issue by replacing expert and
associated knowledge engineers with un-supervised or self-supervised learning systems
that can ”read” natural text and insert it into AI knowledge bases
Text Analytics
Traduction de textes informels
create new techniques for automated translation and linguistic analysis that can be applied
to the informal genres of text and speech common in online and in-person communication
François Yvon
20 / 34
Quelques problèmes non résolus (I)
La loi de Zipf et ses conséquences
Le mur de brique
The formal property is the existence of discontinuities in linguistic material. Such
dependencies occur in morphology (...) to a limited extent, but are omnipresent in
syntax. (...) In such cases, there is normally no upper bound on the amount of
linguistic material that can separate two dependent elements. As a consequence,
any approach that restricts itself to a particular finite window is bound to fail to
identify some proportion of these dependencies. Of course that proportion falls as
the size of the window increases. Unfortunately, attempts to increase the window
size beyond 2 or 3 encounter data sparseness in the shape of a nearly vertical brick
wall known as Zipf law (1935). Zipf law is a well established empirical generalization
about the frequency distribution of words that says that frequency is inversely
proportional to rank. (Gazdar, 1996, p.21)
François Yvon
21 / 34
Le mur de brique
Généraliser mieux ?
de meilleures représentations (cf. supra)
des connaissances / biais linguistiques
François Yvon
21 / 34
Le mur de brique
Généraliser mieux ?
Premiers pas ... difficiles
des a priori sur les paramètres
des contraintes [sur les sorties, sur les distributions]
des connaissances “ approximatives / floues”
De nouvelles fonctions à construire et à optimiser, avec quelles
garanties ?
François Yvon
21 / 34
Quelques problèmes non résolus (II)
Domaine, genre, registre, etc: la diversité des langues
More data =? Better data
In fact, global generalizations are often not accurate at all, because there is no
adequate overall linguistic characterization of the entire language; rather, there are
marked linguistic differences across registers (or sublanguages;cf. Kittredge 1982).
(...) individual linguistic features are distributed differently across registers, and
second, the same (or similar) linguistic features can have different functions in
different registers. (Biber, 1993)
François Yvon
22 / 34
Quelques problèmes non résolus (II)
Domaine, genre, registre, etc: la diversité des langues
More data =? Better data
Mieux adapter ?
sélection / pondération d’instances, de caractéristiques
transfert cross-lingue
Théoriser la constitution des corpus ?
François Yvon
22 / 34
Quelques problèmes non résolus (III)
La définition des unités
Phrasélologie, terminologie, non-compositionalité
although the phraseological nature of language has been thoroughly documented by
corpus studies, there is still a tendency, following hundreds of years of lexicographic
tradition, to think of individual words, rather than phrases, as the basic units of
language. Second, since there are severe problems in defining phrasal units in
corpora, it is difficult to know what to count. (Stubbs, 2002)
François Yvon
23 / 34
Quelques problèmes non résolus (III)
La définition des unités
Phrasélologie, terminologie, non-compositionalité
Mieux segmenter ?
accepter l’ambiguïté de segmentation
exploiter des ressources distributionnelles
optimiser le choix des unités
des représentations optimisées pour les constituants
François Yvon
23 / 34
Angle mort (I): l’acquisition du langage
Des frémissements ?
(...) two emerging areas where I expect the interaction between linguistics and
computational linguistics to become increasingly important: psycholinguistics and
language acquisition. (...) Much of this work can be viewed under the slogan
“structured statistical learning”. That is, specifying the structures over which the
learning algorithm generalises is just as important as specifying the learning
algorithm itself. (...), these computational models become tools for investigating the
effect of specific structural assumptions on the acquisition process.(...) One of the
exciting things about this work is that it permits a quantitative evaluation of the
contribution that specific linguistic representations or constraints might make to the
learning process (Johnson, 11)
François Yvon
24 / 34
Angle mort (II): la linguistique de corpus
Outiller la linguistique expérimentale
Probability models have been responsible for a large share of progress in the field in
the last decade and a half. (Kilgariff, 2005)
François Yvon
25 / 34
L’état du champ et son devenir
un domaine pulvérisé : l’effet de la tâche
researchers have taken a divide and conquer approach and identified several sub-tasks
useful for application development and analysis. These range from the syntactic, such as
part-of-speech tagging, chunking and parsing, to the semantic, such as wordsense
disambiguation, semantic-role labeling, named entity extraction and anaphora resolution.
(Collobert & Weston, 2005)
un corps de doctrine partagé
de plus en plus technique (informatique, mathématiques
appliquées) et computationnellement exigeant
s’éloignant des méthodes / concepts / représentations de la
linguistique
impulsé par les besoins applicatifs
des succès remarqués: le TAL grand public
des défis scientifiques à relever:
I
poursuivre le dialogue avec l’informatique théorique et
l’apprentissage automatique
renouer avec la linguistique expérimentale
I
François Yvon
26 / 34
L’état du champ et son devenir
un domaine pulvérisé : l’effet de la tâche
un corps de doctrine partagé
de plus en plus technique (informatique, mathématiques
appliquées) et computationnellement exigeant
s’éloignant des méthodes / concepts / représentations de la
linguistique
impulsé par les besoins applicatifs
des succès remarqués: le TAL grand public
des défis scientifiques à relever:
I
I
I
poursuivre le dialogue avec l’informatique théorique et
l’apprentissage automatique
renouer avec la linguistique expérimentale
renouer avec les sciences cognitives
François Yvon
26 / 34
Questions ?
François Yvon
27 / 34
Bibliographie I
Biber, D. (1993). Using register-diversified corpora for general
language studies. Computational Linguistics, 19(219–241).
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002). Latent Dirichlet
allocation. In Advances in neural information processing systems
(nips) (Vol. 14, pp. 601–608).
Blunsom, P., Cohn, T., & Osborne, M. (2008). A discriminative latent
variable model for statistical machine translation. In Proceedings
of acl-08: Hlt (pp. 200–208). Columbus, Ohio.
Brown, P. F., deSouza, P. V., Mercer, R. L., Pietra, V. J. D., & Lai, J. C.
(1992). Class-based n-gram models of natural language.
Computational Linguististics, 18(4), 467–479.
Chen, S. F., & Goodman, J. T. (1996). An empirical study of smoothing
techniques for language modeling. In Proceedings of the 34th
annual meeting of the association for computational linguistics
(acl) (pp. 310–318). Santa Cruz, NM.
François Yvon
28 / 34
Bibliographie II
Church, K. W. (2011). A pendulum swung too far. Linguistic Issues in
Language Technology, 6(8).
Church, K. W., & Mercer, R. L. (1993). Introduction to computational
linguistics special issue on large corpora. Computational
Linguistics, 1(19), 1–24.
Collins, M., & Duffy, N. (2002). New ranking algorithms for parsing and
tagging: kernels over discrete structures and the voted
perceptron. In Proceedings of the annual meeting of the
association for computational linguistics (acl) (pp. 489–496).
Philadelphia, PA.
Dunning, T. (1993). Accurate models for the statistics of surprise and
coincidence. Computational Linguistics, 19(1), 61–74.
Eisner, J. (2002). Parameter estimation for probabilistic finite-state
transducers. In Proceedings of the 40th annual meeting of the
association for computational linguistics (pp. 1–8). Philadelphia,
Pennsylvania, USA.
François Yvon
29 / 34
Bibliographie III
Finch, S., & Chater, N. (1992). Bootstrapping syntactic categories. In
Proceedings of the 14th annual meeting of the cognitive science
society (pp. 820–825).
Gale, W. A., & Church, K. W. (1991). Identifying word correspondence
in parallel texts. In Proceedings of the workshop on speech and
natural language (pp. 152–157). Morristown, NJ, USA:
Association for Computational Linguistics.
Gazdar, G. (1996). Computing tomorrow. In I. Wand & R. Milner
(Eds.), (pp. 88–109). New York, NY, USA: Cambridge University
Press. Retrieved from
http://dl.acm.org/citation.cfm?id=242807.242813
Goldsmith, J. (2001). Unsupervised learning of the morphology of
natural languages. Computational Linguistics, 27(2), 153–198.
Goodman, J. (1998). Parsing inside-outside. Unpublished doctoral
dissertation, Division of Engineering and Applied Sciences,
Harvard University.
François Yvon
30 / 34
Bibliographie IV
Hall, D., Jurafsky, D., & Manning, C. D. (2008, October). Studying the
history of ideas using topic models. In Proceedings of the 2008
conference on empirical methods in natural language processing
(pp. 363–371). Honolulu, Hawaii: Association for Computational
Linguistics.
Hearst, M. (1997). TextTiling: Segmenting texts into multi-paragraph
subtopic passages. Computational Linguistics, 23(1), 33–64.
Hofmann, T. (2001). Unsupervised learning by probabilistic latent
semantic analysis. Machine Learning Journal, 42(1), 177–196.
Jousse, F., Gilleron, R., Tellier, I., & Tommasi, M. (2006). Conditional
random fields for xml trees. In Proceedings of the ecml workshop
on mining and learning in graphs.
Klein, D. (2005). The unsupervised learning of natural language
structures. Unpublished doctoral dissertation, Stanford
University.
François Yvon
31 / 34
Bibliographie V
Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random
fields: Probabilistic models for segmenting and labeling
sequence data. In Proceedings of the 18th international
conference on machine learning (icml) (pp. 282–289). Morgan
Kaufmann, San Francisco, CA.
Lari, K., & Young, S. J. (1990). The Estimation of Stochastic
Context-Free Grammars using the Inside-Outside Algorithm.
Computer Speech & Language, 4, 35–56.
McDonald, R., Crammer, K., & Pereira, F. (2005, June). Online
large-margin training of dependency parsers. In Proceedings of
the 43rd annual meeting of the association for computational
linguistics (acl’05) (pp. 91–98). Ann Arbor, Michigan: Association
for Computational Linguistics.
Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. M. (2000). Text
classification from labeled and unlabeled documents using EM.
Machine Learning, 39(2/3), 103–134.
François Yvon
32 / 34
Bibliographie VI
Rosenfeld, R., Chen, S. F., & Zhu, X. (2001). Whole-sentence
exponential language models: a vehicle for linguistic-statistical
integration. Computer Speech and Language, 15, 55–73.
Roth, D. (1998). Learning to resolve natural language ambiguities: a
unified approach. In Proceedings of the annual meeting of the
american association for artificial intelligence (aaai) (pp.
806–813). Madison, WI.
Rozenknop, A. (2002). Modèles syntaxiques probabilistes
non-gènératifs. Unpublished doctoral dissertation, Dpt.
d’informatique, Ecole Polytechnique Fédérale de Lausanne.
Smith, N. A. (2011). Linguistic structure prediction. Morgan and
Claypool.
Stubbs, M. (2002). Two quantitativemethods of studying phraseology
in english. International Journal of Corpus Linguistics.
François Yvon
33 / 34
Bibliographie VII
Taskar, B., Klein, D., Collins, M., Koller, D., & Manning, C. (2004).
Max-margin parsing. In Proceedings of the conference on
empirical methods in natural language processing (emnlp04).
Barcelona, Spain.
Tsochantaridis, I., Hofmann, T., Joachims, T., & Altun, Y. (2004).
Support vector machine learning for interdependent and
structured output spaces. In International conference on
machine learning (icml).
François Yvon
34 / 34

Le TAL statistique

Transcription

Documents pareils

Roger LE ROUX Directeur EPCC CIRQUE

DION Paul - Agence artistique Chantal David

Une STAR en campagne - Association Vie Val d`I

Yvon Mau lance une gamme de vins de pays pour le Canada VSB

Canada : Yvon Mau lance une gamme de vins de pays en bouteilles

Lettre officielle

ski de randonnée dans le vereinatal – loin de la

Yvon Mau fait croître ses grands crus RVI, Février 2010

DPL - Distributions de plaques automobiles - DPL

Télégramme 2014