docUEdin: Translating L1 Phrases in L2 Context using Context
download 
Plainte Transcription
UEdin: Translating L1 Phrases in L2 Context using Context
UEdin: Translating L1 Phrases in L2 Context using Context-Sensitive SMT Eva Hasler School of Informatics University of Edinburgh SemEval 2014 Introduction Model description Experimental Results Conclusions SemEval Task 5: L2 writing assistant Language pairs • English-Spanish, French-English, Dutch-English Proposed system • Phrase-based SMT: L1 → L2 • Language model scoring of translated fragments in target context • Context similarity feature adapts to topic of each context • No explicit modelling of grammatical correctness 2/16 Introduction Model description Experimental Results Conclusions Language model scoring of L2 context • Provide target context as identity translation of source text around L1 fragment 3/16 Introduction Model description Experimental Results Conclusions Language model scoring of L2 context • Provide target context as identity translation of source text around L1 fragment Example test instance (L1: French, L2: English) <f>les manifesteurs</f> want to replace the government. Input to MT system <wall/> les manifesteurs <wall/> <t translation=”want to replace the government.”> want to replace the government. </t> <wall/> 3/16 Introduction Model description Experimental Results Conclusions Context similarity feature • Resolve lexical ambiguities using the topical context • Derived from the Phrase Pair Topic Model [Hasler et al., 2014] • Model learns semantic representations for all phrase pairs in the phrase table, using a variant of Latent Dirichlet Allocation 4/16 Introduction Model description Experimental Results Conclusions Phrase Pair Topic Model Learning semantic representations • Represent each phrase pair as a pseudo document containing context words • Standard MT: source context (French) • Here: target context (English) Train document 2 Le noyau d’un système d’exploitation est lui-même un logiciel, mais ne peut cependant utiliser tous les Le noyau d’un système mécanismes d’abstraction est lui-même qu’il fournit auxd’exploitation autres un logiciel, logiciels. Son rôle central mais ne peut cependant impose par ailleurs des utiliser tous les mécanismes performances élevées. Cela d’abstraction fournit aux autres fait du noyau laqu’il partie la plus logiciels. Son rôle central critique d’un système impose par ailleurs des d’exploitation et rend sa performances élevées. Cela Le noyau d’un systèmeet sa conception ... fait du noyau la partie la plus d’exploitation est lui-même un logiciel, mais ne peut critique d’un système cependant utiliser tous lesd’exploitation et rend sa mécanismes d’abstractionconception et sa ... Train document 3 Train document 1 qu’il fournit aux autres logiciels. Le rôle du noyau central impose par ailleurs des performances élevées. Cela fait du noyau la partie la plus critique d’un système d’exploitation et rend sa conception et sa ... noyau → kernel Le noyau atomique désigne la région située au centre Le noyau atomique désigne nucléons). La taille du noyau la région située au centre (10-15 mètre) est environ d'un atome constituée100 de 000 fois plus petite que protons et de neutrons (lesde l'atome et concentre celle nucléons). La taille duquasiment noyau toute sa masse. (10-15 mètre) est environ Les forces nucléaires qui 100 000 fois plus petite que s'exercent entre les nucléons celle de l'atome et concentre sont à peu près un million quasiment toute sa masse. de fois plus grandes. Les forces nucléaires qui s'exercent entre les nucléons sont à peu près un million de fois plus grandes. noyau → nucleus cellule version défaut Train document 5 d'un atome constituée de Train document 4 protons et de neutrons (les linux recompiler fonctionnel actuel appliquer atomique microscopique matière élémentaires électron correctif 5/16 Introduction Model description Experimental Results Conclusions Phrase Pair Topic Model Learning semantic representations • Represent each phrase pair as a pseudo document containing context words • Standard MT: source context (French) • Here: target context (English) noyau → kernel default noyau → nucleus cell version linux atomic microscopic recompile functional current mass elementary electron corrective noyau → kernel noyau → nucleus θ θ p i p j 5/16 Introduction Model description Experimental Results Conclusions Phrase Pair Topic Model Context similarity feature noyau → kernel version default test context linux recompile noyau → nucleus θp linux cosine(θp,θ) l θl θp cosine(θp,θ)l error module message cell atomic microscopic • compute for each test sentence and phrase pairs 6/16 Introduction Model description Experimental Results Conclusions Variations of Context Similarity Feature • 50-topics Cosine similarity according to model with 50 topics • mixture:geoAvg Geometric average of cosine similarities of models with 20, 50 and 100 topics • mixture:max Choose model (20, 50, 100 topics) that yields lowest entropy of distribution over translations 7/16 Introduction Model description Experimental Results Conclusions Training Data Training data Europarl News Commentary Ted News Commoncrawl En-Es 1.92M 192K 157K 2.1G 50M Fr-En 1.96M 181K 159K 2.1G 82M Nl-En 1.95M n/a 145K 2.1G - Phrase-based baselines (trained with Moses) • Standard feature set + 5-gram LM + context similarity feature 8/16 Introduction Model description Experimental Results Conclusions Simulating Ambiguous Development Data • Trial data contained only few ambiguous examples • Extracted 1076 development instances of 14 ambiguous French words and English translations (mixed corpus of NewsCom, Ted, Commoncrawl) Source words chaı̂ne matière flux Translations chain, string, channel, station matter, material, subject stream, flow, feed 9/16 Introduction Model description Experimental Results Conclusions Simulating Ambiguous Development Data Without LM scoring of target context System Baseline + 50-topics + mixture:geoAvg + mixture:max French-English 0.314 *0.674 *0.670 *0.690 • Score: Average word accuracy 10/16 Introduction Model description Experimental Results Conclusions Simulating Ambiguous Development Data With LM scoring of target context System Baseline + LM context + 50-topics + mixture:geoAvg + mixture:max French-English 0.314 0.726 *0.886 *0.883 *0.889 • Score: Average word accuracy 11/16 Introduction Model description Experimental Results Conclusions Results on Official Test Sets Without LM scoring of target context System Baseline +50-topics +geoAvg +mixture:max English-Spanish best oof 0.674 0.854 0.682 0.860 0.677 *0.863 0.679 0.860 French-English best oof 0.722 0.884 0.719 *0.896 0.715 *0.896 0.712 0.887 Dutch-English best oof 0.613 0.750 0.616 *0.759 0.619 *0.756 0.618 0.753 • Good baseline accuracy • In most cases, one of the systems including the context similarity feature performs best 12/16 Introduction Model description Experimental Results Conclusions Results on Official Test Sets With LM scoring of target context (submitted runs) System Baseline + LM context +50-topics +geoAvg +max English-Spanish best oof 0.674 0.854 0.839 0.944 0.827 0.946 0.827 0.944 0.820 0.949 French-English best oof 0.722 0.884 0.823 0.934 0.824 0.938 0.821 *0.939 0.816 0.937 Dutch-English best oof 0.613 0.750 0.686 0.809 0.692 0.811 0.688 0.808 0.688 0.808 • LM scoring always improves the baseline • Small gains with context similarity feature (but mostly not significant) 13/16 Introduction Model description Experimental Results Conclusions Results on Official Test Sets Weaker performance of topic feature • Topic model only helps with lexical ambiguity • Not all test examples ambiguous without L2 context Example translations Input: Baseline: 50-topics: Reference: There are many ways of cooking <f>des œufs</f> for breakfast. .. <f>eggs</f> .. .. <f>eggs</f> .. .. <f>eggs</f> .. 14/16 Introduction Model description Experimental Results Conclusions Results on Official Test Sets Weaker performance of topic feature • Topic model only helps with lexical ambiguity • Not all test examples ambiguous without L2 context Example translations Input: Baseline: 50-topics: Reference: This project represents one of the rare advances in strenghtening <f>les liens</f> between Brazil and the European Union. .. <f>the links</f> .. .. <f>the ties</f> .. .. <f>the ties||relations||the bonds</f> .. 14/16 Introduction Model description Experimental Results Conclusions Conclusions L2 writing assistant task • Phrase-based SMT + LM scoring + context similarity feature • LM scoring always improves word accuracy • Topic feature mostly effective for ambiguous source phrases 15/16 Introduction Model description Experimental Results Conclusions Thank you! 16/16 Introduction Model description Experimental Results Conclusions Hasler, E., Haddow, B., and Koehn, P. (2014). Dynamic Topic Adaptation for SMT using Distributional Profiles. In Proceedings of the 9th Workshop on Statistical Machine Translation. 16/16
