Presentation - University of Memphis Blogs

O N L I N E T R A N S L AT I O N U S A G E
A M O N G I N T E R M E D I AT E F R E N C H
STUDENTS
W H AT A R E O N L I N E T R A N S L AT O R S ? • Free, web-based sites (Bing Translator, Free Translation, Google Translate...)
• Convert text, from single-word to multi-
paragraph, from one language to another.
• Generally don’t offer definitions or usage notes,
Errol M. O’Neill, Ph.D.
University of Memphis
[email protected]
AATF — 21 July 2014
O N L I N E T R A N S L AT O R S — ALMOST 40 YEARS
First "online" translator on French
Minitel in 1986 (Gachot, ancestor of
Babel Fish)
unlike dictionaries
• Over 500 million words translated by online
translation (OT) on average each day (Koehn 2010).
EXAMPLE OF A POPULAR OT SITE.
FreeTranslation.com
in 2014
O T: I N T E N D E D A N D A C T U A L U S E T R A N S L AT I O N G A F F E S ( S U S P E C T E D O T U S E ) • Online translators were designed as a demo of paid software solutions (e.g. Babel Fish for SYSTRAN’s Trados).
• They were not meant for language learners, although
early on this become one of the main groups of users
(Yates 1998).
• It’s also not meant to offer a professional, polished
translation suitable for business use in the TL but can
often provide a gist of the topic.
T R A N S L AT I O N G A F F E S ( S U S P E C T E D O T U S E ) OT ACCURACY
• Conflicting reports of accuracy and comprehensibility.
• Yates (2006) — 15 out of 20 OTs for law texts from Spanish and
German to English considered failed.
• Ablanedo et al. (2007) — All 10 texts translated from Spanish by
Babel Fish rated usable & understandable
• Aiken and Balan (2011) — OT of 6 sentences between 51
languages, 2,550 language pairs.
• BLEU scores varied from 0 (completely unintelligible translations)
to 100 (fully accurate). French <-> English among the highest,
ranked at 91 out of 100.
• Systematic study would be needed per OT site & language pair.
OT AND LANGUAGE LEARNING
• Online translation (OT) is generally prohibited for
students doing academic work. Ethical question: who
is doing the work, the student, the online translator, or
somewhere in between?
• White and Heidrich (2013) found that 12 out of 18 students
in their study reported using web-based machine translation
on assignments in spite of it being expressly forbidden.
• Anecdotal reports and an idée reçue that OT is harmful, but no
studies found demonstrating L2 writers systematically do
worse with OT than they would on their own.
RED FLAGS?
• Presence of English words (particularly misspelled) left in the
text without explanations.
• Le samedi je vais usuallly à la plage.
• Saturdays I usually go to the beach. (Luton 2003)
• Students though also include English words
• J’ai pris un cours en l’accountancy
• I took a course in accountancy (student leaving in L1
unknown vocabulary — O'Neill 2013)
• Floirda
• Florida → Floride (typo — O'Neill 2013)
RED FLAGS?
• Lisette Luton (2003) suggests ‘‘red flags’’ for OT use in L2 writing
• Compositions containing mistranslated idioms, expressions, &
colloquial terms
• J'obtiens le renversement.
• I get upset. (Luton 2003)
• However, students can also misuse vocabulary or structures on
their own.
• Je gauche
• I left (misuse of dictionary: Hurman and Tull 2002)
RED FLAGS?
• Literal translation of proper nouns. This type of error most
closely approaches a "red flag" and seems unlikely to be
student-produced.
• Brun de John
• John Brown (Luton 2003)
• Monseigneur Paul
• Paul Bishop (personal communication 2011)
• Additional clues for translation use may exist (e.g. font
changes) but not always be easily detectable or certain.
• Unclear cases might be called "yellow cards" (O'Neill 2013):
warnings of possible use
O N L I N E T R A N S L AT I O N S T U D Y
(O'NEILL 2013)
METHODOLOGY
• Study of third- and fourth- semester university
students in French, n = 32
• Two goals:
• Each student performed two tasks (compositions),
and each was scored & rated for OT use by two
instructors.
• to find out if compositions written with OT would
receive higher or lower scores than those without
a translator
• Three treatment groups
• whether or not teachers can identify OT use to
• Group A: No translator access, no training
inform pedagogical practices.
• Group B: Translator access, no training
• Group C: Translator access and training in OT use
E X C E R P T: G R O U P C ' S T R A I N I N G S E S S I O N C) Foreign-language judgments
1) As a non-native speaker of French, you may be able to notice some mistakes or be
able to understand even if there is a mistake, but harder than in your native language.
2) Go to http://freetranslation.com
3) Set translator to “English to French”
4) Enter What a beautiful car!
5) What is the result you get in French? Do you see any mistakes?
!
__________________________________________________________________
6) “What” at the beginning of a sentence often indicates a question (“What are you
doing?” could use “que” / “qu’est-ce que”), the translator didn’t get that this is an exclamation
(context).
7) If your teacher got this sentence, do you think he/she would understand it?
!
8) Go to http://babelfish.yahoo.com/
9) Set translator to “English to French”
10) Enter What a beautiful car!
11) What is the result you get in French? Do you see any mistakes?
!
__________________________________________________________________
12) Babel Fish translated the right idea in a grammatically-correct, colloquial manner.
Online translators are not always wrong.
!
!
!
Qu'une belle voiture!
Quelle belle voiture!
E X C E R P T: G R O U P C ' S T R A I N I N G S E S S I O N !
Online Translation
Some possible strengths and weaknesses
!
A. What online translators often can do.
1. Translate words that have one meaning in isolation
(for example, the television=la télévision)
!
2. Translate some words in context that have a small number of common, clear/
distinct meanings
(for example: I saw the man=J’ai vu l’homme)
!
3. Translate some common, straightforward expressions correctly or with the right
general idea
(for example, What a bummer!=Quelle déception!)
!
4. Make basic agreement between nouns and nearby adjectives and verbs
(for example, She went=elle est allée We’re tired=Nous sommes fatigués)
!
5. Translate tenses correctly when they’re the same in French and English
(for example, I like the Internet=j’aime l’Internet)
!
6. Give a general idea or “gist” of the meaning of a sentence or longer passage
(O'Neill 2013)
E X C E R P T: G R O U P C ' S T R A I N I N G S E S S I O N B. What online translators often cannot do or what they have problems doing.
1. Translate words that have a lot of meanings or uses
(for example, set=ensemble in only some cases)
!
2. Translate many specialized or technical words in context
(for example: the escape key on a keyboard is not la clef d’évasion)
!
3. Translate most expressions and phrasal verbs.
(for example: She wrote it down. is not Elle l'a écrit avalent. What a drag! is not
Quelle drague!)
!
4. Make correct agreement between nouns/adjectives and subjects/verbs, especially if
they aren’t nearby
(for example: She was, as you know, very intelligent. should have fem. adjective
agreement in French)
!
5. Translate tenses when the tenses are different between French and English
(for example: I’ve been doing...since)
!
6. Recognize misspelled or misused words
(for example: their vs. they’re vs. there vs. thier)
!
7. Deal with proper nouns (people’s names, place names, etc.) effectively
(for example: Bill’s house ≠ la maison de la facture)
!
8. Give an accurate, polished translation
R E S U LT S : S C O R E S • On the 1st composition, translator Groups B and C’s
overall scores were higher than control group's, but not
significantly different.
• However, on the 2nd composition Two, Group C
(translator training & access) scored significantly higher
than the control group — by nearly 5 points out of 30 (F (2, 29) = 7.05, p = 0.0032).
• Take-away: OT-aided compositions not judged to be worse (& possibly better) than ones without OT use.
(O'Neill 2013)
TA S K T W O O V E R A L L S C O R E S ( O U T O F 3 0 ) W H AT M I G H T E X P L A I N T H E R E S U LT S ? • Looking at subscores for features
• OT participants scored better on comprehensibility,
grammar, content: Intermediate students using OT might
know enough to throw out any translations that don’t
make sense.
• OT students did better on spelling: unless programmers
were careless, unlikely there’d be many misspelled
words in OT output.
GROUP
A
B
C
M E A N ( X̅ )
20.650**
23.681
**Difference significant at p=0.01 level
25.364**
Group A: no translator access or training (-TA -TT)
Group B: translator access, but no training (+TA -TT) Group C: translator access and prior training +TA +TT
• There was no difference on vocabulary or syntax: word choice and order difficult both for learners and the OT (context, complex structures)
R E S U LT S : D E T E C T I N G O T EXAMPLES FROM STUDY
• “j’élève de l’argent”
• Instructors overall were able to identify correctly
whether a translator was used.
• 70.7% success rate: 181 correct judgments out of 256.
(Significant correlation (χ2 (1) = 28.818, p < 0.01)
• However, this means instructors were wrong nearly
30% of the time.
• 33 compositions wrongly suspected of OT use.
• OT use went undetected in 42 compositions.
EXAMPLES FROM STUDY
• “la décision est jusqu’à tu”
• the decision is up to you (OT: incorrect use of prepositional phrase)
• “Je soutiate tu make le bon chiox”
• I hope you make the right choice (non-OT: spelling, missing conjunction, English word)
• “ma grand-mère maison”
• My grandmother's house (OT: incorrect possessive construction/syntax)
• “Parce que, moi (boss) était sympa”
• Because, my boss was nice (not OT: incorrect possessive adjective, English word inserted)
• I raise money (OT: lexical error)
• “chronométrer consommer”
• time-consuming (OT: grammatical category, lexical
errors)
• “J’amour le théâtre”
• I love the theatre (not OT: grammatical category)
• “Mon université je bon”
• My university is good (OT: typo in input: 'i' for 'is')
SAMPLE COMPOSITION: OT? 8576
Bonjour Marie-Claire,
• Vocabulary (VO) — Are the words that have been chosen accurate and effective in standard or colloquial written French? (This does not
L'université
d'illinois est mieux que les autres écoles parce qu'il a des gens amicaux.
include spelling, accents, or grammatical usage.)
Aussi, l'université a de grands universitaires et un beau campus. L'université coûte cher
!
et il sent comme poop de vache quelquefois. Cependant, l'université est un grand lieu
5 Yes, all or very nearly all of the words used are accurate and effective
pour apprendre et rencontrer de nouveaux gens.
4 Yes, the vast majority of the words used are accurate and effective
!
3 About half of the words used are accurate and effective
Durer
le semestre, j'ai fait beaucoup de choses amusantes à l'école. J'ai joué le
2 No, the vast majority of the words used are not accurate and effective
basketball
de tous les jours et j'ai assisté beaucoup de jeux de football. Je suis allé aux
1 No, all oret
very
nearly
all of the
words
are not accurate
effective
barres
j'ai
joint
une
fraternité.
Jeandn'ai
pas assisté la classe mais j'ai obtenu toujours de
!
bons
degrés. Il y a un nouveau gymnase sur le campus et la mare est stupéfiant. Je vais
nager
là tout
le temps.
2) After noting
the score
for each category, add up the scores and note the Total score (out of 30).
!!
Marie-Claire,
c'est
important
à l'université
d'Illinois
vous pouvez recevoir
3) Do you believe this
composition
was writtenvous
with thevenez
aid of a translator?
Write "Yes" or
"No" on the si
composition.
une
! bonne éducation. Le campus a beaucoup d'argent est investi dans lui et tous les
nouveaux bâtiments sont de pointe. Le campus est plein d'occasions passionnantes et
4) Please briefly explain below the composition (or on the back if you prefer / if more space is needed) your response to number 3 by citing a
specific
reason(s)
or example(s)
of errors
present or lacking
from the paper.
Please be
sure you're
clear about what Marie-Claire,
you mean.
de
grands
gens.
C'est
un hasard
merveilleux
pour
venir
à l'université.
venir
à l'université d'Illinois, vous ne le regretterez pas !
!
7484
Bonjour
S A M Jean-Pierre,
PLE COMPOSITION: J’espère que tout est bon. Aussi, j’espère a devenir un ingénieur un jour.
Dans ingénierie, on gagner plus d’argent. Dans ingénierie, on peut faire
une
différence
monde.
Ingénierie
est
bonne,
mais
le travail
est
• Vocabulary
(VO) — Are thedans
words thatle
have
been chosen accurate
and effective in
standard
or colloquial
written French?
(This does not
includedifficile.
spelling, accents,
or grammatical
usage.)
très
Aussi,
l’argent
est les moins important, si ce n’est pas la
!
meilleure
carrière. ! 5 Yes, all or very nearly all of the words used are accurate and effective
J’ai
prépare
par
beaucoup de travail a l’école
4 Yes, the
vast majority ofpour
the wordsingénierie
used are accurate and
effective
secondaire.
Aussi, j’ai travaillé à une boulangerie pour deux ans.
3 About half of the words used are accurate and effective
L’expérience
m’appris à travailler beaucoup. Et j’ai gagné beaucoup
2 No, the vast majority of the words used are not accurate and effective
d’argent
pour
l’université. A mon avis, vous devez obtenir un
1 No, all or very nearly all of the words are not accurate and effective
emploi
avant de collège. !
!
2) After noting the score for each category, add up the scores and note the Total score (out of 30).
C’est
important à devenir un ingénieur parce qu’il y a beaucoup
!
de
problèmes
dans le monde. L’ingénierie aide les problèmes.
3) Do you believe this composition was written with the aid of a translator? Write "Yes" or "No" on the composition.
L’Amérique
a des bonnes écoles. Vous devez venir à l’école à
!
Amérique.
4) Please briefly explain below the composition (or on the back if you prefer / if more space is needed) your response to number 3 by citing a
specific reason(s) or example(s) of errors present or lacking from the paper. Please be sure you're clear about what you mean.
Au revoir, 7484
Participant 6322
!
SAMPLE COMPOSITION: OT?
7420
Bonjour Jean Pierre! Mon emploi est très bien! Je depense
• Vocabulary (VO) — Are the words that have been chosen accurate and effective in standard or colloquial written French? (This does not
include spelling, accents, or grammatical usage.)
mon argent tout le jour! Les personnes sont très sociable!
!
Mais,
le travaille est difficile. Les houres sont longues.
5 Yes, all or very nearly all of the words used are accurate and effective
! 4 Yes, the vast majority of the words used are accurate and effective
Je travaille à une banque. J’ai etudiè lecommerce à
l’universitie d’Illinois. J’ai intèrresè en le commerce en l’ecole
1 No, all or very nearly all of the words are not accurate and effective
secondaire.
J’ai pris un cours en l’accountancy et decouvrir le
!
sujet.
Vous
devrais
si30).vous étés intèresser.
2) After noting the score for each category,étudier
add up the scoresle
and commerce
note the Total score (out of
3 About half of the words used are accurate and effective
2 No, the vast majority of the words used are not accurate and effective
!
3) Do you believe this composition was written with the aid of a translator? Write "Yes" or "No" on the composition.
Le
commerce est très flexible. On peux travailler en tous les
!
emplois.
Mais, l’èconomie en l’États-Unis est mal. Vous
4) Please briefly explain below the composition (or on the back if you prefer / if more space is needed) your response to number 3 by citing a
specific reason(s) or example(s) of errors present or lacking from the paper. Please be sure you're clear about what you mean.
devrais chercher un emploi en France si vous veux l’argent!
W H AT T O D O A B O U T O T ?
Bonjour Bernard,
!
Une vacances dans Amérique pendant l'hiver peuvent
s'intéresser très. Vous jouerez dans la neige. Nous pouvons faire
un homme de neige. Il peut ennuyer aussi. Il sera très froid.
L'an dernier, l'hiver était très bon. Nous avons beaucoup de
neige. La classe été annulé. Nous avons voyagé dans un sleigh.
Une vacances longues d'hiver sont magnifque. Vous l'aimerez ici.
Vous aurez un bon temps. Vous vous amuserez bien.
!
!
• Recommendations in the literature vary
• McCarthy (2004) suggests imposing sanctions or
grading translated essays “on their own merits.”
Difficult to justify grading suspected OT cases lower:
maybe be incorrectly penalizing some or not
catching others using a translator
With availability of the technology and difficulty
in identifying it, increasingly hard to prohibit OT.
W H AT T O D O A B O U T O T ?
W H AT T O D O A B O U T O T ?
!
• Niño (2009, 2011) suggests translator use in class
• Help students with understanding more advanced
texts written in the L2
• Discuss and compare linguistic features (syntax,
polysemy…), for example through using parallel
corpora.
• For White and Heidrich (2013), OT increases
linguistic awareness.
• “Instead of seeking to detect and punish
students who are using this resource […] it
would be more fruitful to train students how to
use WBMT as a support for learning.”
• How, when, whether to use it may depend on your
institution, level, students, other factors.
BIBLIOGRAPHY
Ablanedo, J., Aiken, M., & Vanjani, M. (2007). Efficacy of English to Spanish
automatic translation. International Journal of Information and Operations
Management Education, 2, 194-210.
MERCI À TOUTES ET À TOUS !
Or as FreeTranslation says:
"Thank you to all and to all!"
Errol M. O’Neill, Ph.D.
University of Memphis
[email protected]
Aiken, M., & Balan, S. (2011). An analysis of Google Translate accuracy.
Translation Journal,16 (2). Available from http://translationjournal.net/journal/
56google.htm.
Hurman, J. and Tall, G. (2002). Quantitative and qualitative effects of
dictionary use on written examination scores. Language Learning Journal, 25,
21-26.
Koehn, P. (2009). A Process Study of Computed Aided Translation. Machine
Translation, 23.
— (2010). Statistical Machine Translation. New York: Cambridge University
Press.
Luton, L. (2003). If the computer did my homework, how come I didn't get an
“A”? French Review, 76(4), 766-770.
BIBLIOGRAPHY
SOURCE DES IMAGES
McCarthy, B. (2004). Does online machine translation spell the end of
take-home translation assignments? CALL-EJ online, 6.1. Available from
http://www.clec.ritsumei.ac.jp/english/callejonline/9-1/mccarthy.html
•
Slide 1: http://dculs.dcu.ie
•
Slide 2: http://en.wikipedia.org/wiki/File:Dictionary_through_lens.JPG Author: Booksworm
Niño, A. (2009) Machine translation in foreign language learning:
language learners’ and tu- tors’ perceptions of its advantages and
disadvantages. ReCALL, 21(2), 241-258.
•
Slide 3: http://www.futura-sciences.com/fr/doc/t/informatique-2/d/traduction-automatique-les-annees-outout-a-change_831/c3/221/p2
•
Slide 4: http://www.freetranslation.com
•
Slide 5 : http://www.userlogos.org/node/2316
•
Slides 6-7: http://dailygeekshow.com/2013/07/18/16-produits-victimes-dune-traduction-francaise-totalementhilarante/, http://www.atanet.org/client_outreach/translation_bloopers.php
•
Slide 21: http://pixabay.com/fr/icône-verre-grossissant-33170
•
Slide 28-30: http://www.bbc.co.uk/schools/gcsebitesize/maths/geometry/parallellinesrev1.shtml
•
Remaining images: Clipart and/or public domain
O’Neill, E. (2013). Online Translator Usage in Foreign Language Writing.
Dimension 2013, 74-88.
White, K. D. and Heidrich, E. (2013). Our Policies, Their Text: German
Language Students’ Strategies with and Beliefs about Web-Based Machine
Translation. Die Unterrichtspraxis, 46.2, 230-250.
Williams, L. (2006). Web-based machine translation: A tool for promoting
electronic literacy and language awareness. Foreign Language Annals 39,
565-578.