Would You Explain the Instructions? Ontario Teachers` Responses

Would You Explain the
Instructions?
Ontario Teachers’ Responses
to Common Testing Dilemmas
Ruth A. Childs, Saad Chahine, Monique Herbert,
Linda Umezawa, & Yunmei Xu
Ontario Institute for Studies in Education of the University of
Toronto
Presented at the workshop titled: “Large-scale assessments of achievement and the issue of official languages”
This Study
We are studying the test preparation and
administration practices of Grade 3 teachers in
Ontario for the Assessment of Reading, Writing,
and Mathematics, Primary Division
• Phase 1: Interviews with Grade 3 teachers
and elementary principals in the Toronto area
(8 teachers and 3 principals)
• Phase 2: On-line survey (in English and
French) of elementary teachers across
Ontario – Preliminary Results
So far…
In Phase 1, we found:
• Sometimes, what teachers are asked to do
as test administrators conflicts with what they
believe they should do as teachers
• Teachers resolve these conflicts in many
different ways
In Phase 2, we are asking:
• What do teachers believe they should do
when administering the tests?
• Is what teachers say they would do related to
their beliefs?
Beliefs
All teachers should administer Teachers should use
their professional
the test in "strict adherence
judgement in deciding
to the policies, procedures,
how to administer the
and instructions," as
test. /
instructed in the administration
guide. /
Le personnel
enseignant devrait
Tout le personnel enseignant
utiliser leur jugement
devraient administrer le test
professionnel lors de
en suivant strictement les
la prise de décision
politiques, les procédures et
concernant
les directives telles que
l’administration du
décrites dans le guide
test.
d’administration.
Teachers
in
Englishlanguage
Schools
N = 33
Teachers
in
Frenchlanguage
Schools
N = 17
Agree / En accord
Disagree /
En désaccord
58%
18%
Agree / En accord
Agree / En accord
9%
47%
Disagree / En désaccord
Agree / En accord
21%
12%
12%
24%
Unsure or Missing
Dilemma #1
You work in a school where, when the boxes of tests
arrive, the Grade 3 teachers get together and review
the test. A question on the test asks students to
estimate the volume of a box. You were planning to
teach about volume after the test. There's a week
before you will be administering the test.
What would you do?
Vous travaillez dans une école où, lorsque les boîtes
de tests sont livrées, les enseignants et les
enseignantes de 3e année se réunissent et font la
révision du test. Une des questions du test demande
aux élèves d’estimer le volume d’une boîte. Vous
aviez planifié enseigner le volume après le test. Il y a
encore une semaine avant l’administration du test.
Que feriez-vous?
Responses to Dilemma #1
Strict Adherence
“Do a quick unit on volume.” (more than 50%)
“On n'est pas supposé ouvrir les boites avant
l'administration du test.” (about 25%)
“Ignore it. It is one question.”
Professional Judgement
“Teach volume.” or “Make classroom references to
meaning of volume.” (more than 80%)
“That wouldn’t happen because we get through the whole
math program before the test and then spend June doing
all the fun things we missed – cooking, handwriting, etc.”
Dilemma #2
During the test, you walk around the classroom.
You notice that one of the students is answering
questions with only a word or two. The
instructions say students should explain their
reasoning.
What would you do?
Pendant le test, vous circulez dans la classe.
Vous remarquez qu’un des élèves répond aux
questions avec un ou deux mots seulement. Les
directives demandent aux élèves d’expliquer
leur raisonnement.
Que feriez-vous?
Responses to Dilemma #2
Strict Adherence
“Point to the instructions that say to explain.” (about 35%)
“I would announce to the entire class to remember to read
instructions carefully and try to use the entire space
provided.” (about 15%)
“Je lui demanderais si son travail avance bien, s'il a des
questions et s'il comprend bien les directives.”
“I would ask the child if they are doing their best and
encourage the child to continue with a pat on the back.”
“Again, students must complete the assessment
independently. As difficult as it is, we have to remove the
teacher hat for a little while and wear the hat of a test
administrator.”
Responses to Dilemma #2
(continued)
Strict Adherence (continued)
“Ignore it. It is too late now and if I talk often all students
take it to heart and become nervous they are doing
something wrong. I would remind students before the next
booklet starts to always put all your thinking down.”
Professional Judgement
“Je pointe du doigt le mot "raisonnement" pour faire
réfléchir l'élève.” (about 70%)
“Nothing for that student, but I might [make] periodic
general announcements: ‘Don’t forget to read all the
instructions carefully.’” (about 20%)
“You cannot do anything about that.”
Dilemma #3
As you collect the multiple-choice answer
sheets, you notice that one student has coloured
two bubbles on several questions.
What would you do?
En recueillant les feuilles-réponses des question
à réponse choisie, vous remarquez qu’un élève
a noirci deux bulles pour plusieurs questions.
Que feriez-vous?
Responses to Dilemma #3
Strict Adherence
“Nothing” (about 60%)
“Je pointerais les cercles ou je lui demanderais lequel il a
voulu effacer sans succès.” (about 40%)
Professional Judgement
“You cannot do anything about that” (about 50%)
“I would ask them to change their answer to one bubble. It
is not a reflection of what they know if they mess up the
directions. Is the EQAO a test of directions or
knowledge??” (about 50%)
Questions
• When teachers say they believe in strict
adherence to the instructions, do they mean
the literal instructions or do they include
modifications that they believe will help EQAO
better achieve its goal of measuring what
each student knows and can do?
• What are the implications for our
understanding of teachers’ beliefs, actions,
and motivations?
• What are the implications for the interpretation
of test results and the comparison of results
across classrooms?