Presentation

Mixed-mode on Victimisation Survey : Safe or Risky ?
Results of a French Experiment
Tiaray Razafindranovona, Carine Burricand,
Bruno Dietsch, Gaël de Peretti
September 5th, 2014
2
Introduction
Context of households surveys
•
Demand for better quality, more complexity …
•
… in a general context of budgetary reduction !
•
Mixed-mode to face these challenges ?
•
Experimental surveys will help to answer
3
Principles of French experiments
•
An experiment for (almost) each new household survey
–
–
•
No interference with main survey
–
–
•
General results on mixed-mode surveys not sufficient
Experiments in parallel of each new CAPI survey
Workload
Sampling frames
Each survey must test something new
–
–
Accumulation of experience
Test of new specific points not already handled
4
Standard mixed-mode protocol
•
Sample frames
–
–
•
Questionnaire
–
–
•
Name and adress for massmailing
Available in taxpayers files
Fitted subset of the CAPI questionnaire
Length around 10/15 minutes
Notification letter, reminders
–
–
First, a postal mail with a notification letter (URL, login, password)
2 reminders (first one with paper questionnaire)
5
Examples of experiments
•
Housing 2010 (Amiel, Denoyelle, 2012)
•
Quality of Life at Work 2013 (Razafindranovona et al., 2013)
•
Safety Survey 2013
•
Housing 2014
–
–
•
Wealth 2015
–
–
•
Focus on rental amount
External documents without the interviewer ?
Focus on value of real estate property
Deviations from market value without the interviewer ?
Dedicated Project on Labour Force Survey 2014-2017
6
Living environment and safety survey (CVS)
Thefts, violence and safety (VVS)
•
Living environment and safety survey (face-to-face survey)
Allows to estimate rates of victimization, to study profiles of victims, to
analyze relationship between victimization and feeling of insecurity …
Confrontation with administrative records
Protocol : face-to-face survey with audio-CASI (headphone) for sensitive
questions (sexual violence, violence within the household)
•
Experimental parallel survey : Thefts, violence and safety
Tackles same topics : shorter version of the questionnaire, questions on
violence within the household are not asked for security concerns.
7
VVS – Protocols and problem of selection
•
40 000 individuals drawn in a base built on taxpayers files
•
Two different protocols to prevent from auto-selection
(persons from the same household that respond instead of the
selected individual)
- A second couple of login and password is given : another
person in the household could respond to the questionnaire
(these voluntary respondents are excluded from the analysis)
- The questionnaire includes a table where all the members of
the household could report their sexe, age and victimizations
•
Control with variables of the sampling frame (sex, age)
to detect suspicious cases of auto-selection
VVS – Response rates
Response rate of 32 % (about 13 000 questionnaires)
Balanced repartition between paper and web
- 53 % paper
- 47 % web
8
No significant difference between the 2 different protocols
(response rate, rates of victimization)
VVS - Characteristics of respondents
9
Web respondents younger, with a higher level of education
and richer …
- Median age
44 years (web), 58 years (paper), 48 years (sample)
- Median annual income
37.000 € (web), 29.000 € (paper), 28.000 € (sample)
- Proportion of individuals with tertiary education
45 % (web), 22 % (paper), 24 % (general population)
CVS / VVS comparison
Victimization rates don’t converge …
… even after multiple calibrations !
Thefts without violence (in 2011 and 2012)
2,8 %
VVS after non-response correction
+ sociodemographic calibration
+ calibration on number of complaints
+ calibration on feeling of insecurity
6,4 %
6,4 %
5,4 %
4,6 %
1
CVS
Uncontrolled selection (1)
divergence between rates of CVS and VVS
mostly explained by uncontrolled selection
Our hypothesis :
people that are interested by the topic of the survey, and, in
particular, those who are victims, tend to respond more than the
others (problem of nonignorable nonresponse when participation
is correlated to the key variables)
1
Interviewers convince even the less interested people to participate
but this is not possible with auto-administered questionnaires
Uncontrolled selection (2)
Illustration of these problems of uncontrolled selection within
the protocol of VVS : one proxy for « motivation » could be
the timing of the response (for the web respondents),
before or after the first reminder.
victimization rates are higher for early respondants
Threats (in 2011 and 2012)
1
Before reminder
After reminder
10,3 %
7,2 %
Mode effects ?
Difficult to disentangle selection and mode effect …
… but, for example, some suspicions of double counting
for some victimizations due to :
1
-
structure of questionnaire
ambiguousness of some concepts
(ex: thefts with violence vs physical violence)
1
Conclusion
Victimization survey can’t rely solely on auto-administered
questionnaires
Before integrating web mode in the protocol, robust results on
mode effects are needed (but difficult to measure because
of uncontrolled selection)
Mixed-mode not to be forced without prior evaluation of quality
1
References
Amiel, M. H,. and Denoyelle, T. (2012), Enquêtes en ligne : comparaison de modes de questionnement sur le
thème du logement, XIèmes Journées de Méthodologie Statistique de l’Insee
de Peretti, G., and Razafindranovona, T. (2013), Les enquêtes multimode : multi-solution ou multiproblème ?, 45e Journées de Statistique de la SFDS.
de Peretti, G., and Razafindranovona, T. (2014), Les enquêtes multimode : attention aux effets de mode,
Statistique et Société, vol.2 No 2.
Gombault V. et Duée M. (2012), Un exemple d’enquête multimode à l’Insee : l’enquête TIC auprès des
ménages, 7ème colloque francophone sur les sondages.
Razafindranovona T. (2013), La collecte multimode et le paradigme de l’erreur d’enquête totale, Séminaire de
méthodologie statistique du Département des méthodes statistiques (Insee)
Razafindranovona T., Barrau A. and de Peretti G. (2013), The philosophy of French experiments on Internet
and mixed-mode data collection, Seminar on Statistical Data Collection
Sautory O. (1993), La macro CALMAR : redressement d’un échantillon par calage sur marges, Série des
documents de travail de la Direction des Statistiques Démographiques et Sociales