An Evaluation of a Large Scale CATI Survey using Random Digit Dialling - a Household Survey

Based on a paper in preparation by Don Bennett
Queensland Government Statistician's Office
and David Steel
University of Wollongong.

CATI

• quick
• economical
• good response rates
• direct data entry
• computerised survey management system (SMS)

Random Digit Dialling (RDD)

• reaches silent numbers and recent connections
• more disconnected, non-household, fax
• not completely random

1995 Queensland Migration Survey

• percentage of movers, characteristics of movers
• 110,887 numbers
• 37,721 responding households from estimated 44,858 households
• 5,733 mover households

Evaluate

• operational methodology (protocols etc) using statistics obtained from computerised SMS
• use of RDD versus White Pages

Survey Management System

• logging interviewer activity
• scheduling repeat calls
• random selection of interviewees
• removal of numbers from call queue
• operational reports
• automatic dialling

Call Scheduling and Protocols

• single randomly ordered call queue
• repeat calls scheduled by time interval or
  period
• limit of 3 calls for fax, unidentified tone;
  8 calls for engaged, no answer, answering machine
• general limit of 2 weeks to contact
• concept of hierarchy status

Outcome of the Survey

• 234,791 call attempts to 110,887 numbers: avg 2.12
• estimated 44,858 households: avg 2.47 numbers/household
• final status of survey units
  • 37,721 usable responses: 84.1% response rate
  • 7.3% refusal rate

Findings

1. Time of day and day of week

• interviewed weekday afternoons and evenings to 8.30 pm and Saturday daytime
• compared % desired outcomes for first calls
• null result; evenings better for completions, but worse for screening

2. We should have allowed more than 20 minutes before calling back an engaged number

3. A small improvement in efficiency can be obtained by limiting either:

• number of calls to contact a unit eg limit of 6 saves 5.5% of time for loss of 1.6% completes
• total number of calls to a unit eg limit of 7 saves 3.7% of time for loss of 1.1% completes
• Warning:
  • limiting calls to 6 or less can significantly affect some estimates eg household type, age group, income bracket
  • care needed in truncating interviewing

4. Time spent on Call Back - Mover calls

• contacting a particular adult adds 12.5% to time.

5. Computing time spent on No Answer calls beyond 6th

• limit of 6 No Answer calls saves 4% of time for loss of 0.8% of completed interviews

6. Time spent on disconnected, faxes, non-private dwellings not also in White Pages

• use of RDD increases time by about 20%

7. There is significant regional variation in rates of various outcomes

8. Point of Refusal

9. Interviewer Differences

• operational differences
• differences in responses obtained - interviewer effects
  • Deff = 1 + (m-1)q
    where m = interviewer workload and q = intraclass correlation coefficient

10. Comparison of RDD and White Pages

• subset on numbers in EWP and recompute estimates
• several significant differences

Authorised by: Executive Officer, NPHP
Contact: Darryl Kosch, nphp@dhs.vic.gov.au
Date of publication: 1999
Page last updated: 27 November, 2003

Copyright | Disclaimer | Privacy statement