Methodological Processes for Health Surveys - Issues for Survey Design

Presented by Eden Brinkley - Director,
Population Survey Development Section,
Australian Bureau of Statistics

Aims of Presentation

• With an emphasis on survey design
• contrast the various collection methodologies used for Health surveys
• highlight some of their relative strengths and weaknesses
• highlight other general collection issues

Collection Methods

• Paper and Pencil Interviewing (PAPI)
  • Face to face interviewing (FFI)
  • Telephone interviewing (TI)
  • Self-enumeration (SE)
    
    
• Computer Assisted Interviewing (CAI)
  • Computer Assisted Personal Interviewing (CAPI)
  • Computer Assisted Telephone Interviewing (CATI)
    
    
• Computer Assisted Self Interviewing (CASI)

So which method is best?

• Choice of mode is determined by many different factors
  • cost and sampling considerations are usually the dominant ones
    
    
• Choice of mode has important implications for data quality and collection design
  • from a data quality perspective several broad sets of variables are impacted

Cognitive variables

• Sensory channels
  • with SE forms, questions are visually displayed and read by the respondent
  • with TI, questions and response alternatives are read to the respondent
  • with FFI, both modes of presentation are available
  • sensory channels impact significantly on how respondents understand and provide answers to questions
    
    
• Order of presentation
  • with TI and FFI questions are strictly sequential
  • affects the context in which answers are provided
    
    
• Time pressure
  • pace for TI and FFI is dictated by the interviewer
  • TI has no visual cues to bridge silences
  • pressure on respondents can interfere with recall processes

Social interaction variables

• TI and FFI are both affected by the interviewer
  • Some affects are positive (e.g. motivating respondents to participate/continue with the survey)
  • Others are negative (e.g. respondent's perception of the interviewer may encourage socially desirable answers)
  • SE forms least affected by social variables

Other variables

• Perceived relevance and/or importance
• Perceived confidentiality

How does this impact on TI surveys?

• Respondents' concentration and patience are shorter over the phone
  • long blocks of text should be avoided
  • concepts should be relatively simple
  • recall periods should be kept to a minimum
    
    
• After 20 minutes respondent interest will, in general, tend to wane and their co-operation reduce
  • luckily Health is an interesting topic!
    
    
• Pauses/silences are much more noticeable over the phone
  • increases pressure for 'top of the head' responses
  • should encourage respondents to take their time
  • Personal interviews with all household members can be more difficult to arrange
  • Pressure for socially desirable answers (but better than FFI)
    
    
• Anything requiring visual input is not feasible
  • careful thought is required to translate these concepts to a TI environment
    
    
• List effects can be an issue
  • keep response alternatives short (max 5 or 6)

Development of CAI

• Centralized CATI facilities were first established in North America in the 1960s
• Move to CAPI has largely occurred over the last decade with the advent of portable computers
• Why has CAI become so popular?

Advantages of CAI

• Automatic sequencing leads to lower item non-response and more consistent responses overall
• Can readily target specific subgroups in the population and deal with complex issues in depth
• Question wording can be tailored based on information collected earlier in the interview
• Edits and consistency checks can be programmed into the questionnaire
• Processing time and costs reduced because data is already in an electronic format
• Coding procedures can be programmed into the computer
• Information already known about a respondent (e.g. from a previous interview) can be readily used (dependent interviewing)

Disadvantages of CAI

• Larger cost for initial set up
• Software specifically developed for questionnaires is usually required
• More time and effort required to develop and test a questionnaire
• More difficult to validate
• Greater training effort required to train interviewers to use computers
• Can be difficult to produce a usable/suitable paper questionnaire

Some general collection issues

• Be conscious of the significant issues surrounding sampling frames
  • e.g. RDD and White Pages have significant quality implications
    
    
• Good rapport has always been seen as a key feature of a successful interview
  
  
• Look for crosscutting standards
  • maximise comparability between states and other health collections such as the NHS
    • benchmark to other surveys
    • efficiencies in development and testing

Authorised by: Executive Officer, NPHP
Contact: Darryl Kosch, nphp@dhs.vic.gov.au
Date of publication: 1999
Page last updated: 27 November, 2003

Copyright | Disclaimer | Privacy statement