Questionnaires offer an objective means of collecting information about people's knowledge, beliefs, attitudes, and behaviour. 2 ,3 Do our patients like our opening hours? What do teenagers think of a local antidrugs campaign and has it changed their attitudes? Why don't doctors use computers to their maximum potential? Questionnaires can be used as the sole research instrument (such as in a cross sectional survey) or within clinical trials or epidemiological studies.
Randomised trials are subject to strict reporting criteria, 4 but there is no comparable framework for questionnaire research. Hence, despite a wealth of detailed guidance in the specialist literature, 1 - 3 ,5 w1-w8 elementary methodological errors are common. 1 Inappropriate instruments and lack of rigour inevitably lead to poor quality data, misleading conclusions, and woolly recommendations. w8 In this series we aim to present a practical guide that will enable research teams to do questionnaire research that is well designed, well managed, and non-discriminatory and which contributes to a generalisable evidence base. We start with selecting and designing the questionnaire. questionnaire.
You and your co-researchers may have different assumptions about precisely what information you would like your study to generate. A formal scoping exercise will ensure that you clarify goals and if necessary reach an agreed compromise. It will also flag up potential practical problems—for example, how long the questionnaire will be and how it might be administered.
As a rule of thumb, if you are not familiar enough with the research area or with a particular population subgroup to predict the range of possible responses, and especially if such details are not available in the literature, you should first use a qualitative approach (such as focus groups) to explore the territory and map key areas for further study. 6
People often decide to use a questionnaire for research questions that need a different method. Sometimes, a questionnaire will be appropriate only if used within a mixed methodology study—for example, to extend and quantify the findings of an initial exploratory phase. Table A on bmj.com gives some real examples where questionnaires were used inappropriately. 1
Box 1: Pitfalls of designing your own questionnaire
Natasha, a practice nurse, learns that staff at a local police station have a high incidence of health problems, which she believes are related to stress at work. She wants to test the relation between stress and health in these staff to inform the design of advice services. Natasha designs her own questionnaire. Had she completed a thorough literature search for validated measures, she would have found several high quality questionnaires that measure stress in public sector workers. 8 Natasha's hard work produces only a second rate study that she is unable to get published.
Research participants must be able to give meaningful answers (with help from a professional interviewer if necessary). Particular physical, mental, social, and linguistic needs are covered in the third article of this series. 7
Using a previously validated and published questionnaire will save you time and resources; you will be able to compare your own findings with those from other studies, you need only give outline details of the instrument when you write up your work, and you may find it easier to get published (box 1).
Increasingly, health services research uses standard questionnaires designed for producing data that can be compared across studies. For example, clinical trials routinely include measures of patients' knowledge about a disease, 9 satisfaction with services, 10 or health related quality of life. 11 - 13 w3 w9 The validity (see below) of this approach depends on whether the type and range of closed responses reflects the full range of perceptions and feelings that people in all the different potential sampling frames might hold. Importantly, health status and quality of life instruments lose their validity when used beyond the context in which they were developed. 12 ,14 ,15 w3 w10-12
If there is no “off the peg” questionnaire available, you will have to construct your own. Using one or more standard instruments alongside a short bespoke questionnaire could save you the need to develop and validate a long list of new items.
A valid questionnaire measures what it claims to measure. In reality, many fail to do this. For example, a self completion questionnaire that seeks to measure people's food intake may be invalid because it measures what they say they have eaten, not what they have actually eaten. 16 Similarly, responses on questionnaires that ask general practitioners how they manage particular clinical conditions differ significantly from actual clinical practice. w13 An instrument developed in a different time, country, or cultural context may not be a valid measure in the group you are studying. For example, the item “I often attend gay parties” may have been a valid measure of a person's sociability level in the 1950s, but the wording has a very different connotation today.
Reliable questionnaires yield consistent results from repeated samples and different researchers over time. Differences in results come from differences between participants, not from inconsistencies in how the items are understood or how different observers interpret the responses. A standardised questionnaire is one that is written and administered so all participants are asked the precisely the same questions in an identical format and responses recorded in a uniform manner. Standardising a measure increases its reliability.
Just because a questionnaire has been piloted on a few of your colleagues, used in previous studies, or published in a peer reviewed journal does not mean it is either valid or reliable. The detailed techniques for achieving validity, reliability, and standardisation are beyond the scope of this series. If you plan to develop or modify a questionnaire yourself, you must consult a specialist text on these issues. 2 ,3
Questionnaire items may be open or closed ended and be presented in various formats ( figure ). Table B on bmj.com examines the pros and cons of the two approaches. Two words that are often used inappropriately in closed question stems are frequently and regularly. A poorly designed item might read, “I frequently engage in exercise,” and offer a Likert scale giving responses from “strongly agree” through to “strongly disagree.” But “frequently” implies frequency, so a frequency based rating scale (with options such as at least once a day, twice a week, and so on) would be more appropriate. “Regularly,” on the other hand, implies a pattern. One person can regularly engage in exercise once a month whereas another person can regularly do so four times a week. Other weasel words to avoid in question stems include commonly, usually, many, some, and hardly ever. 17 w14
