Developing a Downtown Survey

Basic principles of good surveys

It is not easy to develop a truly useful survey. Although many texts are available on "survey research" techniques, there are "Seven Immortal Principles" which give a good base to developing a survey questionnaire.
  1. Brevity
    • Avoid long introductions.
    • Try to simplify the grammar used throughout.
    • If a question and its answers are too long, there is a bias for respondents to answer the last answer presented.

  2. Clarity
    • Use very common, unambiguous terms.
    • Avoid questions with double meanings.
      (e.g., "Have you stopped beating your wife? o Yes o No" )
    • If an uncommon term is used, make sure to define it in the survey form.
    • Do not use technical or specialty jargon.
    • Be careful using the word "you"; it can be singular or plural.

  3. Reality
    • What people say and how they respond doesn't always represent how they will behave; assume natural biases in their responses.
    • For the general public, do not rely on hypothetical questions.
      (e.g., "If such and such, then would you...?")

  4. Unidimensionality
    • Don't combine two questions in one.
      (e.g., "Do you shop downtown or do you shop at malls?")

  5. Completeness
    • If multiple alternative responses are included, they should be mutually exclusive.
      Also, they should include all possibilities.
    • Try to avoid questions that can be answered "I don't know."

  6. Evenhandedness
    • Avoid leading questions, "loaded" words, and authoritative statements.
    • Questions should ask for opinions, rather than fact.
      (e.g., Don't ask, "Is downtown a dangerous place at night?"
      Instead, ask, "Do you think downtown is dangerous at night?")

  7. Dignity
    • Take respondents and their answers seriously.
    • Avoid condescending slang.
    • Don't present challenging questions.
      (e.g., "Why don't you shop downtown more often?")

Other suggestions:

A survey questionnaire should include a brief introduction indicating who is conducting the survey, how the information will be used, and reassuring participants that their responses will be kept confidential. Credibility can be further established by suggesting the sponsoring organization has a neutral viewpoint and no inherent bias.

One of the most critical questions on any questionnaire is the first one. It is the question that determines whether or not a person will complete the questionnaire. Therefore this question must be intriguing, yet easy to answer. It should draw readers into the survey topic and make them feel like they will have something to contribute. It should not make the readers feel they don't have enough knowledge to continue, yet it should not be overly simplistic and make them feel their answers are inconsequential.

A good guide for developing questionnaires is "Total Design Method" format described in the book, Mail and Telephone Surveys: The Total Design Method, by Don Dillman.1 The Total Design Method (TDM) emphasizes the use of a small booklet format made from standard size sheets of paper folded in the middle and stapled. According to this format, a full questionnaire should have no more than 12 pages (i.e., three sheets of paper). The front and back pages are used as "covers" and have no questions. The TDM also stresses a graphic layout which emphasizes the framework of the questions.

Biases commonly found in surveys:

It is common for surveys to have biases. These should be thought of beforehand, and methods found for minimizing their impact. If such biases are too great, the legitimacy of the survey findings could be called into question. Three of the most common types of biases are the following,
Hawthorne Effect:
One bias commonly found in survey research is the Hawthorne Effect, which says that respondents tend to respond differently simply because they have been selected for a survey. Because of the special recognition which has been given them, it is sometimes found the respondents tend to answer in the way which will most please the researcher. To minimize this bias, the questioner should be as neutral as possible in presenting the survey.

"Self-lifting" bias:
Closely associated with the Hawthorne Effect, the "self-lifting" bias recognizes respondents want to make themselves appear in a positive light, and will respond accordingly. This bias can be minimized by positioning personal questions about respondents at the end of the questionnaire, where they would tend not to affect other, more substantive responses.

The "Habit" bias:
If given a series of similar questions, respondents will fall into a habit of answering them similarly without considering each on its merit. This bias can be minimized by changing the format of questions throughout the questionnaire. The format may range from simple "check the box" questions to one-word responses to open-ended responses to completing information on simple graphs and maps. Through these variations, each question is given its own personality, avoiding the "habit" response.

Non-respondent bias:
One type of bias often found in surveys is based on the assumption that individuals who haven't responded in a survey tend to feel the same as those who have responded. However, studies have found that non-respondents generally have a more negative outlook, which may not be otherwise represented in the collected data.

Stopping Rule:
A bias sometimes found in survey research is based on the fact that the survey procedure is terminated when a researcher has obtained the desired results, and the amount of data collected is determined by results. This was avoided by establishing survey parameters beforehand.

Some sample questions

Below are listed some types of questions that could be included in a survey that tried to develop information on revitalization strategies. They are meant only as general examples.
  1. How often do you come to the downtown?
    More than once a week/ More than once a month/ Less than once a month

  2. What are the reasons you come to the downtown?
    Shopping/ Business/ Entertainment/ Other __________

  3. What do you like best about the downtown?

  4. What do you like least about the downtown?

  5. How would you like to see downtown changed?

  6. How would you rate your downtown compared to other downtowns similar in size?
    Much worse/ Worse/ About the same/ Better/ Much better
After preparing such a survey, give it to a few friends or associates to pre-test it and make changes to refine it and meet the seven "immortal" principles listed above. When you have reviewed these sample responses, analyze the meaningfulness of the results. For this purpose, it makes little difference whether there are three responses or three hundred, since you are looking only at the effectiveness of the questions, rather than the data itself. From this sample response, if it held true for a much more inclusive survey, would you have useful information which indicated what should be done to develop a revitalization program, or do you have a lot of data which looks impressive, but from which little inference can be drawn.

1 Don A. Dillman. 1978. Mail and Telephone Surveys: The Total Design Method. New York City: John Wiley and Sons.

Suggested other pages...
Profile of Residents Conducting Surveys