Methods

Quantitative
Qualitative

Quantitative

Telephone Surveys

Telephone surveys are very useful research tools when designed to control for demographic data by skilled interviewers. They allow for the collection of a large body of data for a relatively low cost, and maintain a fairly high level of reliability. In addition, the use of bilingual interviewers, particularly Spanish language interviewers, allows more residents the option of participating in surveys.

DHM Research incorporates a mixed sample design in our telephone surveys that includes random digit dialing (RDD) and cell phone sample. Over the years, securing respondent cooperation for telephone surveys has become more difficult with the increased use of cell phones. While DHM Research sets quotas to ensure a representative sample, especially for 18-34 year olds, there is also concern that there are attitudinal differences between cell phone only residents and their counterparts. While research has shown that the difference between cell phone only and land line respondents is nominal (2-3 percentage points, if at all), we believe incorporating cell phone sample is important to secure the participation of younger and more mobile residents.

Online Surveys

At DHM Research, we are experts designing and deploying online questionnaires: valid and reliable online surveys.

There are many web sites promoting their own methodology for online surveys that are inexpensive because you have to build, launch, and conduct your own survey. They seem user-friendly, with a great appearance and wonderful testimonials. The biggest thing missing: if you plan to make difficult decisions and allocate resources based on the results from these surveys, you will need to ask tough questions about validity and reliability.

When the panel is large enough to demographically represent the general population of the area, and it is maintained by a reputable firm, online surveys are effective ways to enable members of a community, including voters or the general population, to provide feedback in a confidential and private manner, at their convenience, and to collect unfiltered responses to open ended questions.

There are several benefits to online surveys when an appropriate panel is obtained:

  • They can be lower cost
  • More questions can be asked because of the ease of response
  • Respondents can answer the questions 24 hours a day and at their convenience
  • Respondents can be more candid in their open-ended responses, which are not filtered by an interviewer

There are some limitations that must be noted. First, there is no true equivalent to random digit dialing since there is no central directory of all email addresses. In addition, not all people have internet access, especially the elderly and low-income residents, and therefore some opinions may not be collected.

MaxDiff

This advanced technique, most easily administered online, forces respondents to choose between key variables or values (versus rating or ranking them in conventional survey designs). This approach is more realistic than asking respondents to simply rank the importance of various options without asking them to rank these options against each other. MaxDiff is also valuable for validating findings derived from the more conventional telephone survey.

  • Research has shown that MaxDiff scores demonstrate greater discrimination among items and between respondents on the items. Rating scales can artificially constrain differences and thereby dampen differences.
  • The MaxDiff question is simple to understand, so respondents with a variety of educational and cultural backgrounds can provide reliable data.
  • There is no opportunity for scale bias since respondents make choices rather than express strength of preference using numeric scales. This is an extremely valuable method for diverse audience research studies.

Conjoint

Conjoint Analysis and Discrete Choice are advanced research techniques that help show how people make choices and decisions. These techniques enable DHM Research to significantly augment conventional research by providing a much more in-depth understanding of the choices that people make. Survey respondents are initially exposed to different choice configurations and are asked to choose which they prefer. The outcome reveals the attributes that are most important to them and the relative acceptance of each choice.

These research techniques essentially create a market or real world simulation that results in the ability to assess preferences of the targeted population. The outcome reveals the attributes that are most important to them and their relative importance. Armed with this information, organizations can make much better decisions.

Although similar, each technique has its strengths. Conjoint Analysis is very effective when numerous attributes are tested. This means that at each step, previous answers are used to decide which question to ask next to obtain the most information about a particular respondent’s preferences. Discrete Choice offers the ability to present a variety of choices with different attributes and is well-suited for more defined proposals.

Online Panels

DHM Research is a leader and pioneer in developing, implementing and building online panels to supplement research and public involvement strategies. We develop panels that are community engagement tools, allowing members to weigh in on important issues in their community.

A panel consists of a group of people with a shared connection – they live in the same area, or are members of the same organization – who sign up to give feedback to an organization or government agency through short surveys or online forums. Panels are an excellent way to collect feedback about an issue in a short amount of time, as well as to inform members about issues that are relevant to them. Members’ input to surveys from these panels often inform policy-making and planning decisions.

In an increasingly digital and wireless era, panels allow for technological flexibility for members. People can be involved in decision making processes through responding to surveys on their own time and in their own words, and from anywhere with internet access.

Qualitative

Focus Groups

Focus groups are small group discussions with typically 8 to 12 people who are led by a moderator. Other qualitative research designs include one-on-one executive interviewing and large group studies. All have the presence of a moderator or interviewer in common, whose role it is to follow a discussion guide designed to elicit the information required by the client. Discussions or interviews usually last 90 to 180 minutes and participants are usually given an honorarium for their time. This technique is superior for gaining an in-depth understanding of how people feel about a particular issue or topic because it allows for exploration of why respondents gave the answer they did. Qualitative research also allows the client an opportunity to discover the arguments, statements, and messages that will persuade participants of another point of view.

Often, multiple discussions or interviews are completed for validation purposes and to provide enough information to quantify findings.

To gain an accurate understanding of how the population feels about a particular issue, it is essential that individual values and beliefs be measured without peer influences. This can be accomplished by having participants respond to questions about a topic in a written exercise before discussion about the topic. Besides providing more valid information for quantifying attitudes, the results establish a baseline against which changes in opinion during the discussions can be measured.

Large Group Study

A large group study is a qualitative research method based on the focus group technique. It strikes a balance between the collection of valid and statistically reliable information and getting to the underlying motivations behind residents’ expressed attitudes. Instead of the 8-12 participants typical of a focus group, the large group approach uses 20-50 participants, depending on budgetary and time constraints. Like focus groups, large groups are a superior technique for gaining an in-depth understanding of how people feel about a particular issue or topic because it allows for the exploration of why participants gave the answer they did. More than one study is completed to further assure the generation of enough data to support quantitative findings. At the same time, the small group “feel” needed for discussions and qualitative findings is largely retained with appropriate changes in discussion moderating to deal with the larger numbers.

This research technique also provides an opportunity to share information with participants, like detailed service descriptions, and to provide a better understanding of an organization or topic. Participants are often better equipped to act as community ambassadors and share what they learned with other residents after participating in large groups.

Often, multiple discussions or interviews are completed for validation purposes and to provide enough information to quantify findings. It is recommended to hold a minimum of two large groups with at least 20 participants in each group.

To gain an accurate understanding of how the population feels about a particular issue, it is essential that individual values and beliefs be measured without peer influences. This can be accomplished by having participants respond to questions about a topic in a written exercise before discussion about the topic. Besides providing more valid information for quantifying attitudes, the results establish a baseline against which changes in opinion during the discussions can be measured.

In-Depth Interviews

In-depth interviews (IDIs) provide the opportunity for a one-one-one conversation, and are effective at assessing opinions and motivators on a variety of issues in a private and confidential manner. The conversation is guided by a set of questions, but flows freely and allows the respondent an opportunity to elaborate on their responses for a clearer picture and representation of their overall views. IDIs provide robust information, builds goodwill with participants and often makes them feel more supportive of the sponsor, and can be more flexible on an individual basis. IDIs also work well when participants might be reluctant to share their opinions or ideas in front of others, as they may in a focus group.

In these interviews, interviewees have the opportunity to share their thoughts, open-ended, in a discussion with a third-party interviewer, and are ensured of anonymity and confidentiality. Reporting and analysis of in-depth interviews removes any identifying information. Findings from in-depth interviews can be further tested and quantified through online surveys to further explore results.