Skip Logic & Conditional Branches in Surveys

Survey respondents can be routed through questionnaires using three types of branching logic:

• Skipping specific questions: A skip pattern will jump a respondent over a group of questions that isn't relevant to them. For instance, a common skip pattern has a respondent rate a particular attribute on a scale and, if the rating is high, skips the respondent over one or two follow-up questions designed to probe why other respondents gave the item a low rating.
• Conditional branching: A branch pattern will route a respondent to the appropriate section of the questionnaire: each respondent follows one of the branches. A common branch pattern has a respondent classify a product purchase or a type of service interaction and then asks follow-up questions specific to that answer.
• Unconditional destination: Often a branching pattern is terminated with an unconditional destination, which returns respondents back to a common path. This has the effect of jumping respondents over other branches.

When you find yourself writing a question that starts with something like "If you answered ‘No' to the previous question", that is a sure sign that you should set up a skip or branch pattern. Making people do this logic in their head is fine in a paper survey, but it is in appropriate for online surveys. Such manual skip patterns slow down the respondent, increase the amount of reading that they must do and make completing the survey tedious. Conversely, when skip and branching patterns are well implemented, they make the survey highly relevant and engaging.
A great example of well-implemented branching is a web-site feedback survey created by one Vovici client. With over 15 million unique visitors a month to their web site, the client determined that visitors typically came to the site with one of 17 different purposes in mind. Visitors to the site were randomly invited to take the feedback survey, which began with five standard questions that all respondents were asked, ending with a question asking "So what are you here to do today?" The answer to this gating question would then branch respondents to one of 17 different paths, each asking an average of 8 questions unique to that action. The result was that no respondent answered more than 20 questions of a 148-question survey, and the questions they did answer were very specific to their experience.
Your own skip and branching patterns don't need to be as complex. Some best practices for skip and branching logic:

• Remember that page breaks indicate logical jumping-off points. Sometimes survey authors like to have one question per page; other authors like to have related questions together on a page. Skip logic overrides the preference for how to group questions, as it is only applied at the end of a page.
• Skip logic routes respondents forward through the survey, never backward. Think of the question flow like a river, going downstream, with different sluice gates channeling the water into different canals and sometimes back into the main watercourse.
• Due to the wide variety of possible synonymous answers to open-ended questions, skip logic is primarily used with closed-ended questions.
• Before inviting respondents to take your survey, make sure you test that each path through the survey matches the logic you intended.
• Some survey software applications, include Vovici v4, support Boolean logic ("and", "or" and "not" and nested parentheses). This makes programmers happy, but can be confusing for first-time survey authors. For instance, "and" doesn't have its English meaning: a survey author who wants the system to skip to Q9 when Q1 is answered "blue" and "green" needs to write "Q1=blue or Q1=green" not "Q1=blue and Q1=green" (which is a logical impossibility from a Boolean perspective). If you are using advanced branching for the first time, get some help from technical support or a programmer and test your survey logic carefully.
• For a questionnaire already collecting responses, you sometimes might find that you should have added a skip pattern. Good survey software will let you do this, then provide a utility to validate all previous answers against this new skip pattern, setting any answers back to unanswered for questions that shouldn't have been asked.
• If you are displaying progress bars, respondents may suddenly find themselves very far through the survey, because of a skip pattern. Curious respondents may hit the Back button, then answer differently to see where they go next. Respondents are more likely to do this in the screener if the survey offers a financial incentive to qualifying respondents. In such cases, configure the survey to not publish a Back button beside the Next button.

What other best practices do you follow when using skip logic?

Less Really is More When it Comes to Response Scales

Two years ago, Ziggy Zubric, owner of Marketing Endeavors, and I engaged in a passionate debate about a survey we were collaborating on. Ziggy made the case for replacing our traditional five-point scales with something more innovative. I decided his approach wouldn't work for the task at hand, but his enthusiasm and unique point of view stuck with me. Recently, when writing a questionnaire, I recalled his advice and decided to invite him to write a guest post about his work.  - Jeffrey

Every researcher (and every client!) should be interested in developing surveys that:

1. Look quick and easy to take
2. Generate maximal variance (i.e., good distribution among all response options)
3. Avoid "response set" (i.e., respondents mindlessly giving the same answer for every question)

After of years of conducting online surveys, we've developed some strong opinions on how best to do this, and it starts with (a) replacing traditional five-point scale items with three-point scales, then (b) using a non-traditional answer order.

For context, let me state that we focus heavily on the in-store customer experience. That is, after customers have visited a bank, restaurant, grocery store, retail outlet, etc., we want to discover their thoughts and feelings about the experience. 

Importantly, we do not believe in asking customers about specific details. For instance, unless a client demands it, we never ask customers to recall whether eye contact was made, if employees were wearing name badges, etc. Customers rarely retain accurate memories of such issues (why should they?), and if asked such questions, they simply guess.  

Thus, the most you can reliably do in a post-experience customer survey is to explore overall perceptions. Customers can't accurately tell you if a teller introduced herself, smiled, and expressed appreciation for their business, but they can report if the teller created an impression of professionalism, competence, compassion, patience, etc. (They can also accurately share their feelings on satisfaction, loyalty, brand perceptions, etc.) 

Benefit #1- It looks easier to take

First, our experience shows that respondents are far more willing to take and finish surveys that are visually simple and appear to require little effort. This leads to less fatigue and far greater completion rates. 

And let's face it, using only three answer options takes up a lot less visual space and is far less daunting than five options.
Thus, we always try to limit the response options to three, as that number offers the robustness to capture what we need while still remaining visually inviting.

Benefit #2- It generates more variance

One of the first objections people usually raise is that limiting respondent options to only three answers greatly reduces variance. But our experience shows that exactly the opposite occurs. 

Years of data show that the spread of a five-point scale usually skews heavily towards the top two answer options. Thus, when using five-point scales, we'll find that:

• 88% of respondents answer 4 or 5
• 10% answer 3
• 2% answer 1 or 2

We see this pattern whether using numbers (e.g., 1 through 5) or any variation of verbal descriptors (e.g., Very good, Above average, Average, Below average, Poor). 

The bottom line is that variance can't get much worse than it already is. 

This can't be valid

How believable is it that 88% of all customer experiences are ‘very good' or ‘excellent'? Does that jive with your experience? I've asked a lot of people that question, and not one of them has ever said "Yes."

The reality is that service experience data collected with five-point scales greatly exaggerates the quality of service. And practically, telling customers that their average score on a five-point scale is between 4.6 and 4.8 for every item simply isn't helpful. 

What clients ultimately want to know is: When are we so good/bad that the experience is memorable and noteworthy, and how often are we extremely good/bad?

According to five-point scale data, 88% of service experiences are Good or Excellent. But we all know that's not true. The majority of service experiences are average. 

But occasionally customer service is extremely wonderful or remarkably horrid, and those are the experiences we want to make sure we capture.

With five-point scales, a typical, efficient, mildly positive experience will be rated as a 4 by some respondents, and a 5 by others. And a negative experience will be rated as a 4 by some respondents, and a 3 by others. In effect, our categories stop being mutually exclusive, vitiating the validity of the data.  Ultimately, what we want to accomplish is to differentiate truly positive/negative experiences from the vast majority of average, just-fine experiences, and a five-point scale just doesn't get us there.

So why do respondents provide such glowing reviews? 

That requires a lot more discussion, but we're dealing with at least three psychological tendencies. 

First, respondents feel a sense of guilt about giving scores that are very low, so they avoid the bottom categories.

Next, respondents are far too willing to describe an average, typical, yet perfectly acceptable service experience as Excellent or Very good. 

Finally, respondents who are simply trying to get through the survey (aka, response setters) will mindlessly click the top answer option every time, creating higher positive scores. 

The three-pointer

Let's deal with overcoming the first two psychological tendencies. Basically, they both result in respondents being far too generous in their assessments. When we employ a three-point scale, we can word the response options to help customers provide more accurate feedback. 

Thus, our most negative option needs to sound very mild, making it less guilt-inducing. Our middle option needs to sound very positive, making it more acceptable as an option. Finally, our positive option needs to sound so stupendously, impossibly, incredibly tremendous that respondents understand to reserve it only for truly excellent experiences. 

Some wording examples we've used when assessing customer service:

• Above and beyond
• Very good
• Could use some tweaks

Or:

• Truly outstanding
• Perfectly acceptable
• Not the best

Clients that we have migrated to this system show more variance in their data, and the response options overlap far less, which increases validity. Now, the strong majority of responses fall into the middle category, which is just what we want.

And when someone checks "Truly outstanding," we are far more confident that our client's staff genuinely did provide extraordinary care. Similarly, when we see a "Not the best," we know that something was truly amiss. 

Also-and this might make traditionalists uncomfortable-don't be afraid to play with the wording. Different clients and different contexts require nuances. As long as you're consistent within that client (or within that industry if you're providing benchmark data), you'll be fine. 

Not to mention the fact that respondents are sick to death of traditional wording options, and they find unexpected answer options refreshing.

Benefit #3- Avoid response set

Sometimes respondents will put their brain on cruise control while taking a survey, and simply click on the same option (often the top one) for every question. To overcome this, list the "middle" option first. For instance, try:

• Very good
• Could use some tweaks
• Truly outstanding

Does this violate traditional ordering of categories? Yes. But so what? 

By listing your middle, unremarkable option first, people just mindlessly clicking the top option won't give you a false sense of good or bad news; and the respondents who truly have an extremely good or bad experience to relate will look for their appropriate response and check it, providing us more certainty that these really were exceptional experiences (for better or worse).

So overall ...

Vovici Online Survey Software

- Ziggy Zubric, owner, Marketing Endeavors

Checkboxes vs. Radio Buttons in Web Surveys

A survey from one of our customers was recently lampooned in a popular blog about “curious perversions in Information Technology”.

Clearly, the author of the questionnaire didn’t intend to set the validation for question 8 to limit to one choice, given their instructions to the respondent (“please tick all that apply”).  This was simply an honest mistake on their part.

But why should our survey software even allow the author to limit a choose-all-that-apply question to one choice?  If only one choice is permitted, then the system should use radio buttons or a dropdown box.  Early on, in fact, I wanted to make sure that this wasn’t even an option for questionnaire authors, but our customers pushed back that they wanted this functionality, for two reasons:

1. Some users don’t value the difference between radio buttons (shown in question 9, limited to one choice) and checkboxes, a distinction in user interfaces that dates back to at least 1984.  These users like the visual esthetic of the checkboxes better and choose to use checkboxes on all questions, even those where only one choice is permitted.
2. Some users dislike the fact that you cannot unselect a radio button, and prefer to use checkboxes so that a respondent can uncheck a choice if they decide no choice is applicable.

So, while you can certainly use checkboxes in this way, the best practice should be to use radio buttons when you only want the respondent to select a single choice, and to provide a “Not applicable” or “Does not apply” choice on radio-button questions when you want respondents to be able to “unselect” a choice.

Survey Software vs. Enterprise Feedback Management

How do survey software applications typically differ from enterprise feedback management solutions? 

Survey software is typically designed for the individual user, working on one or more survey research projects.  Core functionality includes:

• Survey Creation - Writing multi-page web surveys with open-ended and closed-ended questions.
• Templates - Starting with a wide selection of customer satisfaction, employee satisfaction, course evaluation and other standard survey instruments to choose from.
• Basic Conditional Logic - Skipping over certain questions and pages based on the answers to other questions.
• Multi-Language Support - Fielding one survey in multiple languages.
• E-mail Invitations - Sending out invites with hyperlinks to potential survey respondents and following up with periodic reminders.
• Basic Reporting - Viewing a frequency report or a list of verbatim responses for each question.
• Exportable Data - Saving survey data to file formats such as Excel and text files.

Vovici knows survey software well.  The PC Magazine roundup of six survey software applications in February 2000 featured three products - Perseus SurveySolutions for the Web, Websurveyor 2.0 and EZSurvey 99 for the Internet -  that are all part of Vovici's heritage (Vovici was formed from the merger of Websurveyor and Perseus Development, and purchased the Raosoft EZSurvey assets).  While Vovici knows survey software well, we've made a business of supporting customers as they've outgrown the needs for such software.  Vovici pioneered enterprise feedback because so many survey software customers needed more functionality.

Today, enterprise feedback management systems typically extend the core functionality of survey software with:

• CRM Integration - Where users grow tired of exporting email lists from their CRM system, CRM connectors can keep the EFM system in sync automatically.
• Advanced Analytics - When users want to drill down on results in detail, EFM survey analytics go beyond  simple reports.
• Triggers & Alerts - When users begin to think about survey processes, rather than projects, they can set up email alerts triggered by specific answers to the survey.
• Report Distribution - When users want to share results widely across an organization, or customize reports for different audiences or even for each individual, EFM report distribution can save time and energy.
• Quotas - When users want to limit participation in surveys, for purposes of offering incentives or for balancing responses by key segments, they can easily set survey quotas.
• Panel Management - When customers or other key constituencies are complaining about receiving too many surveys, panel management can be used to set up limits on frequency of surveys.
• Page Rotation - When respondents complain about surveys that are too long, page rotation is one tool for shortening the questionnaire from the respondents' perspective.
• User Management - A key part of sharing usage of surveys throughout an organization is defining different roles for different types of users:  some can write questionnaires, some can create reports, some can only view reports, and so on, as appropriate.
• Workflow Management - Users can draft surveys and then submit them to reviewers to edit, approve and publish, thereby by ensuring that research best practices are followed.
• Online Communities - In the past year, EFM systems have expanded to support online communities, providing better qualitative research than ever before.
• Advanced Branching - As questionnaires grow in sophistication, authors often add more complex rules around which questions are shown to respondents, so that they are answering only the most relevant questions and do not have to manually skip over questions that aren't appropriate to them.
• Response Randomization - Finally, to remove any order bias, the choices for questions can be shown in random order.

If your organization has any of the above needs, it is time to consider moving up from survey software to enterprise feedback management.

Don’t Vote, Don’t Take Surveys, Don’t Give Feedback

On Election Day in the United States, it's appropriate to momentarily consider the argument for not voting.  I predict that not one election today will come down to a single vote.  Therefore, casting your vote doesn't make a difference, and by skipping it you can save yourself the hassle of standing in line.  Or, to put voting in economic terms, as Glen Whitman does, "the marginal benefit of voting is essentially nil, since a single vote almost never decides any election, while the marginal cost includes the opportunity cost of your time, the cost of travel, the risk of getting hit by a truck on the way to the polls, etc."

This sprang to mind last week when I was introduced to Cathy, a woman who-after discovering what I did for a living-was quite curious about web surveys.  "Why would anyone take the time to fill out a survey?  It just seems like a waste of effort."  She concluded proudly, "I have never taken a single survey!"

Living in Massachusetts, I then went out on a conversational limb.  "Do you like Starbucks?" I asked.

"Oh, I love Starbucks!" Cathy said.  (Most Bay Staters love Dunkin' Donuts coffee over Starbucks.)

I then explained to her about MyStarbucksIdea and how people could submit their own ideas, because they were passionate about their Starbucks experience and wanted to see Starbucks serve them better.  I said people filled out surveys for the same reason, to provide that feedback. 

She instantly understood.  "I once wrote the president of Starbucks a letter!"

"Well," I said, "people answer surveys and participate in online communities for the same reason you wrote that letter."

And people vote for the same reason.  No, it may not be rational to think that that one survey response is going to make the company a better place to buy a cup of coffee from.  But the collection of survey responses and community feedback helps Starbucks tremendously.  No, if you're a U.S. citizen, your single vote today  won't decide any outcome, but your public commitment to voting and your engagement in the process will help improve the country.  On a day when both major-party candidates advocate change, your vote may even help transform the country.

See also:

• Idea Ownership in Research Communities (MyStarbucksIdea specifically)
• When Instant Feedback is Like Instant Coffee

 

How to build an online survey community

This is an art but you can do it. Beware though, traditional market research firms to help may not be your best advisors. They certainly can assist with helping you to design the survey and analyze the results but you really need expert advice on how to keep your community members, typically customer and perspective customers coming back to your brand time and again. Some of you should build online survey communities of your employees and partners too.

The research industry as a whole has not been very friendly to us. Have you taken a survey online recently designed by a research firm? Their online surveys look like offline surveys that have been put on the Internet. You read a question and check off your answer. You are rarely told how long the survey will take to complete and many times after you invest several minutes in taking the survey you reach a point where you are told "you don't qualify".   (Do that to someone and see if they will come back to take another survey or have a positive feeling towards your brand.)

Did you know that today in 2007 more customer satisfaction surveys are done on the telephone than on the Internet? I know I keep bringing this up but really, how stupid can we be? People have signed up in droves to the Do Not Call list, telling us that they don't want to be bothered with phone calls. Yet companies continue to do customer satisfaction survey on the phone. That is like telling one of your customers... we just don't care about what you want - we need to ask you some questions.

Off track for a minute there...

So what should you do?  At Vovici we are providing companies the necessary technology and expertise to manage their customers, employee and partners in survey respondent communities. Advanced knowledge management turns survey data into a permanent, shared asset across survey projects and teams. It has proven to be a differentiating factor when competing for new customers.

Direct feedback creates a combined voice to provide a formal voice, and therefore the feeling of influence. The community feels able to shape the future direction. Incentive programs within the survey process delivers reward for participation. Providing community members comparatives - what are other people like me thinking - is often viewed as a valuable incentive. Special options that create a deeper sense of identity and membership like double opt-in panels and special survey groups provide extended opportunities for involvement and more consistent feedback from key demographic groups.

Reflections about the online survey evolution in the marketing research industry

I attended the CASRO (Council Survey Research Organizations) conference in Scottsdale during October. This was my 5th conference in the last seven years; my first conference was in 2001. It was held in Amelia Island during early October, a few weeks after 9/11. I drove from New York to Florida because I just wasn't ready to fly. I had flown flight 91 out of Boston many times and I had been in NYC the day before the attack. I also knew someone that lost her husband on that day. Flying wasn't an option for me then. It must have been a problem for many others too because the conference was not well attended.

At the 2001 conference we reviewed information collected from the industry that said most surveys were still conducted on the telephone and in malls. I recall in 2001 less than 10% of all surveys conducted in North America were via the Internet. Europe was even less.

What a difference six years have made. This year's conference had hundreds more people in attendance and I have never seen so many exhibitors at CASRO in the past. In 2001 Greenfield Online was the only company exhibiting that was talking about using their online survey panel to conduct surveys via the Internet. This year, E-Rewards, Survey Sampling, Research Now, Luth, Greenfield Online, GMI and even a couple of others were there to exhibit their online survey panels.

CASRO now reports that greater than 50% of all surveys are being done on the Web and the conversion to online surveys is happening around the world.

I believe we are entering a new stage of online survey growth. Companies love the speed and access to people at lower costs. Not only are they converting from other methodologies, but they want to do more and more online surveys as a result.

Survey Data: Security and Protection - It should matter to you

With the recent string of high-profile data losses, companies are focusing on securing and protecting their IT infrastructure and customer data. A loss of data results in the obvious  costs associated with the investigation and eventual remediation to close the gaps identified in the perimeter. But the bigger and costlier result of such breaches is the loss of customer trust.

Is survey data any less important than your other data sources? Surveys provide an important tool for asking your customers, employees and prospects important questions. Depending on the nature of the survey, the responses result in detailed profiles that include information such as:

• Opinions
• Preferences
• Buying habits
• Demographics
• Contact information

Customers provide you this data based on their trust that it will be used appropriately and handled securely. Loss of this data represents a breach of this trust and at a minimum will greatly affect your ability to gain feedback in the future. Of course, worst case scenarios can include a negative impact on stock price, lawsuits, and lost customers.

Low-cost survey software - How do they treat your data? There are many survey solutions on the market, some at very low costs. However, choosing a low-cost survey solution could mean that your data is stored on an unprotected server that would be an easy target for a hacker. There are also questions of data integrity and protection from damage or loss. Your data could be stolen in minutes, or destroyed by a hardware failure. You don't want that kind of monkey on your back.

Protect your survey results. We have been working for years to protect our customer's survey data because we understand the value of their survey results. Low-end survey providers don't provide the proper environment to protect your survey investment. Here at Vovici, we host your survey data in our SAS70 approved hosting environment.

Our survey hosting environment has been designed with security as a foremost consideration, with features such as 128 bit SSL encryption and redundant firewalls. Physical considerations have also been made in our data centers with 24/7 monitoring and protection, uninterrupted power supplies, fire-suppression and detection, redundant cooling systems and sophisticated data backup procedures. While our security polices have satisfied the most stringent of financial institutions, we are also constantly monitoring and making improvements.

Your survey participants are entrusting their responses and data to you and you can't take that responsibility lightly. You need to make sure that whatever online survey solution provider you use values data security as much as you do.

Leveraging Surveys for Successful Events (part 3)

Was all your hard work worth it? Did your attendees get what they were expecting from the event? Did the speakers show expertise in their field? Would they be willing to come back next time? Conducting an online post-event survey will provide you with all of this information and more.  This invaluable data allows you to cater your future events to meet the ever-changing needs of your audience.

Here are some helpful samples of post-event survey questions:

• How do you rate the event location?
• How do you rate the content that was presented?
• Please rate the quality of the speaker's performance.
• Would you be interested in volunteering at our next event?
• Please list any other comments or suggestions about the event.

With these question and more you can get feedback on the various aspects of your presentation, booth, or speakers. This data is essential when determining the success or failure of the event.

Remember to analyze your Survey Responses and then act on that data!

What is the use of conducting a great survey and asking all the right questions if you don't know how to analyze your responses? A good survey software tool is important, as you need to be able to easily analyze the data to fully leverage for decision making.

Be sure to allow your attendees an opportunity to give feedback and provide suggestions for future events. Consider asking them to provide contact information such as their name, email address, or telephone number so that you can follow-up with them directly, but be sure not to require this information as you may discourage them from responding.  However, the right survey tool does provide data security-which allows respondents to feel more comfortable providing their information.

With an online survey tool, you can survey event participants easily during each phase of the event process, determine the strong and weak points of your event planning and execution, and you will have the data you need to make improvements that will ensure the success of your event programs. Over time, these survey tools will help you understand how your events are meeting your target markets expectations, giving you the data you need to maximize your return on investment for all your events.

Leveraging Surveys for Successful Events (part 2)

What better time to ask your attendees about their experiences than when it is fresh in their minds? It's easy to obtain attendees' feedback-getting rid of the overwhelming burden of collecting, entering, and analyzing paper-based surveys by using on-site trade show surveys.

Many event organizers have a network of computers available for participants to check e-mail or surf the Web. These environments are often set-up with a designated home page that is ideal for conducting a survey during the event. Additionally, you can have volunteers armed with PDAs ask attendees to take on-the-spot surveys and have the data displayed on various monitors throughout the venue.

Using electronic surveys also allows you to engage your attendees by showing up-to-the-minute results and stir up dialog between individuals-- especially if your survey results are surprising or controversial.

Next, post-event surveys