Home 9 Blog Posts 9 Can One Question Do the Job?

Can One Question Do the Job?

by | Jul 10, 2024 | 1 comment

#

By Steven Bell, Associate University Librarian, Temple University Libraries

There’s a concept in the world of design and innovation that makes the case for placing constraints, possibly even severe constraints, on one’s design options. 

Photo by Matt Walsh on Unsplash

When choice is constrained, our creativity must be more deeply mined to achieve the best possible solution within a limited range of options. Imagine, for example, having to construct or renovate a library with a $100,000 budget as opposed to a $200,000 budget or just three floors instead of four, but in the same footprint.

These types of constraints would force us to dig deep for ways to maximize the resources with which we had to work. It would require us to concentrate our thinking in such a way as to avoid careless decisions and focus our creative energy intently on getting it just right.

Now, imagine applying a design or innovation constraint to the next library survey instrument you construct.

No questionnaire can serve up an unlimited number of questions. Those that take more than 10 or 15 minutes will test anyone’s patience and are best avoided. Surveys design experts advise brevity over comprehensiveness

Eliminating a few questions from a collection of 20 or so is likely an exercise in making some tough choices. That survey will still serve well to collect useful information on just about any topic. 

What if you had to design a survey with just one question? Could you do it? Would you want to? How would you construct that one question for maximum return?

I, along with a colleague, was faced with that exact severe constraint. You know what? Coming up with just one survey question is really hard. There are reasons to do it.

Our institutional Textbook Task Force, a group of representatives from multiple campus offices, along with faculty and student government representatives, desired to update a now several years old survey on faculty OER adoption. OER advocates know the challenge of trying to figure out who is an OER adopter at their institution. Conducting a survey is one of several possible options for cracking that nut.

There’s one problem. Increasingly our institutions seek to drastically limit the distribution of surveys to avoid survey fatigue from hampering those one or two institution-wide surveys where maximum participation is mission critical. Think NSSE or an annual staff satisfaction survey.  The rise of the Survey Review Committee, or whatever it’s called at your institution, is the primary safeguard against campus wide survey fatigue. That’s good for students and faculty, but not so good for those of us who need to get information from them.

All hope was not lost. Instead of the multiple question survey we wanted to administer, we were offered the opportunity to conduct a one-question pop-up poll. Here’s how it works.

Most of our institutions have some sort of information portal where all staff, faculty and students go to access vital employee and institutional policies, procedures and data. That might be payroll or benefits for faculty and staff. For students, it could be access to financial aid or registration resources. In other words, everyone needs to visit the portal at some point. When they do, they get the pop-up poll. 

That one-question limitation is challenging yet offers a significant bonus. We are learning that pop-up polls get significantly better response rates than longer survey instruments, whether they are distributed to everyone in the target population or just a statistical sampling. It’s just one question. That’s easy and fast. Once responded to, the pop-up poll ceases to pop.

Additionally, because respondents have logged into the portal with their institutional account, the pop-up poll can automatically provide demographic data such as class or faculty status, academic unit and more – or none at all.

What goes into the design of a single-question survey, and what helps to make it as useful as possible for data collection and decision making?

A good starting point is understanding what the options are. Can you ask a yes/no, a multiple choice or an open-ended question? Can there be a follow-up question or some degree of question logic? Even when the opportunity for a follow-up is a possibility, it may be better to keep it simple if the goal is to achieve the highest response rate.

If you are looking for tips and suggestions for designing a one-question survey, you can find ample advice via a browser or AI search. Simplicity, clear and neutral language, and specificity are among the characteristics of successful one-question polls.

In our case, on the advice of colleagues who regularly construct these pop-up poll questions, we decided to go with a multiple-choice question where only one option could be selected. It did offer “other” as one of the options to allow for respondents to share a comment. 

Depending on the portal, it may be possible to add an additional instruction that is separate from the actual poll. For example, directly under the poll question, you might add “Please complete this form if you are willing to participate in a focus group.” We did add one such question to the faculty poll.

Figure 1 is a screenshot from our faculty pop-up poll results. Figure 2 is a screenshot from our student pop-up poll:

 

Figure 1, a screenshot from our faculty pop-up poll results

 

Figure 2 , a screenshot from our student pop-up poll

The pop-up poll was available through the faculty and student portals for a total of three weeks. As illustrated in the charts, the response level exceeded our expectations. Given the subject and course materials, it’s possible the 6,000+ student response was driven by the nature of the question. If you’re interested in my summary and findings related to this pop-up poll, feel free to get in touch. 

What did we learn from our one-question portal pop-up poll? While it’s a definite challenge to design it, getting a good response makes it worthwhile. Here’s a quick list of some pros and cons.

PROCON
Less time to construct and administerYou only get to ask one question
Better response rateChallenge to design
Portal polls offer useful respondent dataQuestion type limitations
Fast time from launch to reportLess information can be gathered
Reduces survey fatigueMay seem intrusive to portal users
Rarely needs Survey Committee or IRB review/approval 

So, can one question do the job? Did we get the results we wanted?

Sure, we would have wanted to ask more than one question. But, in lieu of a more robust survey, our tradeoff was a better response rate. The results do shed some light on the current state of OER and other zero-cost course material adoptions among our faculty, as well as the impact of these materials on our students. Surveys are rarely perfect, and a one-question poll is even less so.

Still, I’d recommend a one-question pop-up poll if your institution offers it as an option. Perhaps it does, but no one has ever asked if it is possible. You could be the groundbreaker.

 The question design challenge was a good one. As is often the case with constraints, it pushed my colleague and I to think really hard about what we wanted to ask. Then we needed to ask it with the most simple, clear, concise wording possible. Our Task Force helped to refine the question and response options. We’ll be following up with faculty respondents to learn more about what OER resources they’ve adopted as course material.

What’s next? A one-question library survey, of course. What would you ask?

1 Comment

  1. Lynda James-Gilboe

    Fascinating topic. Survey fatigue is definitely a thing, and limiting number of questions helps to mitigate that. Pop-up surveys are especially effective when they are perceived as highly relevant to the task at hand – i.e. when asked about the application the respondent is currently using or when they are focused on a burning issue or topic of interest to constituencies. When you really do need more information but want to keep it simple, a way to leverage this approach is to plan your list of single questions to be asked over time — once a week, once a month, once a day.

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

LATEST NEWS

ATG Job Bank for 7/21/24

Image via Pixabay NORTH Reference & Digital Services Librarian, Bristol Community College, (Fall River, MA) Music Librarian for Learning and Engagement, Eda Kuhn Loeb Music Library - Harvard University (Cambridge, MA) GIS Librarian, Brandeis-Waltham Campus...

Tea Time With Katina And Leah

I have noticed that Artificial Intelligence  is  thrust on us at  times that we don’t need to use it. Has that happened to you? I was interested to see on Publishing Perspectives that Elsevier has done a study (Insights2024:Attitudes Towards AI) of researchers-writers...

SUBSCRIBE TO OUR PODCAST

Share This