Screener Best Practices

Screeners are an important step in the recruitment process. The screener helps determine whether a member is qualified for the community. A correctly targeted screener ensures that the data that you are collecting is actually coming from people who fit the target demographic that your research project is built for.

Below is a high-level overview of what the recruitment process looks like:

Compose the Screener

While not all screeners will be the same, some best practices should be implemented for all community screeners:

• Keep it short and sweet—ideally 10 questions or less. A shorter screener leads to a better user experience which in turn will yield lower drop-out rates.
• Questions that do not have any term, quota, or, logic associated with them should be removed from the screener and moved to a Profiling Survey or Member Profile. This helps to keep the screener short and sweet.
• Community opt-in should be shown after a respondent qualifies (see Appendix for sample text).
• Any questions that are used as a term or quota should term if “Prefer not to say” is selected.

If using special question types, make sure to test out whether they will import as Profile Points. One such example is the Cascading Drop-Down Menu. Due to API configurations, this question is not compatible with profile points. It’s always best when writing a screener to stick to text boxes, single/multi-select, and grids. These standard question types will always import correctly. 

If you need assistance with testing a certain question type with Profile Points in Qualtrics or Alchemer, please contact your Fuel Cycle account team.

Programming a Screener in Qualtrics

For more comprehensive, step-by-step instructions on creating a screener in Qualtrics, please refer to this article.

Redirects

Success Redirect URL

${e://Field/redirectURL}&surveyId=${e://Field/SurveyID}&responseId
=${e://Field/ResponseID}&qualified=true

Failure Redirect URL (use for qual failure)

${e://Field/redirectURL}&surveyId=${e://Field/SurveyID}&responseId
=${e://Field/ResponseID}&qualified=false&reason=LOGIC&disqualifiedStep=Q1

Over Quota Redirect URL (use for quota logic)

${e://Field/redirectURL}&surveyId=${e://Field/SurveyID}&responseId
=${e://Field/ResponseID}&qualified=false&reason=OVER_QUOTA

The red text portions in the Failure and Over Quota Redirects can be changed so that you can track disqualification numbers through the FC platform dashboards. These designations should not include spaces and typically should be written in all caps. (e.g. HOUSEHOLD_INCOME)

Recording Terminations

If using a Qualtrics set up by Fuel Cycle, standard practice is to select the Screen-out Response option on your End of Survey blocks.

This option will be available on all Fuel Cycle configured Qualtrics accounts. If using a corporate account, the Screen-out Response option may not be available. Please contact your Fuel Cycle account team for recommendations on configuring terminations. 

Survey Options

The following survey options should be implemented on any screeners programmed in Qualtrics.

• Survey Protection Settings:
  • “Open Access” enabled
  • “Prevent Ballot Box Stuffing” enabled
  • ”Prevent Indexing” enabled
• Partial Completion
  • Record responses in progress 6 months after the respondent’s start of the survey

Programming a Screener in Alchemer

For more comprehensive, step-by-step instructions on creating a screener in Alchemer, please refer to this article.

Redirects

Paste the following into the text field of the URL redirect action and use the configurations outlined below for your success, term, and over-quota redirects:

https://util.fuelcyclecore.com/util/surveyGizmo/redirectProxy

Success Redirect Configuration

• URL Variable:
  • Under “URL Variable” put: redirect_url
  • Under “Variable Name” put: redirectUrl
• Static Value:
  • Under “Variable Name” put: qualified
  • Under “Default Value (opt)” put: true

Failure/Overquota Redirect Configuration:

• URL Variable
  • Under “URL Variable” put: redirect_url
  • Under “Variable Name” put: redirectUrl
• Static Value 1
  • Under “Variable Name” put: qualified
  • Under “Default Value (opt)” put: false
• Static Value 2
  • Under “Variable Name” put: reason
  • Under “Default Value (opt)” put: LOGIC or OVER_QUOTA
• Static Value 3
  • Under “Variable Name” put: disqualifiedStep
  • Under “Default Value (opt)” put: {customText}

The red text portion in the Failure and Over Quota Redirects can be changed so that you can track disqualification numbers through the FC platform dashboards. These designations should not include spaces and typically should be written in all caps. (e.g. HOUSEHOLD_INCOME)

Duplicate Protection

Duplicate protection can be accessed through Tools > Response Settings. The following duplicate protection settings should be used on Alchemer screeners:

• PANEL/NON-CRM RECRUIT: Enable duplicate protection on any panel and non-CRM recruits.
  • Use IP-based for non-employee communities
  • Use cookie-based for employee communities
    * Cookie-based is used for employee communities because a company’s VPN may cause multiple members to have similar IP addresses
• CRM RECRUIT: Disable all duplicate protection for a CRM recruit – no IP, no cookies

Profile Points

After programming the screener, it is recommended to import it as soon as possible so that the Community Manager can start creating Profile Points.  Please contact your Fuel Cycle account team if you would like a Profile Point template to better communicate which questions should be implemented as Profile Points. 

Quotas

Setting Quotas

In order to set up quotas, please refer to the link below that pertains to your survey tool:

• Qualtrics
• Alchemer

When setting up your quotas, you can deal with over-quotas in a couple of different ways. Setting quotas manually is the recommended method because it’s based on active true members and therefore is based on the most accurate community composition numbers. 

• Manually (count-based): Set quotas as a count (with no associated survey action when the quota is met) and manually set up certain subgroups in your survey to term as necessary
• Automatically: Set quotas to close automatically within survey flow when a maximum is reached

If setting quotas to automatically OQ respondents, it is always best to set your quotas slightly higher than your target community composition. The reasoning is that the number of screener completes will almost always be higher than the number of Active True members in the community. The recommended buffer is +10-20% per quota.

If using census-based quotas, refer to the US Census Bureau website for the demographic breakouts.

Setting Over Quotas Manually

When setting quotas, you will need to work with the Community Manager and Field Associate to keep track of Active True member counts. Once a quota has been met, you will need to program the over-quota point into the screener.

Make sure to pause recruitment (if possible) before programming the overquota point. Otherwise, respondents who enter the screener during programming will be taking an incomplete version of the screener.

Setting Over Quotas Automatically

When setting your over quotas to automatically close, you will need to be careful when setting your over-quota logic.

Below is an example of incorrect quota logic. In this example, once the All Canada quota is filled up then all respondents will start to term regardless of which quota they fill.

Below is an example of correctly set up quota logic. Based on this logic, respondents will term if they reside in Canada when the “All Canada” quota is met.

Screener Programming Checklist

Below is a quick checklist that should help you as you program and test your screener. Happy screening!

• Are redirects correctly in place? Are they configured correctly to track term rates?
• Are all terms programmed correctly?
• Are appropriate duplicate protection settings being used?
• Are quotas set up correctly?
• If recruiting via gen pop, have you discussed quotas and targeting with your Fuel Cycle account team?
• Are appropriate question types being used for profile points?
• If Javascript is being used, is it working correctly?

Appendix

Commonly Used Screener Demo Questions

Are you… [single-select]

• Female
• Male
• Prefer not to answer

What is your age range? [single-select]

• Under 18
• 18-24
• 25-34
• 35-44
• 45-54
• 55-64
• 65-74
• 75 or older
• Prefer not to answer

Sometimes we look for people who work in particular industries. Do you, or does anyone in your family work in the following industries? [multi-select]

• Advertising/Marketing [TERM]
• Entertainment
• Market Research [TERM]
• Online or Internet Commerce
• Discount or Department store
• Television / Radio / Cable
• None of the above [EXCLUSIVE]

In what state do you reside? [single-select]

• [list out states]
• I reside in a country other than the United States [TERM]

What is your ethnicity? [single-select or multi-select]

• Asian
• Native Hawaiian or other Pacific Islander
• Hispanic/Latino
• Black or African American
• Native American
• Caribbean Islander
• White/Caucasian
• Prefer not to identify
• Other (please specify) ____________________

*Quick note on the ethnicity question: If setting up census-based ethnicity quotas, it is preferable that the question is single-select with a separate question that asks if the respondent is Hispanic.

Which category best describes your total annual household income before taxes? [single-select]

• Less than $15,000
• $15,000 to $24,999
• $25,000 to $34,999
• $35,000 to $49,999
• $50,000 to $74,999
• $75,000 to $99,999
• $100,000 to $124,999
• $125,000 to $149,999
• $150,000 to $174,999
• $200,000 to $224,999
• $225,000 to $249,999
• $250,000 or more
• Prefer not to answer

Which of the following best describes your marital status? [single-select]

• Married/ Domestic Partner
• Widowed
• Divorced
• Separated
• Single/ Never Married
• Prefer not to answer

How many children (under 18) do you have living at home?

• None
• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• More than 10
• Prefer not to answer

Please indicate which of the following activities you’ve done on any of the electronic devices you own. [TERM if “None of these” or less than 4 answers are selected]

• “Liked” brands on social sites (e.g. Facebook, Twitter, etc.)
• Bought products from shopping apps
• Downloaded shopping apps to check out products and look for deals
• Received texts and messages from companies that have deals, coupons, and sale reminders
• Managed finances or bank accounts online or in an app
• Managed flight or travel reservations online or in an app
• Played games with others online or in an app
• Posted content on social networking sites (e.g. Facebook, Twitter, Instagram, Pinterest, etc.)
• Read or posted a blog
• Searched the internet for deals and coupons
• Searched the Internet for news and information
• Sent and/or received text messages
• Surfed the web on your smartphone
• Used an app or searched the internet while shopping in a physical store
• Watched online videos (e.g. YouTube)
• None of these

Sample Community Opt-In Text

In the Community Name, you will be part of a group of about X of our customers. Community members will learn from each other and provide feedback to Company that will guide the approach for engaging other member groups across the country.

We hope that you’ll be able to log in regularly to discuss topics related to INDUSTRY or INDUSTRY with your fellow community members and facilitators. For your time and participation, you will have a chance to [INCENTIVE PROGRAM]. You will also be able to opt out at any time.

Does this sound like a good fit for you and are you willing to become a member of the Community Name?

A) Yes
B) No (You will not become a member of the community) [TERM]