Introduction of attention check question within a multiple-choice question

Survey Building
1

I have run into a problem while programming the survey, specifically in the part where we introduce attention check question. I am attaching a screenshot of the question so you can see exactly what I mean.

Since our survey will be conducted through a paid panel, our approach with the panel provider is to allow only respondents who are paying attention while completing the survey to finish it. To ensure this, we plan to include a multiple-choice question where respondents must select one specific option in order to proceed with the survey, confirming their attentive participation and eligibility to receive compensation. The intention is to present a multiple-choice question with several alternatives, where one specific option acts as the attention check that participants must select in order to continue with the survey. The specific option is highlighted in gray in the screenshot below. I attempted to use the branch option, following suggestions from your forum, but it did not work well due to my limited programming knowledge.

Thank you in advance for your prompt reply.

Best regards,

Hristo

image
image931×1077 58 KB

2

Let’s answer this is 2 parts.

  1. How do do what you want
  2. pitfalls in your method.
  3. Suggestions

The best place to start is to look at the Dictionary so you can see how your question q18 actually produces multiple data points or columns
q18_1 (for Big supermarkets)
q18_2 (for Supermarkets)
…. etc.

Then for multiple choice there are 3 possible values (empty - not seen or answered, 1 for seen but not selected and 2 for selected).

So your test in pseudo code is only pass if at least one option is selected AND I am not a bot is selected.

The code for passing the bot test is and this should be attached to a branch that goes to the next question

(($q18_1==2) ||
($q18_2==2) ||
($q18_3==2) ||
($q18_4==2) ||
($q18_5==2) ||
($q18_6==2) ||
($q18_7==2) ||
($q18_8==2) ||
($q18_9==2) ||
($q18_10==2) ||
($q18_11==2)) && ($q18_12==2)

Then add a branch that goes to the screen out directly after. Don’t worry - the respondent will stay on the page if they don’e answer anything.

2. Pitfalls
This is not likely to do what you want as AI agents can be programmed to click this with a suitable prompt - and I suggest you check this by going into agent mode in ChatGPT and trying it.
Secondly you will lose some valid respondents.

3. Suggestions
AI Bots a re a real threat and they are clever enough to mimic humans. So you need several methods. both up front and post analysis.

Make the trap question unreadable by humans
Create an invisible or semi invisible optional question “Are you a bot?”. Bots don’t have eyes - they read html - so add a question called ‘bottrap’ and then branch all with the display condition simply ($bottrap ==1 || $bottrap ==2)

Then add a style element (click text element then click the </> codeview to paste the following

<style>

#se_item_bottrap {
   display: none; 
}

</style>

Some bots are pretty smart, perhaps better is to make is non invisible but merely opaque or unreadble by a human.

<style>

#se_item_bottrap {
   opacity:0.01; 
}

</style>

or very small.

<style>

#se_item_bottrap {
transform: scale(0.01);transform-origin: top left;
}

</style>

There is a whole article other methods here: Fraud Detection Toolkit – SurveyEngine GmbH

3

Thank you very much for your very helpful and detailed explanation. It was very clear and helped me understand how to solve the problem.

I really appreciate your time and support.

I have added one more question, under a new topic, related to the visualization of choice experiment cards on different screens. I would be very grateful if you could help me resolve this visualization issue promptly. I have explained the problem in my new tag.

Best regards,
Hristo