I am planning to use survey engine for a profile-case best-worst scaling survey based around a descriptive system with 7 attributes and 5 levels. I have been struggling to design this experiment properly using the online platform and I am looking for some help. Can anyone provide any advice on how to set up the experiment on the platform? Attached below are the levels and attributes and a print screen of the pilot BWS survey which shows how the survey should look.
Levels
Never
Almost never
Sometimes
Often
Always
Attributes
I get tired
I struggle to keep up when I am walking around with others
I avoid doing sports
I struggle to concentrate on my studies/work
I feel embarrassed shopping for clothes
I feel unhappy because I am unable to do the same things as others
Will,
I’ll chip in here. Firstly the columnar Best Worst columnar layout. This is actually my fault - I personally developed this layout with Jordan Louviere about 20 years ago - and have regretted it since. It breaks all the Left/Right Top /Bottom rules.
So unless you’re planning replication - I’d just use vertical options and have the best-Worst at the bottom.
Secondly - Best-Worst itself has problems we identified during online execution and in data collection.
Methodologically - Worst is not the inverse of Best
As with the layout - it breaks the implicit ‘User Experience’ within the survey. Presumably the respondent has been answering in the affirmative all the way through the survey: do you consent? Yes. Are you a student? YES etc. Which do you like best? which do you like second best?
Then suddenly they have to flip their brainsand answer in the negative.
The consequence of this is that you are most likely to get bad ‘worst’ responses.
I strongly suggest Best, second Best.
Finally the ‘profile case’ is just a DCE with multiple questions. This is simple to implement. Just set it up as a DCE with 7 attributes and 5 levels.
Here are the specifics
Create a DCE with 3 alternatives, 7 attributes each with 5 levels
Under Spec/Settings choose ‘Best - Worst’ or ‘Full ranking’ to do best-second best.
Generate an ‘orthogonal design’, ngene or upload
Put it in your survey.
For extra points - use the ‘progressive choice’ This makes it very easy for the user as each choice is hidden allowing the user to focus.
For even more extra points - why not make it 4 or 5 profiles - this will generate vastly more data for you than 2 options. 2 options generates 3 data preference data points. 5 options generates 9 response data points.
