I currently have an experimental design of 40 rows, 4 blocks (i.e., 10 choice tasks per respondent).
Before my DCE begins in my survey (i.e., before the respondents are presented with their 10 choice tasks), I have added an additional “practice” task, currently this is a random design row from my experimental design to help respondents become familiar with the choice tasks.
However, instead of a random design row here, I would like to use a custom choice task that is seen by all respondents. I would like it to have a dominant alternative to serve as a consistency check i.e., one alternative is clearly better than the other alternatives so I can double-check respondents are paying attention to the question or as an indication that they understand the task. I realise that this consistency check can come anywhere in the design (not just at the start or indeed as a ‘practice task’).
I think the way to implement this is to upload a new experimental design with the original 40 rows, adding one additional row at the bottom that specifies the dominant alternative (i.e., now 41 rows in total). Then, on the Edit > Pages > Experiment tab I can now add a “practice” choice task and specify that it is row 41 in my design (i.e., the dominant alternative). When I download the results from the DCE, I will be able to use the “RID” and “Design row” columns in the excel spreadsheet to identify answers that are the consistency check (i.e., not to confuse them with the main choice tasks).
My questions are:
(1) Is the way you would advise to add a consistency check?
(2) Will this interfere with my blocking and for the new dominant task (i.e., design row 41), would I need to implement a new block (i.e. block 5) when I upload the new design and would SurveyEngine know not to include this row in the main choice tasks (would I need to change anything else)?
Robin,
Regarding 1.
Two ways of doing this depending on how you want to manage this.
Your suggested way will work but the easiest is to duplicate your experiment, call it “dominance_check” and replace your experimental design with one or more clear dominant alternatives.
You can then simply ensure the (say) 3rd choice task is from this dominance experiment.
The tradeoff is really - what’s more likely to change - your experimental design or your attributes and levels. As you will need to repeat the above exercise if you make post pilot changes.
I would also differentiate between a
Dominance test and a
Consistency test
These are different.
A dominance test will verify that respondents are making rational decisions according to economic theory - which is that an alternative with the lowest cost + lowest risk + most benefits should always be chosen and if not there is something fishy going on.
A consistency test is testing whether the respondent makes the same choices when presented with the same choice task.
In clinical trials with DCE’s both are used - often 2 of each.
Regarding 2) - if you put your dominance tests in a separate experiment - your blocking scheme won’t need to change.
If you want to include consistency checks then just repeat an earlier choice task later in the set.
Robin,
Regarding 1.
Two ways of doing this depending on how you want to manage this.
Your suggested way will work but the easiest is to duplicate your experiment, call it “dominance_check” and replace your experimental design with one or more clear dominant alternatives.
You can then simply ensure the (say) 3rd choice task is from this dominance experiment.
The tradeoff is really - what’s more likely to change - your experimental design or your attributes and levels. As you will need to repeat the above exercise if you make post pilot changes.
I would also differentiate between a
Dominance test and a
Consistency test
These are different.
A dominance test will verify that respondents are making rational decisions according to economic theory - which is that an alternative with the lowest cost + lowest risk + most benefits should always be chosen and if not there is something fishy going on.
A consistency test is testing whether the respondent makes the same choices when presented with the same choice task.
In clinical trials with DCE’s both are used - often 2 of each.