Hi SurveyEngine! We’ve gathered 1,125 responses for our DCE on specialists’ deployment preferences. We intended to have 125 respondents for each of the 9 specialties, but unfortunately, some specialties did not reach the target, while some exceeded. I’m curious as to how I can generate a model that reflects differing weights? Would appreciate your guidance!
This is potentially unmodelable by speciality as it´s possible that some of the strata (specialities) are missing key experiment profiles and will simply not converge. Weights may not be able to help.
What should have happened is that strata controls should have been used in the setup. At this stage before you try to correct with weights, you should check your data. A simple crosstab of design row by speciality will show if you have any holes in the data.
if your sample size is low, this is highly likely unless you have used strata controls.
if you have at least 10 respondents per block per speciality you should be fine. then it depends on how you are modelling the speciality. Perhaps a followup here.
Lets go through a worked out example in the case its not. assuming you had 10 scenarios per respondent and (say) 4 blocks. (a 40 row design). As SurveyEngine balances the block allocations, this would mean 4 respondents would provide one full replication (every profile seen). 40 respondents would mean 10 replications / which is just modellable.
Now randomly assigning those to the 9 specialities to sets of 4 blocks. The chance that just one of those specialities is missing or deficit a block is almost certain.
Here is a simulation run
| Speciality | 1 | 2 | 3 | 4 | Total |
|---|---|---|---|---|---|
| 1 | 5 | 3 | 8 | 3 | 19 |
| 2 | 4 | 3 | 6 | 13 | |
| 3 | 6 | 4 | 6 | 8 | 24 |
| 4 | 4 | 2 | 2 | 3 | 11 |
| 5 | 4 | 4 | 4 | 3 | 15 |
| 6 | 2 | 4 | 3 | 1 | 10 |
| 7 | 2 | 4 | 4 | 1 | 11 |
| 8 | 4 | 4 | 2 | 2 | 12 |
| 9 | 1 | 3 | 2 | 4 | 10 |
| Grand Total | 32 | 31 | 31 | 31 | 125 |
Note how although blocks are very well balanced and specialities seem to have enough sample, speciality 3 has zero joint observations of block 3. This may mean this is unmodellable.
Here’s how it should be setup as I suspect you’ll need to go back to field.
Use the speciality as the segment (this is a SurveyEngine object)
For all choice sets after that, SurveyEngine will maintain an independent allocation deck by segment and make random without replacement assignments of blocks.
Then the same data will look like this.
| Speciality | 1 | 2 | 3 | 4 | Grand Total |
|---|---|---|---|---|---|
| 1 | 4 | 4 | 4 | 5 | 17 |
| 2 | 4 | 4 | 4 | 5 | 17 |
| 3 | 4 | 4 | 4 | 5 | 17 |
| 4 | 4 | 4 | 4 | 3 | 15 |
| 5 | 4 | 4 | 4 | 5 | 17 |
| 6 | 4 | 3 | 3 | 5 | 15 |
| 7 | 2 | 2 | 2 | 3 | 9 |
| 8 | 2 | 2 | 2 | 3 | 9 |
| 9 | 2 | 2 | 2 | 3 | 9 |
| Grand Total | 30 | 29 | 29 | 37 | 125 |
It’s still pretty weak because of the sample size, but you won’t have colinearity.