Block allocation in a DCE: national survey (N=3000) and deliberative focus groups (N=60)”

Support
1

Hello,

I am designing a Discrete Choice Experiment (DCE) on policies to mitigate environmental injustices in France. I would like your advice on how to handle block allocation between a national survey and deliberative focus groups.

  • Number of attributes: 7

    1. Air quality (4 levels: very degraded, degraded, medium, good)

    2. Walking distance to the nearest green space (4 levels: <5 min, 5–15 min, 16–30 min, >30 min)

    3. Energy performance of the dwelling (DPE, 4 levels: A, B, C, D)

    4. Rent (tenants) or price per m² (owners) (5 continuous levels: +5%, +10%, +15%, +20%, +25%)

    5. Citizen participation (3 levels: consultation, co-management, opposable right)

    6. Information on environmental inequalities (2 levels: yes, no)

    7. Annual local tax (5 continuous levels: €55, €110, €165, €220, €275)

  • Parameters to estimate: around 11 main parameters, with some continuous (rent, tax) and others categorical (air quality, distance, DPE, participation, information).

  • Design: unlabeled, including a status quo alternative.

  • Samples:

    • National survey: N≈3000 respondents, 6–8 choice tasks each → global design but we need the same DCE for the focus group

    • Focus groups: N≈60 participants (6 workshops of 10 people), with pre/post deliberation.

My question:
For the focus groups, would it be better to:

  1. Give all 60 participants the same block, or

  2. Assign different blocks drawn from the same master design (e.g., 6 blocks × 8 tasks, one block per workshop), even if this means only about 10 respondents per block?

I am concerned about:

  • The statistical and econometric validity of splitting only 60 participants across several very small blocks (~10 per block).

  • The consequences for D-efficiency and comparability if the focus groups do not use the same blocks as the national survey.

  • Best practice: should the focus groups always reuse the exact blocks from the national survey, or is it defensible to rely on just one block for all participants?

Thank you very much for your insights,

2

That is an interesting one.

Without the focus group, this is easy to answer. Normally, the design would be blocked -although blocking has its origins in the practicalities of pre-internet days when DCEs were administered as physical “blocks” of printed surveys. Blocks are now used to control balance and dominance. Without proper blocking, it’s possible that a respondent receives a sequence of profiles where one attribute doesn’t vary.

So, blocking is a good idea. Eight scenarios are standard; however, there is published research showing that respondents can handle many more.

Now to the focus groups.
It really depends on the purpose. If this is a cognitive pre-test, then it doesn’t really matter, as the objective is to measure (through moderation) whether the task is meaningful.

If you want to build models from the focus groups, feasibility will be constrained by your design size. You can determine this in SurveyEngine - even with a free account.
Assuming an unlabelled design with an opt-out and certain attributes being linear (distance, rent price, local tax) and no priors, I get an orthogonal design in 49 rows and a D-efficient design in 36 rows.

A solid plan, then, would be six blocks of nine tasks for both studies. This will mean that in your focus group, the sample will be blocked the same way. It will also mean that you’ll have ten replications of the design, which is enough to build a model on the focus groups.

An added advantage is that you could then build joint models for the focus group and the general population (Gen-Pop) and perform an LR test to verify whether the two samples behave the same way (or not).

So, to answer your questions and concerns:

  • Use six blocks of nine tasks.

  • Use the same blocking scheme for both samples.

  • Splitting a small sample: although we recommend 20 observations per row, 10 should be fine. In any case, you will be able to tell.

  • Consequences of comparability: it shouldn’t matter if there were different allocations - this happens all the time in RP data, in any case, use the same scheme.

  • Best practice: it depends on what you want to achieve with the focus groups, but using the same blocking scheme avoids this issue.

  • Bonus suggestion: consider an engineer’s risk/return and risk-mitigation perspective.

I would investigate a higher number of profiles - 36 rows is a very nice number, as it gives you a lot of block/scenario options that satisfy the equation Blocks × Profiles = 36,
e.g. 4×9, 9×4, 6×6, 12×3, 3×12, 18×2, and 2×18.

Do the Gen-Pop study first.

Run a split pilot on six tasks and 18 tasks - you’ll only need about 120 completes, which is very inexpensive. I would also invest in making the 18-task version user-friendly, with a few “rest spots” between the 1st, 6th, and 12th tasks.
Build and compare models from both, and look at the dropout rates and comments.
This will resolve a lot of questions at very low risk and cost.

  1. It will tell you whether 18 scenarios are too many or acceptable.
  2. It will validate the viability of the planned focus group size.
  3. It will support your eventual blocking decision.
  4. You’ll be able to compare (using the LR statistic) whether the joint models differ.

You could then proceed with the rest of the sample or start the focus groups or, if the models are not great, revise the design and resample another 120.

I hope this helps.

Ben