Base·Note Data

Blog · OOH · B2C · Geo testing · DMA

OOH measurement for B2C: how to pick your test DMAs

How to choose test and control DMAs for a B2C out-of-home geo test, using public data to screen markets and your own data to match them.

The hardest part of measuring B2C out-of-home is choosing the markets. Pick well and a simple geo test gives a clean answer. Pick badly and no model can rescue it.

Why geography is the unit of measurement

You can't track a person from a billboard to a purchase, so you measure places instead. The standard design is a geo experiment: run OOH in some markets, hold it out of comparable markets, and compare what happens to signups, installs, sales or whatever outcome you care about.

In the US, the usual unit is the DMA (designated market area), the 210 regions Nielsen uses to define local media markets. Most ad platforms, TV buying and OOH vendors already work in DMAs, and you can map your own customers to them by zip code.

The design is simple. The hard part is picking which DMAs go in the test and which go in the control. That choice decides whether you get a clear answer or a noisy shrug, so it's where I'd spend the most time. (For how OOH differs for B2B brands, see OOH measurement for B2B.)

Step 1: screen all 210 DMAs with public data

Before you look at your own numbers, narrow the field using data anyone can get. The goal here is feasibility and representativeness: which markets could host a clean test, and which look like the customers you want to reach?

SignalPublic sourceWhy it matters
Population and market sizeCensus Bureau population estimatesToo small and your outcomes are noisy. Too large and the buy gets expensive
Target demographics (age, income, household type, education)Census American Community SurveyKeeps test markets representative of your real customer
Commuting patternsCensus LEHD commuting dataOOH reaches commuters, not just residents. Markets with heavy cross-DMA commuting leak exposure into your control
Category and brand interest by marketGoogle Trends by DMAA free read on baseline demand for your category and brand
Weather and seasonalityNOAA climate dataKeeps a snowstorm or heat wave from driving a difference you'll misread as lift
Local events and holidaysPublic event calendarsBig sports, festivals or elections in one market and not its match will distort results
OOH inventory and costVendor rate cards and availabilityNot public data in the strict sense, but you can't test where you can't buy

Two more filters belong here. Drop any DMA where you're running something else that would contaminate the test, such as a local promotion, a new store opening or a TV flight. And be careful with the biggest markets. New York and Los Angeles are so unlike anywhere else that they rarely have a good match, so they're usually better left out of a first test.

What you want at the end of this step is a shortlist of roughly 30 to 60 DMAs that are buyable, clean and representative.

Step 2: pick matches using your own data

Public data tells you which markets could work. Your owned data tells you which ones will actually behave alike, and that's what makes a control market a good control.

Pull weekly outcomes by DMA, using customer zip codes mapped to DMAs. Aim for at least 52 weeks, ideally two years so you can see seasonality. Then look at four things:

  • Pre-period similarity. For each candidate pair, how closely do their weekly outcomes track each other before the test? High correlation and a similar level are what you're after. Markets that rise and fall together are the ones where a gap afterward means something.
  • Volatility. Check the week-to-week noise in each market. A market with wild swings needs a huge effect to stand out, so it's a poor test market even if it looks perfect on paper.
  • Share of your business. Make sure the test markets add up to enough of your customer base to matter, and that no single market dominates the result.
  • Overlap with other marketing. Look at your existing media and promotion by DMA, especially TV, paid social with geo targeting and any local partnerships. A match is only a match if both markets have been getting similar treatment.

Then run a power analysis. Simulate the test using your historical data: if OOH truly lifts outcomes by 3%, 5% or 10%, how often would this design detect it, and how many weeks would you need? If the answer is "almost never at 3%," you have two choices. Add more test markets, or accept that you can only detect a larger effect. Better to find that out now than after you've spent the budget.

Putting it together

Here's the whole process in order:

  1. Define the question and outcome. What decision will this test inform, and which metric will you judge it on? Pick one primary outcome, such as new customers, app installs or revenue.
  2. Start with all 210 DMAs and map your customers to them by zip code.
  3. Screen with public data. Remove markets that are too small, too big or too different from your target customer, plus any with commuting leakage, contamination or no buyable inventory.
  4. Join your owned data. Build a weekly outcome series for each remaining DMA.
  5. Match markets. For each candidate test market, find the control markets (or a weighted blend of them) whose pre-period outcomes track it most closely. Synthetic control is a good option here because it builds the comparison from several markets instead of relying on one perfect twin.
  6. Run the power analysis to see how large an effect you can detect and how many weeks you need. Adjust the number of test markets or the length of the flight until the design can answer your question.
  7. Check the final picks. Make sure the test markets aren't all one region or one type of city, and that nothing is scheduled there that would muddy the result.
  8. Lock the plan. Write down the markets, dates, metric, analysis method and decision rule before the flight starts.

One trade-off to keep in mind: the markets that match best aren't always the ones most like your whole customer base. If your test markets are small and unusual, you'll get a clean read that doesn't generalize. When that happens, I'd choose a slightly worse match in a more representative market, and say so in the readout.

Reading the results and feeding your MMM

When the flight ends, compare test and control markets against their pre-period trend. The usual methods are difference-in-differences, which compares the change in test markets to the change in controls, or synthetic control, which compares test markets to a weighted blend of controls. Both give you an estimate of the lift and a range around it.

Turn that into something finance will recognize: incremental conversions, cost per incremental conversion and incremental ROAS. Report the range, not just the midpoint, and say what the test could and couldn't detect.

Then put it to work. The test result is a clean, causal read on one channel, so use it to calibrate your marketing mix model. If the MMM says OOH drives much more or less than your test showed, that's a sign the model needs a tighter prior or more data. And if the test is inconclusive, that's still a finding, because it tells you how big the effect would have to be to matter.

Want help choosing markets for an upcoming flight? Get in touch, or take a look at the Measurement Audit.

More posts Services and pricing

Work with me

Want a second set of eyes on your measurement?

I help marketing and growth teams figure out what their spend is actually driving, with MMM, incrementality testing and attribution that finance trusts too.

Book a 30-min call