Email tests for marketplaces
The highest value lifecycle tests in a marketplace are the ones that move liquidity: steering demand toward under-supplied categories, getting a new seller to a first complete listing, and turning a first transaction into a habit. Transactional messages around matches, bookings and payouts carry more weight than campaigns, and every read has to check both sides of the market.
- Why marketplace testing has to read both sides at once
- Test ideas for supply, demand and the transaction itself
- The tests most teams skip, and how to run one properly
- Guard metrics, two-sided constraints and graded evidence
Why marketplace lifecycle testing is different
A marketplace has two audiences and one goal, and lifecycle messaging is where the two sides meet. That makes testing harder than it looks: a change that lifts demand can quietly drain supply, and a read that ignores the other side of the market is not a read at all.
Liquidity is the thesis behind almost every worthwhile marketplace test. Steering demand toward under-supplied categories, getting a new seller to a first complete listing, and turning one transaction into a habit all move the same underlying number.
Transactional messaging carries more weight here than in most verticals, because matches, bookings, messages and payouts are the moments people actually care about, and they are usually shipped by engineering and never revisited.
Two spines, one liquidity goal
Demand side
Supply side
Lifecycle test ideas by stage
Supply acquisition
- Test a first-listing sequence that asks for one field at a time against one that asks for the full listing.
- Test showing demand data for the seller's category as the reason to finish.
- Test the timing of the reminder to complete a half-finished listing.
Demand acquisition
- Test steering new buyers toward categories with spare supply against showing the most popular categories.
- Test a saved search or alert prompt in the first week.
- Test how much the first message asks the buyer to specify.
First transaction
- Test reminders on an abandoned booking or checkout, including how soon the first one lands.
- Test surfacing trust signals such as reviews or guarantees at the point of hesitation.
- Test a nudge to the other side of a stalled conversation.
Repeat and habit
- Test a relevance-based alert against a scheduled digest.
- Test post-transaction prompts that suggest the natural next booking or purchase.
- Test asking for a review at different points after completion.
Retention on the supply side
- Test a performance summary that tells a seller something actionable.
- Test a nudge when a listing goes stale against leaving it alone.
- Test payout and earnings messages as a retention surface.
Transactional
- Test clarity in match, booking and payout notifications, read on completion rather than opens.
- Test how quickly the other side is notified of a message or request.
- Test one contextual next step inside the transactional message.
The highest-value tests most marketplace teams skip
Marketplace teams tend to test the demand side because it is easier to measure. The value below usually goes unclaimed.
The supply side inbox
Seller onboarding, stale listing nudges and payout messages get a fraction of the attention that buyer campaigns receive, and they gate the whole market.
Notification timing
How fast the other side hears about a message or request is a lifecycle decision owned by engineering, and it is testable.
Steering rather than promoting
Pointing demand at under-supplied categories is worth more than another discovery campaign, and it rarely appears on the roadmap.
The stalled conversation
Threads that go quiet between the two sides are a large, mostly untouched recovery surface.
How to run a marketplace lifecycle test
Write the test as a card before you write the copy. One audience, one change, one primary metric, and a guard metric you agree to respect. If the card cannot be written in three lines, the test is really two tests.
Set the sample size and the read date before the test ships, not after you have seen the first day of data. If the cell cannot reach the sample you need within a sensible window, change the test rather than the standard: pick a lever with a bigger expected effect, or widen the audience.
In a marketplace, name the other side of the market in the card. Every test needs a metric on the side it is not aimed at, or you cannot tell a liquidity gain from a transfer.
A test card template for marketplace
- Hypothesis
- For [side of the market], changing [one element] will improve [primary metric] because [insight from a test you have read].
- Success criteria
- A relative lift on the primary metric that beats your minimum detectable effect, read per side of the market.
- Guard metric
- The other side of the market is unaffected: no drop in listings, bookings or response rates outside the test cell.
Free to copy and use in your own program. Fill the brackets from a test you have read in the library.
Guard metrics: protecting trust while you test
A marketplace test can lift one side while quietly costing the other. Read these together.
- Match or fill rate, not just requests created
- Supply-side activity: new listings, response rate and churn
- Cancellation and dispute rate on the transactions produced
- Notification unsubscribes on both sides
- Repeat rate rather than first transaction rate alone
Testing across two sides of a market
The structural constraint is that the two sides interact. Randomising buyers alone can spill onto sellers, so where the effect is likely to cross over, split by market, city or category rather than by user.
Liquidity also varies enormously by category and geography, which means a result from a dense category rarely transfers to a thin one. Read tests per segment where you can, and be explicit about which market the learning describes.
Transactional and notification changes usually need engineering time, so batch them and treat the notification layer as a roadmap item rather than a marketing request.
Evidence from real marketplace programs
The library holds 7 graded tests run by marketplace teams: 7 report a win for the variant, 0 a loss, and 0 no clear difference.
- First-name in welcome subject line lifts opens
Personalizing the welcome subject with the guest's first name lifted opens an exact amount with no downstream drag. The win compounded into a small first-booking uplift.
Grade AVariant wonAirbnb2019 - Location-based mobile promotions drive 6-12x purchases
Location-triggered push drove large same-day and 12-day-delayed purchase lifts, and measuring only the immediate response undercounts the true effect by half.
Grade AVariant won - Pattern-interrupt win-back email, no discount, at Getir
Creative novelty beat discounting for 30-day+ churned users, and it protected margin by winning them back at full price.
Grade BVariant wonGetir2025 - Grubhub Campus staged welcome stream vs holdout
Measuring a welcome flow against a true control group, not against opens, is what lets you claim incremental activation.
Grade BVariant wonGrubhub - Behavior-triggered omnichannel win-back at MPL
Whether a user won or lost their first session is a usable churn predictor, letting reactivation messaging fork before the user fully lapses.
Grade CVariant wonMobile Premier League2025 - BigBasket AI-personalized dormant-user reactivation
At grocery-app scale, personalization can reactivate roughly a fifth of dormant users, but a vendor pre-post gives no read on incrementality.
Grade CVariant wonBigBasket2025 - Switch-and-Save post-trip re-engagement at Ibotta
Behavior-triggered timing (24h post-trip) plus dynamic content is the reusable idea here, even though the case reports no hard delta.
Grade CVariant wonIbotta2025
Direction, grade and source are free. Exact figures open up once you sign in.
What a winning marketplace test looks like
A winning marketplace test raises completed transactions or fill rate at the pre-set sample, without shrinking activity on the other side of the market. Demand growth against flat supply is a queue, not a win.
Read direction before magnitude. Liquidity conditions differ so much between categories that the transferable part of another team's result is the lever, not the number.
Frequently asked questions
What should a marketplace team test first?
Seller onboarding to a first complete listing, if supply is the constraint, and search to first transaction if demand is. Identify which side limits liquidity before choosing the test, because optimising the abundant side changes nothing.
How do you test without hurting the other side of the market?
Make the other side a guard metric. Every test card should name what must not move: listings, bookings or response rates on the side you are not testing.
Are review requests worth testing?
Yes, mainly on timing. Review asks that land after the value is delivered behave very differently from ones that arrive with the transaction, and review volume feeds trust for both sides.
Do marketplace email programs still work in 2026?
The library holds 7 graded tests here, and 7 of them report a win for the variant against 0 losses and 0 with no clear difference. That is enough to say the direction still holds, and not enough to promise a number for your own program.
What is the most credible marketplace test in the library?
First-name in welcome subject line lifts opens. Personalizing the welcome subject with the guest's first name lifted opens an exact amount with no downstream drag. The win compounded into a small first-booking uplift.
What tends to fail in marketplace email programs?
No losing test has been recorded in this cell yet, which is a gap rather than a signal. Treat the wins here as directional only.
How fresh are these marketplace learnings?
The newest record in this cell is from 2025 and the oldest from 2019. 0 records have been reviewed and vouched for by the CacheMagpie editor.