SMS marketing tests and benchmarks
SMS has no room for filler, so the tests that matter are about restraint and relevance. Who is eligible, how often you send, and whether the message is worth the interruption decide opt-out rate and revenue per message far more than copy does. Every SMS test should be read with cost and opt-outs next to the result.
- Every testable surface in an SMS program, from eligibility to link
- Test ideas for each lifecycle stage, linked to the matching stage page
- Consent, cost and length constraints that shape what you can run
- How fast SMS learnings fade, and graded evidence from real programs
What is actually testable in SMS
There is very little inside an SMS to change, which makes the surrounding decisions the real test surface.
Eligibility and suppression
Who is on the list, how they joined, and who is excluded from this send. Consent quality is the single biggest driver of results in this channel.
Frequency
How many messages a contact can receive in a period. This is the variable most programs get wrong and the one most worth testing downwards.
Trigger and timing
Event triggered against scheduled, and the hour of the day. Timing errors here produce complaints rather than lower open rates.
Length and structure
One sentence against two, whether the brand name leads, where the link sits, and whether a message needs a link at all.
Offer and reason
The reason the interruption is justified: a real event, a genuine deadline, or an offer. Generic promotions are the fastest route to opt-outs.
Reply handling
Whether the number accepts replies, and what happens when someone uses one. Two way conversation is testable and rarely tested.
The anatomy of a testable SMS
Consent source, recency, and who is suppressed.
Real event or schedule.
Local time, quiet hours respected.
Brand identification and the reason.
One sentence or two, with or without an offer.
Present or absent, deep link or landing page.
Wording and placement of the stop instruction.
One way broadcast or a monitored two way number.
Every slot above is a variable you can hold constant or change. A clean SMS test moves one of them at a time.
SMS test ideas by lifecycle stage
Each stage below has its own page with the full set of levers, guard metrics and graded evidence.
Cart abandonment
See the cart abandonment page (3 tests here)- Test SMS as the first recovery touch against email first, read on incremental recovery.
- Test a reminder with no offer against one with an offer, with opt-outs in the read.
- Test a lower monthly frequency cap against the current one, read on revenue per subscriber.
- Test event triggered messages against scheduled campaigns.
Welcome
See the welcome page- Test what the first SMS after opt-in does: deliver the promised thing, or set expectations for cadence.
- Test collecting a preference in the first exchange on a two way number.
- Test a single SMS touch against a sequence, with complaint and opt-out rate as primary reads.
- Test suppressing SMS entirely for contacts who went quiet on it.
Constraints that decide what you can test in SMS
SMS is the most regulated and most expensive lifecycle channel per message, and both facts shape what is worth testing.
- Explicit consent is required in most markets, and the consent source predicts performance more than any creative choice.
- Quiet hours and local time rules restrict when a test can run, so send time tests need a defensible window.
- Every message has a real unit cost, so revenue per message sent is the honest read, not revenue per click.
- Segment length limits change the cost of a message, which makes brevity a commercial variable rather than a stylistic one.
- Opt-out is permanent and instant, so a frequency test spends an asset you cannot rebuild.
How fast SMS learnings fade
SMS learnings about consent, frequency and relevance are durable, because they describe tolerance for interruption rather than a format trend.
Novelty effects are strong and short lived. A program that has just launched SMS will see results it cannot reproduce a year later, once subscribers stop treating each message as unusual.
Anything tied to carrier filtering or local regulation should be re-checked rather than inherited, since the rules move faster than the learnings.
SMS burns novelty fast. A new alert style or a first time discount tends to read strong once and weaker on repeat, so treat older SMS reads as a starting point rather than a settled answer.
Format learnings from the corpus
- Character limits force one idea per message, and tests that try to carry two tend to lose.
- Send time sensitivity is higher than in email because the message arrives with a sound.
How to run an SMS test
Randomise on the subscriber, keep the eligibility rule identical across arms, and read revenue per message sent alongside opt-out rate. An SMS test that ignores cost and opt-outs has not produced a result.
Set the sample size and the read date before the test ships, not after you have seen the first hour of data. If the audience cannot reach the sample you need inside a sensible window, change the test rather than the standard.
A test card template for SMS
- Hypothesis
- Because subscribers who joined through [source] tolerate [frequency], changing [one variable] from [control] to [variant] will increase revenue per message sent within [window].
- Success criteria
- Revenue per message sent, measured on everyone assigned.
- Guard metric
- Opt-out rate and complaint rate per send.
Free to copy and use in your own program. Fill the brackets from a test you have read in the library.
Guard metrics: what a win must not cost
Opt-outs are permanent, so they belong next to every SMS result.
- Opt-out rate per send and cumulative over the test period.
- Cost per incremental order, since message cost is real.
- Complaint and carrier filtering signals.
- Subscriber list size at the end of the period.
Evidence from real SMS programs
The library holds 6 graded tests run by SMS teams: 6 report a win for the variant, 0 a loss, and 0 no clear difference.
- Behavioral text + email nudges for aid renewal (FAFSA)
Loss-aversion and plan-making SMS/email nudges lifted deadline renewals, and the paper shows targeting the right students matters as much as the nudge itself.
Grade AVariant wonCUNY / ideas422023 - SMS-plus-email vs email-only win-back workflow
Layering SMS onto email win-back reaches the lapsed user on the channel they have not been ignoring, though this figure is a cross-study average not one test.
Grade CVariant wonAggregate2026 - Adding SMS between cart email 1 and 2
SMS layered onto the cart flow is repeatedly cited as the single biggest recovery lever after basic setup, in the 30an exact amount range.
Grade CVariant wonAggregate2026 - Non-price retention levers vs discounts by segment
Matching the lever to the segment (credits for high-value, discounts only for price-responsive) recovers churn without training the whole base to expect discounts.
Grade CVariant wonAggregate2026 - Frances Valentine email-plus-SMS on high-price items
For considered purchases, a second-channel SMS nudge after email helped convert buyers who do not buy on impulse.
Grade CVariant wonFrances Valentine2025 - Lush hybrid email-plus-SMS high-intent flows
Targeting SMS only at the highest-intent moments (checkout, restock) rather than everywhere is where the channel earns its cost.
Grade CVariant wonLush2025
Direction, grade and source are free. Exact figures open up once you sign in.
What a winning SMS test looks like
A winning SMS test increases revenue per message sent without raising opt-outs, and it still wins after you subtract the cost of the sends.
The most durable winners reduce volume and improve targeting. Wins built on a new offer type usually fade as subscribers learn the rhythm.
Frequently asked questions
Does SMS messages still work in 2026?
The library holds 6 graded tests here, and 6 of them report a win for the variant against 0 losses and 0 with no clear difference. That is enough to say the direction still holds, and not enough to promise a number for your own program.
What is the most credible sms test in the library?
Behavioral text + email nudges for aid renewal (FAFSA). Loss-aversion and plan-making SMS/email nudges lifted deadline renewals, and the paper shows targeting the right students matters as much as the nudge itself.
What tends to fail in SMS messages?
No losing test has been recorded in this cell yet, which is a gap rather than a signal. Treat the wins here as directional only.
How fresh are these sms learnings?
The newest record in this cell is from 2026 and the oldest from 2023. 0 records have been reviewed and vouched for by the CacheMagpie editor.