Test one thing, and know which thing#
Variants are only worth running when the difference between them is a single decision: this subject or that one, this opening or that one, an ask for a call or an ask for a reply. Two entirely different emails will produce a winner and no knowledge, because nothing about the result transfers to the next test.
The corollary is that the change should be big enough to matter. Two subject lines differing by a word will not separate at the volumes outbound actually runs at.
Assignment has to be sticky#
A contact assigned to variant A at step three must stay on the A path for the rest of that enrollment. Reassigning on each send produces people who receive a mixture, which makes the result meaningless and the experience incoherent.
Variants also belong outside the timeline. They are alternatives to a step, not additional steps — a model that treats them as positions produces a sequence that reads as having twice as many touches as it does.
Be honest about the sample#
Outbound volumes are small. A step sent to two hundred people, split in half, producing six replies against four, is not a result — it is noise with a winner painted on it. At realistic volumes a variant test needs weeks, and many differences will never separate at all.
Which is a reason to test the things that move the number most — the audience and the reason for writing — and to treat copy variants as a slow, secondary refinement rather than the main lever.
Step variant in SalesShift#
Variants in SalesShift are alternative versions attached to a parent email step, weighted, and sticky for the duration of an enrollment. They are never walked as separate timeline positions, so a two-variant step is still one touch in the sequence.
Further reading#
See it running
Signals, prospect search, sequences, deliverability and pipeline on one record.