Dear {{first_name|Friend}} ,
We spent months building a model to route the right petition to the right person. Josh Nelson beat it nine days out of ten.
Josh helps run Civic Shout, an ad platform for progressive causes that specializes in email acquisition, which mails a petition a day to its active members. We ran a randomized controlled trial (RCT) with them to test whether per-member routing could outperform Josh’s daily editorial pick. Routing in our RCT turned out to be real and measurable: ranking the day's options against someone's preference profile lifts their clicks by around 18% and their signing rate by 10%. But the petition itself swings response much more than fit does, and Josh has excellent news judgment. TLDR: whether a story broke that morning matters more than whether it's a given subscriber’s preferred issue.
Last email, I argued that the power of content routing based on supporter preferences will mean writing more content. But there is another side of that coin, which is that we can use that same knowledge to send supporters slightly less content that is better suited to their preferences and get another lift in performance.
We took the RCT data and asked: Who won't sign today's petition?
Before each send, the idea is to flag the established members least likely to sign that particular petition, and then hold back the worst-matched 30%. Those audience members would sit that day out and come back the next one, when the topic changes. Civic Shout would still collect about 95% of the signatures those sends would have produced, because the people it skipped were the least likely to sign anyway.
But for 5% fewer signatures right away, Civic Shout would improve its list health and prevent 30% of its unsubscribes on a given email. That’s awesome, because a retained member is worth around 20 future signatures. Keeping a few more of them means the list a year from now is bigger than it would have been. We model the difference at about 300,000 additional signatures over a year. We’re calling this method strategic suppression.
Strategic suppression requires a really good model to choose whom to leave off a given day’s email send. We ran the same projection holding back a random 30% of the list, and it loses 1.2 million signatures.
We haven't run this as an RCT yet. Strategic suppression is a projection built on Civic Shout’s data and the content routing RCT we completed, and it needs a live run against a control group before anyone banks on it. We've proposed one to Civic Shout. One exciting dimension to a new RCT is we can see if the unsubscribe rate itself improves, because we can see fewer unsubscribes on the sends we'd cut in our modeling.
The fatigue conversation in our sector is always an argument about cadence: how many sends a month or how many appeals is too many in December. With strategic suppression, Civic Shout wouldn't slow down at all. The awesome team there would write the same petitions but get them in front of fewer of the wrong people.
I love the idea of reducing the amount of emails in people’s inboxes – especially when they’re not likely to act on those emails anyway. A big reason for the handwringing about email is well-founded. We’re all receiving too much stuff we find irrelevant, or at best milquetoast, and it conditions our supporters to tune out our sends. The promising scenario would be sending only highly-relevant, interesting messages that spark excitement. I know from working with Chorus’s amazing partners that we have a surplus of that! Now let’s get it to the right people.
Thanks,
Sam
P.S. Want to see what your own list looks like through this lens? Reply to this email or book a walkthrough.