The Sample publisher report

I spent a good deal of time last week paralyzed by thoughts of strategy/”what direction should we take The Sample in.” My conclusion was that a large part of me still wanted to build a product for myself, and I think in our specific situation that would hold us back. Building a product for yourself can be a useful way to make sure you don’t build something that no one wants—if one person genuinely gets value from it, there are probably others who will too—however The Sample already has real users and stuff, and it appears that with a bit more optimization our Facebook-to-Substack advertising funnel will be profitable. So, I’m trying really hard to put my blinders on and just focus on shipping stuff in the short term.

About that “Facebook-to-Substack” thing: I just ran a relevant query. In the past two weeks we’ve had $1,608 in revenue, and it’s come from publishers (with paid forwards enabled) on the following platforms:

• Substack: 66.5%
• Ghost: 13.6% (13.0% on Ghost(Pro), 0.6% self-hosted)
• Revue: 7.6%
• ConvertKit: 2.9%
• Mailerlite: 1.6%
• Mailchimp: 0.8%
• Other: 7.0% (I’ve only added platform-detection code for a few platforms)

So... yeah. Anyway, my cofounder and I did a little planning powwow today and our todo list looks roughly like this (order isn’t exact since we’re working concurrently):

1. Add a line to the emails, underneath the “subscribe in 1 click” and “show more/less like this” buttons, which says “Not sure yet? [Read another issue] from this newsletter.” It’ll lead to a web page which displays the newsletter’s previous issue in the same format as The Sample’s regular emails. You can keep clicking the “Read another issue” link to see up to three previous issues. (I've almost finished this).
2. Make the “subscribe in 1 click” and “show more/less like this” buttons lead to another web page which shows a list of all the other newsletters we’ve recommended previously, instead of the current mostly blank “we’ve received your feedback” pages. (Also mostly finished).
3. A/B test: try sending people a list of five newsletters twice per week and see if it results in more 1-click subscribes than the current 1-issue-per-day thing. (Each newsletter in the list would lead to the same web page described in #1). Play around with different default frequencies and list sizes. We’ll probably need to run this test for a while since part of the hypothesis is that it’ll improve long-term retention, even if it results in less engagement up front.
4. Various publisher console improvements as discussed previously.
5. Conclude some existing algorithm A/B tests we’ve been running for a while.
6. Make the algorithm understand which users came from our Facebook ad and which are organic. (Improving performance for our Facebook users is what matters most for our future growth, and it’s likely that cohort tends to have different preferences from our organic users).
7. Clean the subscriber regularly to decrease the odds that we start going to Gmail’s promotion tab. This has been in my head for a while since a lot of the subscribers we get from Facebook just don’t engage with our emails at all. (Part of our Facebook-to-Substack funnel is filtering for people who do open newsletters and click links inside them). So we don’t want to accumulate too many inactive subscribers.
8. Improve our internal usage report so we have a better understanding of how our Facebook ad’s performance is changing over time.
9. (Maybe) make some changes to improve the algorithm’s performance for Facebook users who have not yet rated any newsletters we’ve sent them or selected any interest topics.
10. Do some more experiments with the Facebook ad itself (e.g. different images or text, that sort of thing).

There are plenty of other things we could do after this list is done, some of which I’ve discussed, but I’m making a Solemn Promise to stop getting distracted by “strategic” thoughts, and thus I will Most Definitely Not talk about any of those ideas further until we’re actually ready to start working on them.

Jacob