A stylized image of for the 3:03 Podcast by Reply Two with large text '3:03' and the Reply Two logomark in the lower right corner.

The 3:03 Podcast

Where a couple of email nerds help creators crush it.

Listen

WTH is A/B Testing?

Mon, April 28, 2025

Episode 9: A Creator's guide to A/B testing. In this episode we tackle that gut-punch feeling when your carefully crafted newsletter disappears into subscribers' inboxes without a trace. Learn how to shift from guesswork to data-driven strategies that work.

So, this one is really all about answering the questions: "What is A/B Testing?" and "How to get started with A/B Testing?".

We'll cover which elements to test so you can understand what truly resonates with readers. Plus, discover why focusing your tests (and avoiding common mistakes) leads to those compound gains that separate growing newsletters from stagnant ones. Featuring real-world examples where tiny changes delivered major engagement boosts.

Timestamps:

  • 00:00 The newsletter black hole phenomenon
  • 00:55 Why data beats gut feelings every time
  • 01:24 Split testing explained (without the jargon)
  • 03:15 Real creators, real results
  • 05:13 What elements actually move the needle
  • 09:42 The "test one thing" rule you can't ignore
  • 12:39 How most creators shoot themselves in the foot
  • 13:54 Small gains → big results (compound interest for newsletters)
  • 15:30 Your first test: simpler than you think
  • 16:46 From spreadsheet tears to actionable insights

Some free Reply Two resources available 24/7

Shout out to Dan Oshinsky and Inbox Collective for their insights: https://inboxcollective.com/a-list-of-things-you-can-a-b-test/

About

Hosts Mai and Brai share the expensive mistakes to avoid and some secret flexes. A hooman x AI collab powered by Notebook LM.

Hosted by