If you’re a lifecycle marketer, it’s highly likely that your primary area of focus is in the inbox. You invest significant time and resources in boosting engagement and cultivating customer loyalty. But what happens when conversion rates start declining?
In our monthly webinar series, “Ask a Marketer About,” we’ve been sharing stories from Dreamers, Builders, and Makers in the Iterable community about how they use the Iterable platform to deliver joy to their customers.
We sat down with Naomi Eshleman, Sr. Lifecycle Marketing Manager at Rocksbox, to hear how her team tackled the challenge of declining conversions, leveraging Iterable to implement various testing and experimentation strategies.
Check out the full webinar, “How Rocksbox Rocks Subscription Through Testing & Experimentation” for all of Naomi’s best practices, and keep reading for our top takeaways.
Meet Our Marketer
If anyone can speak to the challenges of marketing, it’s Naomi. She’s spent 12+ years focused on growth and lifecycle marketing, often working on small and resource-strapped teams. She started working more closely with experimentation and testing in the past few years, bringing her expertise to jewelry subscription business Rocksbox.
Rocksbox encourages and diversifies jewelry discovery through rental and subscription models. Acquired by Signet Jewelers in 2021, Rocksbox offers its members access to boutique designers and coveted trends for only $21 per month.
When Naomi initiated Rocksbox’s testing and experimentation journey, it all started with defining KPIs.
The Challenge
To combat the challenge of declining conversions, Naomi began by asking four major questions:
- What is the business problem we’re trying to solve?
- Which KPI does it impact?
- Which customer segments is it prevalent with?
- What customer problems tie back to the business problem we’re trying to solve?
After conducting this exercise with her team, Naomi identified the following business problems: users were purchasing fewer items from their rentals, converting from a trial to a membership at a lower rate, or pausing their membership once subscribed.
Using the above four questions as a framework, Rocksbox mapped these three business problems to KPIs, segments, and customer problems.
| Business Problem | KPI | Segment | Customer Problem |
| Users purchasing fewer items from their rental set vs. historical trend | Purchase conversion | All members | Items that customers want to rent aren’t available |
| Low volume of members converting from a trial to a membership | Voluntary churn | Month-zero members | Customers aren’t excited about the items in their first box |
| High volume of users pausing their membership | Monthly active members | On-hold members | Customers are cutting their monthly expenses |
The Solution
With their biggest issues properly scoped out, Rocksbox could come up with ways to resolve them. Naomi brought together a diverse team to brainstorm solutions for both the business problems and customer problems.
Once all the ideas for solutions were captured, the team used the ICE framework (Impact, Confidence, Effort) to gauge their viability.
- Impact measured a solution’s value or magnitude to the business
- Confidence measured the team’s knowledge or experience in a given area
- Effort measured the time and resources it would take to achieve insights
Solutions that were high effort but low confidence were tagged as having high risk, while ones that were inversely low effort with high confidence were tagged as low risk.
By measuring each idea by these factors, Rocksbox was able to create a matrix with impact along the x-axis and risk along the y-axis, identifying four groups of potential solutions.
| Low Impact | High Impact | |
| Low Risk | Quick Wins | No Brainers |
| High Risk | Duds | Big Bets |
The goal for the lifecycle marketing team was to have a healthy mix of Quick Wins—easy ideas they could implement right away—and Big Bets—investments that would pay off massively if properly executed.
Once the best solutions were prioritized, Rocksbox could map them to their problem set.
| Business Problem | KPI | Segment | Customer Problem | Solution | ICE |
| Users purchasing fewer items from their rental set vs. historical trend | Purchase conversion | All members | Items that customers want to rent aren’t available | Send messages to users that items are available to rent right now | Impact: High Confidence: Low Effort: High Risk: Big Bet |
| Low volume of members converting from a trial to a membership | Voluntary churn | Month-zero members | Customers aren’t excited about the items in their first box | Drive more users to customize and approve their first box | Impact: High Confidence: High Effort: Medium Risk: No Brainer |
| High volume of users pausing their membership | Monthly active members | On-hold members | Customers are cutting their monthly expenses | Offer test incentives to unpause membership |
Impact: Medium |
Setting Up Tests
The benefit of mapping solutions to business problems is that it’s much easier to determine what kind of experimentation you need to run to test your hypotheses.
Because the first solution—send messages to users that items are available to rent right now— was categorized as a Big Bet, the Rocksbox lifecycle marketing team decided that rather than create an entirely new campaign, they would send a limited-release test to measure its return before making further investments.
To prevent voluntary churn, Naomi recommended a holdout test to see whether a subset of users would customize their first box versus the control group. And then to test incentives for customers to unpause their memberships, a standard A/B test experimented with different incentive types.
Key Learnings
As Naomi explained in the webinar, making sure you rigorously contemplate the challenges to be solved and prioritize the solutions to test will set up your experimentation for success.
Here are three best practices to take away from Rocksbox’s key learnings:
- Drive KPIs, not email metrics. Rather than get distracted by email opens and clicks, set up tests that will ultimately improve your primary KPIs, such as increasing purchases or reducing churn.
- Statistical significance is crucial. Naomi recommends using a sample size calculator and Iterable Experiments to achieve 95% statistical significance.
- Consider the impact of your test results. When assessing your results, evaluate how they affect your primary and secondary KPIs, as well as any trade-off metrics. If a test impacts users negatively, iterate on the solution or scrap it.
Hopefully, with these best practices, you’re inspired to enhance your own testing strategy and rock your future campaigns.
For more resources on experimentation, you can read our Support Documentation and take our Academy Course on Experiments and A/B Testing. And to revolutionize your inbox with Iterable Experiments, reach out and request a demo.
