Product & Growth: learnings from Duolingo & others!

💡 JC's Newsletter #165 - 2023-04-03

Apr 04, 2023

Dear friends,

In JC’s Newsletter, I share the articles, documentaries, and books I enjoyed the most in the last week, with some comments on how we relate to them at Alan. I do not endorse all the articles I share, they are up for debate.

I’m doing it because a) I love reading, it is the way that I get most of my ideas, b) I’m already sharing those ideas with my team, and c) I would love to get your perspective on those.

If you are not subscribed yet, it's right here!

If you like it, please share it on social networks!

Share 💡JC's Newsletter

🔎 Some topics we will cover this week:

Valuable insights on Duolingo’s growth strategies
Facebook’s interesting analysis of short-term vs. long-term effects and how we should think about notifications and nudges
How Duolingo leverages its mascot, Duo, to create habits
Good tips on how to innovate

👉 How Duolingo reignited user growth (Lenny’s Newsletter)

❓ Why am I sharing this article?

Very good article about growth
I liked the focus on differentiating “new users” and “users you want to reactivate”
I think we should inspire ourselves a lot for our adoption and take more risks there
Keep being inspired a lot by outside and other industries, but do not “just copy”: also think from “first principles”
Keep very small teams to iterate
From the first principles, I don’t see how referrals are also going to work for us
Good list of questions to ask yourself:
- Why is this feature working in that product?
- Why might this feature succeed or fail in our context, i.e. will it translate well?
- What adaptations are necessary to make this feature succeed in our context? In other words, we needed to use better judgment in adapting when adopting.
Focus on the harder to moves metrics that have a high impact
I love leagues in leaderboards & streaks optimization. I use a product named ELSA for English and Leagues+Streaks makes me use it every single day for months & months
Avoid short-term optimization that destroys a channel

Hundreds of millions of users. User growth was slowing down.
Our first big wins that helped us turn around growth, including launching leaderboards, refocusing on push notifications, and optimizing the “streak” feature.
Phase 1: Increasing gamification.
Our first attempt at reigniting growth was focused on improving retention, i.e. fixing our “leaky bucket” problem. We prioritized working on retention over new-user acquisition.
Armed with a short presentation I co-created with our chief designer, we were able to get just enough buy-in from the rest of the executive team to create a new team, the Gamification Team. The team consisted of an engineering manager, an engineer, a designer, an APM, and me.
Gardenscapes, however, paired its progress bar with a moves counter, which Duolingo didn’t do. The moves counter allowed users only a finite number of moves to complete a level, which added a sense of scarcity and urgency to the gameplay. We decided to incorporate the counter mechanic into our product. We gave our users a finite number of chances to answer questions correctly before they had to start the lesson over.

➡️ Look for inspirations in other industries

I expectantly waited for an unmitigated success. Depressingly, the result of all that effort was completely neutral. No change to our retention. No increase in DAU. We hardly got any user feedback at all. I was deflated.

➡️ If you just copy without thinking from first principles, you will fail.

Phase 2: Referrals.
We implemented the feature and hoped our second attempt would be more successful.
Instead, new users increased by only 3%.
It was positive, but not the type of breakthrough we needed.
Still, the team doubled down and pushed through, shipping iterations to the referral program and making some other bets, but no avail.

➡️ Same as the above.

Learnings on first principles:
In hindsight, it became clear why the Gardenscapes moves counter was not a good fit for our product.
When you are playing Gardenscapes, each move feels like a strategic decision, because you have to outmaneuver dynamic obstacles to find a path to victory. But strategic decision-making isn’t required to complete a Duolingo lesson— you mostly either know the answer to a question or you don’t.
Because there wasn’t any strategy to it, the Duolingo moves counter was simply a boring, tacked-on nuisance.
It was the wrong gamification mechanic to adopt into Duolingo. I realized that I had been so focused on the similarities between Gardenscapes and Duolingo that I had failed to account for the importance of the underlying differences.

➡️ 1st principles

In both of these situations, we had borrowed successful features from other products, but the wrong way. We had failed to account for how a change in context can impact the success of a feature. I came away from these attempts realizing that I needed a better understanding of how to borrow ideas from other products intelligently. Now when looking to adopt a feature, I ask myself:
Why is this feature working in that product?
Why might this feature succeed or fail in our context, i.e. will it translate well?
What adaptations are necessary to make this feature succeed in our context? In other words, we needed to use better judgment in adapting when adopting.
Resurrection of users:
Zynga separated their users and measured retention based on the following weekly retention metrics:
Current users retention rate (CURR): The chance a user comes back this week if they came to the product each of the past two weeks
New users retention rate (NURR): The chance a user comes back this week if they were new to the product last week
Reactivated user retention rates (RURR): The chance a user comes back this week if they reactivated last week
With the model created, we started taking daily snapshots of data to create a history of how all of these user buckets and retention rates had evolved on a day-by-day basis over the past several years.
With this data, we could create a forward-looking model and then perform a sensitivity analysis to predict which levers would have the biggest impact on DAU growth.
We ran a simulation for each rate, where we moved a single rate 2% every quarter for three years, holding all the other rates constant.
CURR is much harder to move, but when it does, it will have a greater impact.
Gamification step 2:
Our failure with the Gardenscapes-style moves counter hadn’t actually disproved any of the original reasons why we believed gamification still had upside for Duolingo—we had only learned that the moves counter was a clumsy attempt at it.
This time, we would be more methodical and intelligent about features we added or borrowed. We made sure to apply the lessons from our prior efforts with gamification.
After some consideration, we decided to bet on leaderboards.
Duolingo already had a leaderboard for users to compete with their friends and family, but it wasn’t particularly effective.
When I started working on Zynga’s FarmVille 2 game, it included a leaderboard similar to Duolingo’s existing leaderboard, where users competed with their friends.
I had hypothesized based on my personal experience as a player that the closeness of the competitor’s engagement would be more important than the closeness of personal relationships.
I thought this would be especially true in a mature product where many users’ friends weren’t active anymore. From our testing at Zynga, that idea turned out to be true.
Based on this, I felt a leaderboard system, similar to what I had helped design at Zynga, would succeed in the context of our product.
FarmVille 2’s leaderboard also included a “league” system.
Beyond getting to the top of a weekly leaderboard, users had the opportunity to move through a series of league levels (e.g. from the Bronze league to the Silver league to the Gold league).
Leagues provided users with a greater sense of progress and reward, an integral element in game design.
They also increased engagement over time, since engaged users move up to more competitive leagues week after week.
We felt this feature would translate well to Duolingo’s existing product because it tapped directly into the common human motivators of competitiveness and progression.
The leaderboards feature had a huge and almost immediate impact on our metrics.
Streak:
The streak vector: He discovered that if a user reached a 10-day streak, their chances of dropping off were reduced substantially.
The concept of a streak is really quite simple: show users the number of consecutive days they’ve done any activity on the app. But it turns out that there is a surprisingly large number of optimization opportunities around streaks.
We got our first big win with the streak-saver notification—a notification that alerts users with streaks if they are about to lose their streak.
This late-night notification proved that indeed there was considerable upside to doubling down on streak optimizations.
After this, several improvements followed: calendar views, animations, changes to streak freezes, and streak rewards, among others. Each helped improve upon the original streak idea and generated substantial improvements to retention.
To date, the streak feature is one of Duolingo’s most powerful engagement mechanics.
Protect your channel:
A cautionary tale from Groupon’s CEO.
For a long time, Groupon stuck to one email notification per day.
But their team started wondering whether sending more emails would improve metrics.
The CEO eventually gave in and allowed his team to test sending one more email to each user each day. This test resulted in a big increase to their target metrics.
Encouraged, Groupon kept experimenting, sending more emails, even as many as five a day. Then, in what felt like a change from one day to the next, their email channel lost most of its effectiveness.
Over time, the accumulation of Groupon’s aggressive email tests had basically destroyed their channel. One often underappreciated risk with aggressively A/B testing emails and push notifications is that it results in users opting out of the channel; and even if you kill the test, those users remain opted out forever. Do this many times, and you’ve destroyed your channel. This was the outcome to avoid. For our push notifications, we established one foundational rule: protect the channel.

👉 Virality is a myth (mostly) (Lenny's Newsletter)

❓ Why am I sharing this article?

It’s easy to make assumptions on what drives organic growth / virality - we too often fail to realize for each product the true machinery at play.
The knowledge regarding repetition could be applied to B2B2C adoption by finding out the right communication channels

Virality is not what we think
We think that products often grow through friends telling friends, who tell more friends, and this cascades to so-called viral growth. It turns out this is almost never how products grow. Instead, products explode in popularity when someone (or a few someones) with a large platform shares the product with their audience.
The gospel of virality has convinced some marketers that the only way that things become popular these days is by buzz and viral spread. But these marketers vastly overestimate the reliable power of word of mouth. Much of what outsiders call virality is really a function of what one might call ‘dark broadcasters’—people or companies distributing information to many viewers at once, but whose influence isn’t always visible to people outside of the network.”

➡️ Virality and high K-factor is often the results of a clear marketing push toward the right audience

➡️ Word of mouth should be engineered and doesn’t simply happen - it is often the results of clear & driven actions. It can be hard to have visibility on some “dark broadcasters” or true growth drivers. (We could think about how to replicate Mind success across companies while missing the internal influencer enabler for adoption / how Alan Clear can spread within an org)

Fire ignition
Even though products don’t grow virally for long, it’s still absolutely worthwhile to optimize mechanisms of virality (e.g. word of mouth, invites, referrals, a remarkable product), since that can drive ongoing (free) growth.
At the same time, to ignite (and re-ignite) moments of “virality,” you’ll need to invest in getting large one-to-many broadcasts. For example, PR, influencers, TV.

➡️ Even if we have product growth loops built in - we will need fire-starters to get it going.

👉 How Facebook saw HIGHER usage by sending FEWER notifications (The Split)

❓ Why am I sharing this article?

I really liked this analysis of short-term vs. long-term and how we should think about notifications, nudges on our side.

This counter-intuitive study shows the importance of understanding how short-term and long-term effects may differ.

👉 Duolingo product: Thread by Ali Abouelatta (PingThread)

❓ Why am I sharing this article?

I love how they leverage their mascot to create habits. We should do the same!
I love the idea of getting special app icon when you completed task
“Commit to my goal” is powerful for health objectives
The perfect streak week would be perfect for Alan week 1, and for our programs!

Our take: Duo gets progressively more frustrated throughout the day if you haven’t done your lesson.
Our take: A special app icon, unlocked after reaching a 30 day streak. The icon emphasizes your streak status. (So every time you unlock your phone, and see the new icon, you are a bit more motivated to extend your streak)
Our take: When a user selects streak goal we ask them to “Commit to their goal” instead of a simple “Continue”
Our take: A new gamification mechanic called “Perfect streak week” for those who go above and beyond to ensure they do their lesson every single day. (Our committed users ❤️ showing these off!)

👉 Tony Fadell (ICONIQ)

❓ Why am I sharing this article?

You need to pick topics that are hard to make (and create value) to differentiate
Good tips on how to innovate

Good ideas chase you. If you continue to circle back to an idea, keep asking yourself these questions:
Will this be good for people, for society, for the earth?
Is it hard to make? (Because, if it’s good but not hard, then have someone else do it!)
Is society ready for it?
Are all the complementary technologies in place or added fairly easily?
Then, what’s the Why?
Can I tell that story to people, so that they want it (emotionally) and need it (rationally)?
Can you get them excited about the bigger mission, to the point where they get behind it and help you evangelize it?
You get others in your organization convinced and excited by spreading the “Virus-of-Doubt.”
That means you need to have them start questioning the status quo: “Why do we do it this way? That’s so dumb!” or “This ugly beige brick in my living room at eye level, and it keeps the heat on when we’re gone on vacation for two weeks? Stupid!”
And then you infect them with the very same questions you keep grinding on. And, boom, they’re on board, because it just makes sense to stop doing something senseless!

It’s already over! Please share JC’s Newsletter with your friends, and subscribe 👇

Let’s talk about this together on LinkedIn or on Twitter. Have a good week!

💡JC's Newsletter

Discussion about this post

Ready for more?