How labworks.io uses In-Skill Purchasing and A/B testing to grow a business with Alexa

Webby award-winning labworks.io is a conversation design and prompt engineering studio based in the United Kingdom. Founded in 2016, labworks.io started by developing chatbots and now specializes in conversation design across a variety of use cases, including large language models.

Conversation design enables developers to build conversational experiences that feel natural and effortless. "You can never do enough user research in conversation design," says Tom Hewitson, founder of labworks.io. "There's a lot that goes into building experiences that make talking to machines seem just as intuitive as talking to humans."

Earning the Alexa Champion title

Hewitson began his career in journalism. His interest in telling stories persisted after he transitioned to product design in the tech industry, where he encountered a chatbot-based digital assistant. He was drawn to conversation design because, like in journalism, the right selection of words was critical to building compelling experiences.

"I saw that conversational interfaces were going to be really big and felt like every business was going to want one," says Hewitson. He launched labworks.io to take advantage of the opportunity. His writing background allowed him to differentiate himself in an industry where most product designers have a visual background. Needless to say, when Alexa debuted in the United Kingdom, Hewitson was eager to create a skill because he anticipated labworks.io clients would want to take advantage of Alexa's rapid momentum.

Hewitson developed his first skill—a game called "Would You Rather"—as a hobby project in his spare time. Six weeks after publishing it, Hewitson learned that the game had become the third most popular Alexa skill. Because of the game's high engagement and the resulting revenue, Hewitson started building Alexa Skills full-time, creating a portfolio of games that subscribers can access through Voice Arcade. One success led to another, and today, Hewitson's company now maintains over 500 Alexa skills.

In 2017, Hewitson was recognized as the first Alexa Champion outside of the United States. Through the Alexa Champion program, Hewitson provides feedback about his experience creating skills to the Alexa team, which helps guide product improvement efforts.

One of the primary ways labworks.io generates revenue from Alexa is through In-Skill Purchasing (ISP), where developers sell premium content to users. In-skill purchasing lets developers sell premium content, such as game features and interactive stories, in custom skills. Buying these products in a skill is seamless to a user. They might ask to shop for products, buy products by name, or agree to purchase suggestions that you make during their skill session. Customers pay for products by using the payment options associated with their Amazon account.

Labworks.io tried several types of ISP before determining that a subscription model was the best fit. During user research, labworks.io hit upon an interesting insight: families accounted for a large portion of its user base. To avoid scenarios where children racked up large bills while playing games, labworks.io decided to pursue a subscription model that steered free of impulse purchases.

While the subscription model has worked for labworks.io, Hewitson recommends that skill developers try multiple ways of monetization before deciding how to use ISP. "The only way you can get to perfect monetization is through iteration and testing," says Hewitson.

Based on his experience, Hewitson shares three tips to help developers successfully use ISP.

Customizing ISP for different Alexa locales

To start, labworks.io found it could drive revenue by making skills available in multiple locales. When doing localization, labworks.io translates and adapts content to be culturally relevant. Because of the cost of doing localization correctly, Hewitson recommends thorough research upfront. "It's very difficult to localize a skill properly to another locale for less than $1,000," says Hewitson. "If you're not earning $1,000 a year from a skill in your home locale, the chances are high that it won't earn that much in another locale either."

When doing localization, Hewitson recommends that developers focus on one release at a time and act strategically to reuse previous work. Though the language spoken in Spain differs from Spanish in South America, skill developers don't need to start from scratch when adapting a skill from one country to another. "It's better to wait until you're ready to roll out in a locale properly than to do it wrong the first time," says Hewitson. "If you mess up your launch and get negative reviews, it's really hard to come back from that."

Fine-tuning with A/B Testing

After deciding on an ISP strategy, skill developers need to set pricing. Using built-in Alexa tools for A/B testing, skill developers can test factors such as the placement of upsell messages.

To set an overall price, Hewitson recommends completing user surveys. For example, labworks.io used Van Westendorp Price Sensitivity Analysis for the Voice Arcade subscription to determine the optimal and "fair" price for a subscription.

"We asked users to link their account at the point where we would have served the ISP offer," says Hewitson. "Account linking allowed us to get the contact information of active users with their explicit permission. We could then connect with these users to get feedback on pricing."

Increasing user retention

User retention is important for skills like games, which depend on ongoing purchases from consumables or subscriptions. "In my experience, it's significantly more impactful to increase user retention than to try to increase the number of users or the price," says Hewitson.

Quality is important for user retention. However, Hewitson recommends taking a more multifaceted approach.

"At the end of the day, building a good skill is a series of tactical actions you take to make a skill work better," says Hewitson. "For example, developers might encourage users to link their accounts, making it possible to send reminder emails if there is a decline in activity."

Hewitson also points out that what constitutes "good" will vary from skill to skill. For example, the company's "On This Day" skill informs users about daily historical events, so labworks.io expects users to log in frequently. The "Star Commander Skill," on the other hand, is built around a story. This means that the skill should have high activity at the beginning that drops off after the user reaches the end.

"It's often more useful to focus on whether your retention number is improving over time rather than what your specific retention number is," says Hewitson. "If you're not improving, take a step back and think strategically about what you're building and whether it needs a high-level change rather than an iterative change."

Labworks.io also analyzes user retention in a more holistic way across multiple skills. The company ideally wants users to interact with several skills in the portfolio multiple times per week, so this level of tracking helps labworks.io develop a more nuanced understanding of customer preferences.

Hewitson looks forward to the future of designing conversational experiences. "With large language models and the democratization of digital assistants, I foresee a massive transformative effect where every business, and maybe every person, will have their own assistant," says Hewitson. "That can lead to a lot of increased productivity and opportunities. I think it's a very interesting time for the industry."

Get started with ISP by reviewing this technical document and design an optimal ISP experience by reviewing the Alexa Design Guide.