Amanda Designs Things

Subscription & Support Modal

The Subscription Modal acts as the guide to take an enthusiastic member of a streamer’s community and allow them to show support in a tangible way with impact to both the streamer and the user’s experience. In a design led initiative that I convinced PM and engineering to join me on, I redesigned this experience and renamed it to the Support Model.

Platforms: Web

One of the most rewarding things about using a live service like Twitch as opposed to a pre-recorded/edited experience like YouTube is the ability to watch your favourite creators in real time. Instead of a solo experience, you are present with the streamer as well as an entire community of like-minded fans. A group who already know all the best in-jokes, remember why September 26th is a big deal (partnerversary!) and know that the crying bear emote doesn’t mean sadness, but elative tears of joy.

Subscriptions are Twitch’s way to monetarily support streamers as an individual and lean into the community with a series of benefits to enrich chatting and viewing experiences

Where we began

Starting at $4.99 a month, over the years Twitch Channel Subscriptions have expanded beyond benefits like the emotes above to also include subscription badges in chat showing your tenure, higher tiered subscriptions, multi-month subscriptions, animated emotes… the list goes on and the subscription modal began to expand to reflect this.

The original subscription flow lived in a small balloon component and was a simple decision. Do you want to directly support your favourite streamer, and if you were a Prime user, did you want to do it for free?

Where it took us

Fast forward to 2020 and the subscription modal evolved to this:

Completely removing you from the viewing experience, the subscription modal had become an overwhelming experience, literally blocking the stream though a darkened scrim and large footprint, but also with an overload of information. To put it bluntly, every feature and improvement that had ever been attached to Subscriptions fought to be put on display, in hopes that it would be the one thing a user would need to see in order to purchase a subscription.

I was getting sick of playing pin 30 tails on the donkey.

But as a UX Designer, it was easy for me to look at this and say “This is bad, we should start over” but significantly harder to get buy-in from my partners to overhaul an experience that “technically worked”. So instead I jumped into the data.

How bad is it?

An easy piece to evaluate was the block of text at the top of the modal. I am a strong believer in the concept that majority of the time, users do not read. Product, however, felt that these particular benefits were some of the most important selling points to surface to the user, hence it’s prominent top placement. So we put it to the test.

In an A/B/C/D test, we tried different variants of exposing these benefits to users:

• A - Control (above)

• B - Slightly different order

• C - Visually with iconography

• D - Completely remove them

After the test ran long enough to hit statistical significance our data scientist reported out the results. In our test, we measured success based on percentage of users who transitioned from modal open to clickthrough to check out.

• Variant B had no statistically significant difference from the Control

• Variant C performed better than A & B

• Variant D performed notably better than all 3 other variants

While this test was running, I used our self serve UXR resources to run a quick study myself. With the help of the sourcing team, I recruited 10 Twitch users with active subscriptions. I then took them through the channel page and had them go through the motions of subscribing to a streamer at their own pace, to jog their memory. I then asked them questions about what was included in a Subscription.

9/10 participants said their main deciding factor to subscribe was in order to support their favourite streamer. While things like emotes and badges were fun they were not the point. They were more a representation of said support (like a cancer awareness ribbon) than having inherent importance themselves.

10/10 participants could not name every benefit shown above.

7/10 participants, once discovering “Ad-free viewing” was a benefit, exclaimed that we should advertise it to users, as they did not know it was a benefit of their existing subscriptions.

This prompted my last question where I learned…

9/10 participants admitted to never having read the subscription modal before, despite currently and previously paying for subscriptions to streamers.

How do we do better?

With this data in my back pocket, I started my explorations of what the next iteration of the subscription modal could look like.

I had a few key points of focus while I brainstormed:

• People subscribe to support their favourite creators, first and foremost.

• Chat assets are a representation of the community of supporters

• People do not read.

Let’s get on the same page

Once I had a couple concepts I felt relatively good about, I got my most trusted PM and front end engineer in a room and told them what I had been doing. I broke it down into detail, the A/B test that was run, the UXR I had done, and how I fed those findings into the explorations I had to share.

And then I asked them if they thought I actually had something here or if I was living in a pipe dream.

Did engineering feel my plans were technically feasible? Did PM agree this aligned with the stated goals on the roadmap for 2020? Would they have my backs if I quoted their resulting enthusiasm?

Yes.

And so we collaborated from thereon out, picking our favourite things from each exploration for me to iterate on and refine. The following is what we presented to leadership.

Keep it small, keep it simple. Show me the fun extras without bogging it down. Don’t overtake the page.

Leadership loved it. They had concerns of course, but they were excited about the refocus on “support” and how visual it was compared to the current experience. We were on our way.

Something about Rome in a day?

With 2020 came some weird times, and a strong upswing in Twitch usage and user spending on platform. Every part of the Monetization product group came under scrutiny. Are we utilizing this product to the best of our abilities? What should we be investing in now?

The details of what a subscription was to users was being examined with a magnifying glass. Were higher tier subscriptions (more money) something to keep investing in or should we streamline efforts into the default subscription? Should users be able to subscribe for multiple months at a time? I made the decision to put my project on hold, arguing that if we were going to be changing any of the details of how a subscription worked, a redesign would be a great time to showcase those changes as opposed to right before them. PM and I went headlong into more UXR and A/B testing.

And then it was FALL

We emerged from the summer with a slightly tweaked design, but the fundamentals still all there.

Stronger focus on the streamer’s own brand and supporting them. Deprioritization of the higher tier subscriptions below the fold. Also, I came up with a panel concept, where the experience felt like it was coming out of the existing UI instead of overlaying on top of it.

This lent itself well to my desire to take ensure the subscription experience interrupted the viewing experience as little as possible.

With the new direction solidified, the rest of the experience was fleshed out.

Did we do it?

One of the things that can be hard to convince Product to invest in is things that will take time to show results. They want “quick wins” and “low hanging fruit”. I warned them that this would not be “quick” and the definition of “win” would be hard to measure. Subscriptions had shown in the past to be decisions that users made over an extended period of time after a lot of investment with a particular streamer, not one made on a whim because we updated some UI. Additionally, any users of the existing experience were at risk of the novelty effect.

Our first good sign after launch was our metrics did not go down. We were so used to the novelty effect either causing conversion percent to go down (lots of lookie-loos with no intent to purchase) or for overall subscription numbers to go down (because things are different and people get lost/give up) that the fact that nothing was going down seemed pretty good. At a bare minimum, I hadn’t horribly confused our existing users.

Tracking after that gets complicated as more features shipped over time and attributing changes to the right source can be difficult, but one thing I can attest to is this redesign helped shift the thinking of the entire product group. While making money is obviously still a priority in a business, features were thought about a lot more carefully in terms of how it supported the streamer. Twitch and the support panel now make a point of showing users how their spending lifts up their favourite streamer first and foremost, with personal benefits simply being icing on the cake.