Product-Led Experience

The Evaluation Framework behind

Learn how the emerging product-led practices change forever the way product experience can be assessed across the board.

Pre-Intro

In this guide, you will learn firsthand how product experience can be assessed. To achieve that we will showcase real examples of product onboarding practices. We encourage you to put those suggestions to work. Always, by taking a concerted focus on the particularities your products’ have.

Introduction

Stop for a minute and think. How many digital experiences do you encounter daily? At work, on your mobile, when you read the daily news? Our life is quite full of them. Daily millions of interactions are taking place by the second.

No matter in which organization you belong to, it is imperative for your product to deliver stellar product experiences. If you are a user, you experience first hand the frustration a broken experience carries. But also, the satisfaction product delights deliver.

The Emerging Product-Led Era

As Product-Led practices emerge, businesses need to fall the weight of responsibility on internal teams’ shoulders. Silos abandonment, following a product-led GTM approach, makes very clear one thing. Customer experience does not have a single owner. But it is rather compiled by the sum of touchpoints that originate and end up to the product itself.

Sales practitioners need to close deals reflecting on organizations’ vision. Product leaders need to deliver calculated results. Customer Success needs to be proactive and educate at scale.

We may think that we are in a transitioning era. But the truth is Product-Led practices have already sow their seeds. They spread like wildfire and their profound influence pass a very clear message. SaaS organizations need to either evolve by delivering a customer-centric product approach. Or become obsolete.

Product Experience

What is Product Experience?

Product experience (PX) is the part of the customer journey executed in-app. It is the point where users get onboarded, learn about new features, and realize value. Today’s product managers must, at all times, understand and improve the product experience. This is the only way to create products customers love and fight churn rates. Pragmatic Institute found that 52% of users said a bad product experience make them less likely to engage with a company

Why is product experience important?

Most organizations fail to understand how users derive value from their products. Studies have shown that 80% of SaaS features go virtually unused. Something that costs around $30 billion in wasted R&D.

At the same time, usability and UX are the points where most organizations invest. While those are serious investments they are not enough to meet customers’ needs. Product experience needs to educate, engage, and adapt to users’ needs. Product-Led organizations, already deliver that kind of product experiences. The kind that makes customers keep coming back for more.

Product Experience Assessment: The Road So Far

The entire SaaS industry has been sold into a false dichotomy. A Sales strategy can be self serve or human-assisted.  Τhere is no in-between- or so we’ve been told. The optimization of product experience is a discussion that does not seem to stop. On one hand, organizations cannot yet deliver products without creating an experience gap. While at the same time product engagements are usually overlooked.

Human-Assisted Onboarding

On human-assisted onboarding, product experience comes secondary to customer-facing teams activations. Partly the disposal of human-assisted activations resonates.  Due to the plethora of stakeholders involved. Product Led GTM practices, however, flip the script and project the product as the main growth lever.

Stellar customer experiences are not defined, anymore, by the buyer-vendor relationship dynamic. Product experience can now be leveraged to product deliverables by capitalizing on context of usage. While at the same time, products must educate, engage, and adapt to their users’ needs.

Self-Serve Onboarding

On self-serve onboarding, “the machines” prevail. Our product experience research showed that self serve adopters invest in activation (61%).  But, retention and expansion fall behind.

This fact alone leads us to the following conclusions:

  • For one it indicates that onboarding prevalence ends after the activation stage.
  • The limited investment in retention makes accounts susceptible to churn.

Product Management Activations

Product Management’s ongoing feature releases, also impact the product experience. Product leaders should invest in a JTBD framework in conjunction with contextual guidance to leverage onboarding. That realization though may be far from true. Our research showed that Product Management is accountable for the onboarding process (81%).  But critical indicators like usage (5%) and product engagements assessment (5%) are neglected.

Product-Led Onboarding

So, how can onboarding assess product experience? For starters, product-led practices radically change its nature. Product-Led Onboarding, driven by product data, can be evaluated on every stage of the customer journey.

Product management can now reactivate the onboarding process on every new release and acknowledge how it can deliver initial value. Following that logic, PMs can assess where onboarding leads to upgrades, and accounts’ expansion. Capitalization in product data derive insights on every move a user makes in-app.

This process abandons the traditional sales model archetype, ending onboarding prevalence during activation. The sales funnel has evolved into a circle where onboarding stands in its center. Always waiting for the next feature release, to deliver initial value again.

Assessing product experience

It won’t be long now until new terms will describe the intimacy levels of the User-Product relationship. The most common term so far is the Product-Qualified lead (PQL). The PQL term refers to prospects that signed up and demonstrated buying intent. While the criteria following it are product usage and behavioral data.

Being limited to the point where a paid conversion is made, PQLs have as their benchmarks early adoption. PQLs as a metric is truly invaluable to Sales teams. But, if organizations want to assess users’ behavior on every stage of the funnel they need to consider Product OQLs(Product Onboarding Qualified Leads) in their day-to-day evaluations.

Product OQLs™ rely on POEs metrics to segment in-product behavior. As with PQLs again in OQLs, there is no absolute on which metric should prevail. Balance among breadth, depth, frequency, and efficiency of use though, is a good foundation internal teams can consider to establish when users get value.

Product Experience Variable: Breadth of Use

A form of (team) activation, breadth helps PMs realize the extent a product is being used on an account level. As a product metric,  it monitors account health and helps PMs manage churn.

High Trajectory Customers

The internal buy-in from end-users discourages onboarding deployment. Heterogeneity on skills and heavy workflows decrease their willingness to adopt a new solution. Focus on breadth, usage and use case will help internal teams overcome this barrier.

Self Serve Customers

  1. In the absence of Sales and Customer Success, in-app flows should double-down on inviting team members to embrace activation rates. Emphasis on how team onboarding increases perceived value should be the number one consideration.
  2. Product teams should define which roles cause various drop-offs on their first-week cohorts.
  3. Training procedures need to focus on both users’ role and team training.

Team Activation Use Case: Hubspot

One of the key activation metrics predicting usage for Hubspot is if team onboarding within an account was realized successfully. Product teams track team activities to realize if the solution is driving value. It is not a coincidence that team activation associates with Hubspot’s long term success and is its north star metric as well.

Breadth of Use- Points to consider

  1. How many users log in during onboarding and how many later on?
  2. The number of users that activate upon a new feature release. (Assuming that the targeted user segments are those realizing value with this feature)?

Breadth of use calculations formula

product_experience

Product Experience Variable: Depth of Use

High Trajectory Customers

Points of consideration here are end-users’ growth within the product itself. Internal teams should track daily usage on an account and individual level. On top of that, milestones per role should be set in conjunction with users’ progression over 12 months span.

Self Serve Customers

On high trajectory, customers desired depth levels are usually met. The increased involvement of customer-facing teams makes this inevitable. Self Serve customers though, may never get there. Lack of personalization and low investment on tailored onboarding flows, usually lead to high churn rates.

Depending on each organization’s practices, adoption may be evaluated within the first 7 days or after the end of the trial period.

At this point milestones should be set considering:

  1. Since SMBs have small teams, depth should be assessed in conjunction with breadth to assess engagement levels.
  2. PMs should consider renewing milestones monthly if self-serve customers are not usually bound by contracts.

User Segmentation Use Case: Gainsight PX

First Time Activation:

Gainsight PX, segment users’ behavior and usage on the first-time activation. It does that by tracking if PQLs use key features during trial. When usage overcomes the freemium’s levels, internal teams measure conversions and revenue generated from the trial source.

New Product Release:

On new product release, users’ behavior is segmented by measuring adoption. The product team releases targeted in-app guides to users based on historical product usage. The Query Builder feature, for example, is presented to users who have shown interest in other analytics areas the past 30 days. If the released feature is a paid-only module, the revenue is measured by attaching a rate on it.

Depth of Use- Points to Consider

  1. Which adoption levels (Depth of Use) correspond to which user role?
  2. Does adoption increase due time? (aka are users growing within the service?)
  3. Are internal teams able to reflect usage levels on Net Revenue and/or Net churn?
  4. How is depth defined during trial and how post-purchase? (Varying parameter per product/strategy/pricing plan)
  5. Which adoption levels show signs of accounts’ expansion?
  6. Which characteristics follow engaged users (retention) and which those abandoning the app?

Depth of use product onboarding calculations formula

product_experience

Product Experience Variable: Efficiency of Use

The difficulty to complete common tasks is critical when evaluating product experience and overall onboarding effectiveness. For the right measurements to be in place product teams need to know two things. The first is the number of users per account who begin a task while the second is the amount of those completing it.

High Trajectory Customers

  1. Being a composite of human-assisted and product activations in this instance,  efficiency does not rely only on onboarding effectiveness. That being said, the emerging investment in product data, turns product experience into the growth lever following accounts’ long term prosperity.
  2. When onboarding many teams the focus should be on users’ role early on to assess usage and adoption. That realization by default indicates Sales involvement in the onboarding process. This how onboarding will become measurable from the very beginning. Even more, this is how in the long term harmonization between high touch & high tech activations will be achieved.

Keypoint One:

Sales teams should claim ownership of trials’ product data and observe prospects’ behavior in and out of the product. Capitalization on passive feedback enables sales teams to pinpoint where the product experience is broken. Sales will also acknowledge how they should capitalize on end-users workflows (context). Last but not least, investment in product data, enables Sales  onboard the first team(s) and pass the necessary feedback to Customer Success.

Keypoint Two:

Customer Success analyzes Sales feedback and suggests to buyers the next steps going forward, based on real insights.

Capitalization on product data, help Success practitioners:

  • Get the internal buy-in easier
  • Be aware of the paths users take in-app
  • To act proactively whenever flows onboarding downgrade product experience
  1. Product management monitors users’ usage and trends.  The criteria here are again their role, profession, and proficiency level. Points of consideration are when the desired adoption levels are realized with or without the help of customer-facing teams. In case the levels of human-assisted activations are above the expected, Product Management should inject those learnings in the product onboarding flows to further optimize them.

Self Serve Customers

Having the product replicating human-assisted activations requires continuous iterations and heavy experimentation. Being autonomous and subject to a faster sales process Self Serve customers should realize initial value fast without internal teams neglecting the required learning curve period.

Internal teams need to know when they strive to shorten trial periods. Even though speed to implement needs to come first the learning curve period is always subject to products’ complexity. A factor intrinsically connected with the onboarding process’ required steps. Internal teams need to define the number of steps leading to adoption by considering that users familiarize themselves with the product by following their own pace.

Use Case: Having onboarding delivering many value points (First-time activation)

Drift’s former onboarding was a quick process constituted by three steps, aiming to get users to install its javascript code. That resulted in high levels of churn as free users did not feel invested in the product. To deal with that effect, the onboarding launched a ten steps onboarding flow. Each step was motivating users to complete three different tasks. The end goal was to create many wow moments and make users feel invested when the set-up is complete. The flow has proved to be a tremendous success as it ended up tripling conversion rates.

Use Case: Trial period (First-time activation) optimization

When Yesware, decided to invest in a product-led strategy, one of its first goals was to build an infrastructure allowing rapid testing.  In this vein, the internal teams decided to run an experiment that halved the free trial length from 28 days to 14.

The Experiment

The hypothesis was that while the conversion rates would remain steady, the product team would benefit by being able to run tests twice as fast. Additionally, there was the expectation that shortening the trial would provide a sense of urgency.

Because of certain technical restrictions, the test was run longitudinally as opposed to an A/B. The product team changed the trial length and compared the 14-day trial cohort to the preceding 28-day. After a month of testing, the results were fantastic! As expected, there was a slight increase in the percentage of users (0,5%) who uninstalled the product during the trial. Overall though, the core hypothesis was valid since early engagement rates increased. That stood particularly with key features, which are the leading indicators for the solution’s power users. An effect assumed to have happened due to the urgency created. Moreover, conversion rates did not only maintain but increased by roughly 18%.

Special note:

After rapid experimentation, the product and engineering team have decided to extend the 14-day trial to 28 days. That would stand only for users who completed specific actions, based around sticky features, in the getting started guide.

Use Case: High-Touch vs. High-Tech Harmonization 

Userlane, launched a while ago, a scalable onboarding process to complement its human-assisted onboarding strategy. The “academy” as it is being called internally, self serve end users all the way through without eliminating the required decision making between the two parties.

Customer Success calls that learning process “getting to Basecamp”. The various teams build their first Userlane guide, publish it and in the end, they get certified. The process is highly transparent since users are aware of the tasks they need to complete. On top of that, the buyers’ acknowledge end-users’ progress at all times.

The Inception Project:

The launch of the “Inception project”, improved a lot the onboarding process delivery. The implementation of targeted in-app guides within the product allowed a thorough personalized, scalable onboarding process that didn’t need active participation from different units. Time to initial value decreased and post-launch, buyers could go through the set up completely by themselves.

Inception Project Desired Deliverables
  • Increase of engagement and activation
  • Reduce time-to-value
  • Decrease time to key features
  • Decrease onboarding completion time
  • Increase trial-to-paid conversions by at least 30%
  • Reduce the costs and efforts connected to high touch activities by 60%
Inception Project Results

product_experience_project_resultsAcross the spectrum, Userlane managed to optimize product delivery without compromising onboarding execution.

Increased adoption rates (+45%) confirm that this approach:
  1. Yields the expected ROI.
  2. Keeps end users engaged and buyers aware of end-results.
  3. Stickiness was reinforced by actions that capitalize on product activations.

Userlane’s Inception project proved one thing. Organizations requiring tailored implementations can indeed harmonize scalability with the human-assisted activations.

Efficiency of Use- Points to Consider

  1. In this instance onboarding flows should focus on historic usage per user role and account level by taking a concerted focus when adoption levels increase.
  2. First-Time Activation:
  3. a) For solutions offering many products first time activation, should break down the onboarding flows in many parts. This is how users will realize ongoing value and increase their skills and investment.
  4. b) Simpler solutions should experiment with shortening trial length by focusing on users’ actions in-app.
  5. Internal teams should track if product onboarding activations decrease customer-facing teams involvement.
  6. Ownership of product data should be dispersed across teams. This is how a feedback loop will be embraced and enable them to foresee any anomalies occurred throughout the customer journey.

Efficiency of use product onboarding calculations formula

product_experience

Product Experience Variable: Frequency of Use

Frequency of use estimates how often and in what degree users engage with features. Reminding users why a specific feature is there in the first place and how it may further optimize their workflow is also something reliant to onboarding activations.

High Trajectory Customers

In regards to high trajectory customers, usage levels are being closely monitored by Customer Success. Product-led practices embrace Product Management involvement in the process too. They hold it accountable to deliver quantitative measurements in regards to adoption or churn levels.

Both teams should create common benchmarks and track whenever features’ usage drops. Following this logic targeted product activations should be in place, to restore engagement levels when necessary.

At the same time internal teams should monitor two things:

  1. When human-assisted activations prompt users return to the product
  2. And if those learnings can be injected into the product onboarding flows.

Self Serve Customers

Having Self Serve customers returning to the product is something reliant both to product and email activations. In this instance, email practices replicate human-assisted activations. So, it resonates that customer-facing teams should adopt the right tools to monitor this correlation. Solutions like Gainisht PX mapping such activities, come handy in this case.

In this instance, internal teams should invest in customer feedback early on:

  • Sales should suggest what learnings to insert into trial, per use case.
  • Product Management should always consider passive feedback
  • Customer Success should release in-app surveys by focusing on the features paid accounts exploit vs. those neglected.

Frequency of Use- Points to Consider

  1. Which features are used more frequently?
  2. Which features correspond to each team’s use case?
  3. How can teams be triggered in-app to increase usage in secondary features?
  4. When and why each feature should be used? ( to better estimate the revenues and losses following them)5. What made users return to the product or abandon it?

Frequency of Use Calculations Formula

Product Experience Adoption Loop

All things being equal, Breadth, Depth, Frequency, and Efficiency of use form an adoption loop.

product_experienceThe loop’s implementation is viable when:

  1. The sum of account’s users exploit a product
  2. use its features end-to-end
  3. And repeat those actions often enough.

Again, the decisive role initiating this circle of events is (team) activation. But for the loop to be consistent, on every stage of the customer journey, all four metrics should be considered. Depending on the onboarding strategy the loop is being supplemented by business KPIs and parameters.

Product Onboarding Efficiency (POE) – Breakdown

Activation stage (Indicative)

POE Variable
KPI
Points under Consideration
Department
Breadth
No of team members entering the solution
How many team members entered the solution?
Sales
Efficiency
No of users reaching initial value
Does the flow encourage team activation?
Sales/ Product Management
Frequency
How often users login in the s/w?
Are onboarding flows creating a habit early on?
Sales/ Product Management
Depth
Features explored during trial
What characteristics follow users exploring key features?
Sales/ Product Management

Key Takeaways

Product Management

The product team should be involved in the activation process, no matter the onboarding strategy at hand.

Sales Organization

The sooner Sales’ decode the messages withheld into product data the better the conclusions drawn, in regards to the harmonization of human-product activations.

Depth of use has some variables to consider:

  1. If during activation users can realize initial and true value. This is most likely to occur on high-velocity customers where the onboarding may take place post-purchase.
  2. Users’ skills need to be defined early on. An advanced user may explore more key features during trial. However, there should always be a consideration in regards to products’ complexity.
  3. On Self Serve onboarding, Retention initiation point varies. Thus Depth of Use may not be measured on the activation stage (during trial) but only post-purchase.

Retention stage (Indicative)

POE Variable
KPI
Points under Consideration
Department
Breadth
No of team members entering the solution
How many team members activate frequently on an account level?
Customer Success / Product Management
Depth
Which features are explored end-to-end
Which key features prevail over others, usage wise?
Sales/ Product Management
Efficiency
No of users reaching true value
Are onboarding flows encouraging users to explore the solution to its full extent?
Sales/ Product Management
Frequency
Have users created a habit out of the product?
What is the percent of users returning to the product autonomously?
Sales/ Product Management

Key Takeaways

Customer Success & Product Management

Both departments are the rightful owners of accounts retention and customer journey optimization.

Depth

The retention stage defines which features prevail over the others. It is optimal at this point, internal teams to define when accounts reach the desired levels of adoption (depth) and show indications of expansion.

Frequency

Whenever frequency of use increases so do the indications of those accounts reaching long term retention.

Expansion stage (Indicative)

POE Variable
KPI
Points under Consideration
Department
Frequency
No of users returning daily into the product
What makes users return daily into the product?
Customer Success / Product Management
Depth
No of users exploiting features to their full extent
Which adoption levels indicate expansion opportunities?
Sales/ Product Management
Breadth
No of users using the product to its full extent inside an account
Are all users realizing true value?
Sales/ Product Management
Efficiency
No of features users discover seamlessly
What upsells/crosells apply to accounts reaching targeted levels of adoption?
Sales/ Product Management

Key Takeaways

Expansion

Accounts reaching expansion meet all four POEs standards. This the point where all variables following the adoption loop take full effect.

Frequency of Use

Despite the importance frequency has on all customer lifecycle stages, renewals and expansion teams should also consider:

  1. Exploring usage-based pricing when the majority of accounts have surpassed nominal expected levels.
  2. Which existing and upcoming features increase upsell and cross-sell opportunities.

Conclusion

What should you do next? Whether you invest in stellar product-led experiences or not, at the end of the day users still evaluate product delivery. A bad experience is still an experience, following customers’ growth or just preventing it. Transforming your product experiences to a data-driven force is not optional anymore. It is the only way to reach and surpass the desired growth levels.

Product-led practices constitute a challenge, as they dictate alignment across all organizational practices. But at the same time, they leverage and map product experiences to the very end.  Take a closer look at the examples provided above and try to go out of your way. Experiment and iterate until you reach the point where you can predict the customer journey’s anomalies. Try to remember, while there is no panacea on evaluating users behavior, by considering only business & marketing metrics will lead to miscalculations.

Keep the conversation going

We are convinced that product-led practices are key to achieving cross alignment and product success. Something not possible if specific benchmarks driven by users’ actions are not in place. Revisit your product delivery, by putting to work some of the practices discussed above and feel free to reach out to discuss the results. We are always keen on taking your feedback, brainstorming and helping you deliver product-led experiences leading to measurable growth outcomes.

More On Product Led Growth

product_led_growth_metrics
Blog

The Beginner’s Guide to Product-Led Growth Metrics

Product-Led growth metrics is a rather conflicting matter for every organization in SaaS currently. This comprehensive guide examines what should organizations do when striving to…
product_led_growth
Blog

Product-Led Growth in SaaS: Identifying The Challenges & Opportunities

Editor's note This article was first published on Userlane's Blog and covers thoughts & conclusions from “The State of Product-Led Experience” a comprehensive report on Product Led…