#ThinkProductGrowth

Evaluating Product-Led Experience

Via the Lens of Data-Driven Guidance

Get the Guide

Introduction

Stop for a minute and think. How many digital experiences do you encounter daily? At work, on your mobile, when you read the daily news? Our life is quite literally full of them. Daily millions of interactions are taking place by the second. Whether you are a SaaS owner, Product leader or Success practitioner you feel every minute how critical it is your product to deliver stellar product experiences. If you are a user, you experience first hand the frustration a broken experience carries and the satisfaction product delights deliver.

With Product-Led practices on the rise, competitive businesses need to fall the weight of responsibility on internal teams shoulders. Silos abandonment make very clear that customer experience does not have a single owner but it is compiled by the sum of touchpoints that originate and end up to the product itself.

Sales practitioners need to close deals reflecting on organizations’ vision. Product leaders need to deliver calculated results. Customer Success needs to act proactively, focus on customers’ growth and educate at scale.

We may think that we are in a transitioning era, but the truth is Product-Led practices have already sow their seeds. Their magnitude spreads like wildfire and their profound influence pass a very clear message: SaaS organizations need to either evolve by developing a customer-centric approach, empowered by the product itself or become obsolete.

In this guide, you will learn firsthand how data-driven engagements can be assessed, yield better results, and defined via the lens of Product-Led onboarding practices. We encourage you to share it with your internal teams and put to work those suggestions by always considering the particularities your products’ have.


The Road so Far..

The entire SaaS industry has been sold a false dichotomy, namely that a Sales strategy can either be delivered self serve or human-assisted – there is no in-between, or so we’ve been told.  The optimization of product engagement practices is a discussion that is not going away anytime soon. It seems that human nature is still either too afraid to accept that products’ superpowers can fill the customer experience gap or does not really know how to handle them.

On a self-serve onboarding, “the machines” prevail among the various product experiences, while on human-assisted they come secondary to customer-facing teams activations. While in the latter instance, the replacement of product engagements with human-assisted activations partly resonates, since multiple stakeholders are involved, Product Led practices flip the script and project the product as the main growth lever.

It is obvious that the realm of stellar customer experience is not solely defined, anymore, by the buyer-vendor relationship dynamic. Serious Investment in targeted, data-driven product experiences are here to successfully bridge the experience gap humans leave behind. A gap that has nothing to do with the lack of exceptional strategic skills. On the contrary, it relates to the undeniable fact that meaningful interactions are driven by the product features’ in conjunction with product guidance that leverages context of usage.

On the flip side, organizations in favor of a self serve approach tend to throw messages at random and force users to follow a predisposed route. A fact that discourages activation rates and makes accounts susceptible to churn. Organizations ingrained into Product-Led practices, on the other hand, already allow product data to dictate the “where” and “why” product engagements should take place.

Make no mistake, this investment does not indicate displacing tooltips or in-app guides just for the sake of experimentation per se. Any iteration should follow a specific logic and executed when the onboarding team is able to justify that change. In the opposite case scenario, displacement of users’ attention will lead to more confusion and friction, while keeping at the same conversions and retention rates low.

The third leg of the tripod, Product Management’s “agile” DNA followed by ongoing feature releases, impracts radically the onboarding experience too. The emerging role of product leaders enables product guidance to introduce changes on time, flawlessly by providing the right context. Product Management needs to measure those interactions daily to reassure their investment yield the necessary results.

Currently, each department holds a different set of business metrics accountable for the productivity and input to the customer journey. Following this logic, product metrics need to be established and add value to existing KPIs by measuring activation, retention, and engagement levels.


Product-Led Experience

Unlike conventional onboarding practices, Product-Led Onboarding  is driven by product data and can be evaluated on all the stages of the customer journey. The ongoing feature releases discourage the iconic sales funnel taxonomy. On every release, the onboarding process is reactivated to deliver initial value, lead to upgrades, and further account expansion. A process that abandons the traditional sales model archetype, ending onboarding prevalence during activation.

The sales funnel has evolved into a circle where onboarding stands in its epicenter waiting for the next feature release to be triggered again. Furthermore, capitalization on product data enables onboarding owners to derive insights on every move a user makes in-app. Those learnings can harmonize, in the long term, the high touch & high tech interactions and make onboarding a tailored process able to be measured end-to-end.

It won’t be long now until new terms will come to describe the intimacy levels of the User-Product relationship. The first associated term so far is the Product-Qualified lead (PQL), referring to prospects that signed up and demonstrated buying intent based on product interest, usage, and behavioral data.

The PQL term is limited to the point where a paid conversion is made, having as main benchmarks usage and early adoption. For prospects’ behavior to be accurately evaluated organizations should consider Product OQLs™ (Product Onboarding Qualified Leads) in their day-to-day assessments. Product OQLs™ rely on POEs metrics to evaluate prospects’ usage and in product behavior. Despite the fact that there is no absolute as to which metric should prevail, balance among all those four measurements indicates that a prospect gets initial value.

Product Onboarding Efficiency (POE)

Product Onboarding Efficiency (POE) is an onboarding evaluation framework relying on four product engagement variables Breadth, Depth, Efficiency & Frequency of use to accurately monitor and guide Product-Led onboarding (PLO) efficacy throughout the customer lifecycle. In an attempt to better explain those elements’ importance we will thoroughly analyze their characteristics and associations following them.

Product Engagement Variable: Breadth of Use

An alternate form of (team) activation and retention constituent, breadth helps product managers realize the extent a product is being used on an account level. As a product metric, it monitors account health and helps product managers & data owners proactively manage churn.

High Trajectory Customers

For Mid Market and Enterprise customer segments, internal buy-in from end users is discouraging onboarding deployment. Heterogeneity on proficiency levels and complicated workflows decrease end users’ willingness to adopt a new solution. Focus on Team Activation (and by extension Team Onboarding), historical usage and use case particularities, is an optimal way forward, internal teams can take to overcome this barrier.

Self Serve Customers

1. Having Self Serve customers inviting team members during trial embrace activation rates. In the absence of Sales and Customer Success guidance, in-app flows can double down on inviting team members and emphasize on how their involvement increases perceived value.
2. Product teams should define which roles may cause various drop-offs on their first-week activation cohorts and iterate flows accordingly.
3. Training procedures need an equally concerted focus on each user role separately versus team training.

Team Activation Use Case: Hubspot

One of the key activation metrics predicting usage and retention for Hubspot, the leading automation suite, is how successfully the solution onboards teams within an account. Product teams monitor weekly team activities within accounts and whether or not the solution is driving value. We should mention at this point that team activation associates with Hubspot’s long term success and substitutes its north star metric as well.

Breadth of Use- Points to consider

1. Breadth of use importance is not limited to the activation stage. Every time a new release is launched product teams should monitor breadth in conjunction with the degree end users’ workflows are affected.
2. How many users log in when onboarding is carried out and how many after it ends?
3. How many users are activated when a new feature release is launched. (Assuming that the release is targeted to users’ finding value in this feature)?
4. How is depth defined during trial and how post-purchase? (Varying parameter per product/strategy/pricing plan)
5. What characteristics follow users sustaining engagement levels (retention) and those who abandon the app?

Breadth of use product onboarding calculations formula

Product Engagement Variable: Depth of Use

High Trajectory Customers

Points of consideration here are end users’ growth within the product itself. Daily usage should be monitored and reported on an account and individual level. Internal teams (Customer Success, Customer Support & Product Management) should focus on setting milestones per each role separately in conjunction with users’ proficiency evolution over a 12-month time span.

Self Serve Customers

While in the case of high trajectory customers depth will be realized on some levels eventually, due to customer facing teams activations, Self Serve customers may never get there. Lack of personalization and investment on tailored onboarding flows lead to high churn rates almost instantly after subscription. Depending on each organization’s internal practices adoption may be evaluated within the first 7 days or after the end of the trial period. Again here milestones should be set but:
1. Since Small Medium Businesses have smaller teams, Depth should be measured in conjunction with Breadth to accurately evaluate engagement levels.
2. Milestones should be renewed monthly since self-serve customers are not often bound by contracts.

User Segmentation Use Case: Gainsight PX

First Time Activation:
Gainsight PX, the leading product experience solution, segment users’ behavior, and usage on first-time activation by having milestones track if PQLs complete setup by using key features during trial. When prospects use features more than the freemium version allows them to, internal teams measure conversion rates and revenue generated from the trial source.
New Product Release:
On new product release, users’ behavior is segmented by measuring success in terms of adoption. Specifically, the product team releases targeted in-app guides to users based on historical product usage. The Query Builder feature, one of the key features, is presented to users who have shown interest in other analytics areas the past 30 days. If the released feature is a paid only module, the revenue is measured by attaching a rate on it.

Depth of Use- Points to Consider

1. Which levels of adoption (Depth of Use) correspond to which user role?
2. Does adoption increase due time? (aka are users growing within the service?)
3. Are internal teams able to reflect usage levels on Net Revenue and/or Net churn?
4. How is depth defined during trial and how post-purchase? (Varying parameter per product/strategy/pricing plan)
5. Which adoption levels indicate signs of accounts’ expansion?

Depth of use product onboarding calculations formula

Product Engagement Variable: Efficiency of Use

The difficulty level to complete common tasks is a critical evaluation of the onboarding flow. In order to be measured accurately, product teams need to be aware of the total number of users per account who begin a task, versus those who complete it.

High Trajectory Customers

1. Being a composite of both human-assisted and product activations, for high trajectory customers efficiency of use does not rely solely on product onboarding flows’ effectiveness. The emerging investment in product data however, flips the script and makes stellar product experiences the growth lever following accounts’ long term prosperity.
2. Organizations having to onboard multiple teams need to invest in product onboarding flows tailored to users’ role early on, when striving to evaluate usage and adoption. That realization by default indicates Sales involvement in the onboarding process.This is the only way forward to make onboarding measurable from the very beginning and harmonize high touch & high tech activations:
Keypoint One:
Sales teams should claim ownership of trials’ product data and observe prospects’ behavior equally in and out of the product. By capitalizing on active and passive feedback sales teams will be able to pinpoint instances where product experience may be broken and how it should capitalize on end users’ existing workflows (context). Investment in product data enables Sales teams onboard the first round of teams and pass the necessary feedback to Customer Success which completes the onboarding process.
Keypoint Two:
Customer Success can further analyze Sales feedback and make suggestions to buyers regarding the next steps going forward, based on real insights. Product data capitalization, help Success practitioners:
a) Get the internal buy-in easier
b) Be aware of the paths users take in-app
c) Act proactively whenever flows onboarding downgrade product experience
3. Product management monitors usage and trends users are forming depending on their role, profession, and proficiency level. Points of consideration here are key features adoption and when that is achieved seamlessly or with the assistance of customer-facing teams. In case the levels of human-assisted activations are above the expected, Product Management should inject those learnings in the product onboarding flows to further optimize them.

Self Serve Customers

Having the product replacing human assisted activations requires continuous iterations and heavy experimentation. Being completely autonomous and subject to a faster sales process Self Serve customers should be driven directly to initial value in conjunction with the learning curve period.

Internal teams need to know when they strive to shorten trial periods, that as much as speed to implement needs to come first, products’ complexity affects the learning curve period and the onboarding flow too. As intimidating factor the learning curve period can be, users need to familiarize themselves with the product by following their own pace. Internal teams should evaluate on how many parts the onboarding flow should be broken down until adoption is realized.

Multiple Value Points Delivery (First time activation) Use Case

Drift the leading conversational platform’s former onboarding strategy was a quick process constituted by three steps, aiming to get users install its javascript code. That resulted in high levels of churn as free users did not feel invested in the product. In an effort to deal with that effect, the onboarding team launched a ten steps onboarding flow where each one of them motivates users to complete three different tasks. The end goal is to create many wow moments and make users feel invested when the set-up is complete. The flow has proved to be a tremendous success as it ended up tripling conversion rates.

Effectively Reducing Trial Period (First time activation) Use Case

When Yesware, the leading email software, decided to invest heavily in a product-led, experiment-driven onboarding model, one of the first goals was to build an infrastructure allowing rapid testing.  In this vein, the internal teams decided to run an experiment that halved the free trial length from 28 days to 14.

The hypothesis was that while the conversion rates would remain steady (the product team believes that 14 days are sufficient for qualified users to experience the product), the product team would benefit significantly by being able to run tests twice as fast. Additionally, there was the expectation that shortening the trial would provide a sense of urgency and spur users to take action quickly.

Because of certain technical restrictions, the test was run longitudinally as opposed to as an A/B. Functionally this means that the product team changed the trial length on a given day and compared it with the new 14-day trial cohort to the preceding 28-day ones on a variety of metrics.

After a month of testing, the results were fantastic! Although, as expected, there was a slight increase in the percentage of users who uninstalled the product during the trial (roughly a 0.5% increase), the core hypotheses were seeming valid, as an uptick in the early engagement of the product was realized. That effect took place particularly with the advanced features, which are critical leading indicators for the solution’s power users. Something assumed to have happened due to the urgency created. Moreover, conversion rates did not only maintain but actually increased by roughly 18%.

Special note: After rapid experimentation, the product and engineering team have decided to extend the 14 day trial to 28 days in Gmail if users make specific actions, based around sticky features, in the getting started guide.

High-Touch vs. High-Tech Harmonization Use Case

Userlane, the popular onboarding S/W with main target audience enterprise customers, launched a while ago an alternate asynchronous onboarding in addition to its Human Assisted strategy. The “academy” as it is called internally exploits the vendor’s features to self serve end users all the way through without eliminating the required decision making between the two parties. Customer Success calls that learning process “getting to Basecamp” where the various teams build their first Userlane guide and get them published. In the end, the users are certified and know the percentage of tasks they can complete. On top of that, the buyers’ are fully aware of the end users progress at all times.

The launch of the “Inception project”, improved onboarding process delivery tremendously. The implementation of targeted in-app guides within the product allowed a thorough personalized, albeit automatic onboarding process that didn’t require the active participation of different units. Additionally, time to initial value was decreased and after launch, buyers could go through the set up completely autonomously.

Inception Project Desired Deliverables

a) Increase of engagement and activation
b) Reduce time-to-value
c) Decrease time to key features
d) Decrease onboarding completion time
e) Increase trial-to-paid conversions by at least 30%
f) Reduce the costs and efforts connected to high touch activities by 60%

Onboarding Completion (D1-2)

38%

Adoption

45%

Activation (W1)

47%

Buyers Engagement

49%

Daily Sessions Per User

67%

Daily Sessions Per Account

82%

Breadth On Account Level

84%

Key Actions Per User (W1)

187%

Conversions

200%

In App Engagement (W1)

225%

Key Actions Per User (D1)

255%

*Onboarding Costs were reduced by 63%* 

Across the spectrum, Userlane managed to optimize product delivery without compromising onboarding execution. Increased adoption rates (+45%) confirm that this scalable approach yields the expected ROI while keeping end users engaged and buyers aware of end results.

The stickiness every product strives to achieve is reinforced by actions that capitalize on product educational activations, to reach desired outcomes. Userlane is a living proof that organizations requiring tailored implementation and deployment can successfully apply in-app guidance and combine scalability with the human-assisted support customer-facing teams provide.

Efficiency of Use- Points to Consider

1. In this instance onboarding flows should laser focus on historic usage per user role and account level by taking a concerted focus when adoption levels increase.
2. First-Time Activation:
a) For solutions offering multiple features and products (aka Hubspot, Intercom, Drift) an optimal strategy, following first time activation, would be to break down the onboarding flows in small parts leading to unique value points that gradually increase users’ proficiency and investment.
b) Overly simpler solutions should heavily experiment with shortening trial length by focusing on users’ actions in app.
3. Internal teams should monitor constantly if product onboarding activations decrease customer-facing teams involvement.
4. Ownership of product data should be dispersed across teams to embrace the feedback loop and enable them act proactively to any anomalies occurred throughout the customer journey.

Efficiency of use product onboarding calculations formula

Product Engagement Variable: Frequency of Use

Frequency of use is concerned with how frequently and for how long users engage with products’ features. Reminding users why a specific feature is there in the first place and how it may further optimize their workflow is also something reliant to onboarding activations.

High Trajectory Customers

In regards to high trajectory customers, where the stakes are higher, usage levels are being closely monitored mostly by Customer Success. Product-Led practices embrace Product Management involvement in the process too by holding it accountable to deliver quantitative measurements explaining levels of adoption or churn.

Both teams should create common benchmarks and be alerted whenever features’ usage drops. Respectively, targeted product activations should be released to restore engagement levels before churn prevails. Despite the fact that this practice is also leveraged for Self Serve customers, in this instance, it should be monitored when human assisted activations prompt users return to the product.

Self Serve Customers

Having Self Serve customers returning to the product autonomously is something reliant both to product and email activations. In this instance, email practices at scale replicate human assisted activations, so it resonates that customer-facing teams should adopt the right tools to monitor this correlation. Solutions like Gainisht PX mapping extensively those activities is an optimal choice to consider.

In this instance, internal teams should invest in customer feedback early on:

1.Sales should suggest what learnings to insert into trial, per use case.
2. Product Management should closely monitor passive feedback
3. Customer Success should release scalable in app surveys by taking a concerted focus on the features paid accounts exploit vs. those neglected.

Frequency of Use- Points to Consider

1.Which features are used more frequently?
2. Which features correspond to each team’s use case?
3. How can teams be triggered in-app to increase usage in secondary features?
4. When and why each feature should be used? ( to better estimate the revenues and losses following them)
5. What made users return to the product?
6. What made users abandon the product?

Frequency of use product onboarding calculations formula

Adoption Loop

All things being equal, Breadth, Depth, Frequency, and Efficiency of use form an adoption loop where its implementation is viable when the maximum number of an account’s users exploit a product, use its features extensively and repeat those actions frequently.

Again, the decisive role initiating this circle of events is (team) activation. But for the loop to be consistent, on every stage of the customer journey, all four metrics should be considered. Depending on the onboarding strategy at play, the circle is being supplemented by additional business KPIs and parameters.

Product Onboarding Efficiency (POE) - Breakdown

Activation stage (Indicative)

POE Variable
KPI
Points under Consideration
Department
Breadth
No of team members entering the solution
How many team members entered the solution?
Sales
Efficiency
No of users reaching initial value
Does the flow encourage team activation?
Sales/ Product Management
Frequency
How often users login in the s/w?
Are onboarding flows creating a habit early on?
Sales/ Product Management
Depth
Features explored during trial
What characteristics follow users exploring key features?
Sales/ Product Management

Key Takeaways

Product Management

The product team should be highly involved in the activation process, no matter the onboarding strategy at hand.

Sales Organization

The sooner Sales’ decode the messages withheld into product data the better the conclusions drawn, in regards to the harmonization of human-product acquisition activations.

Depth of use has some variables to consider:

1) Whether or not during activation period users can realize initial and true value. This is most likely to occur on high-velocity customers where onboarding in many departments takes place post-purchase.

2) Users’ proficiency levels need to be defined early on. An advanced user may explore more key features during trial, but this is also subject to a product’s complexity.

3) On Self Serve onboarding, Retention initiation point varies. Thus Depth of Use may not be measured on Activation (during trial) but only post-purchase.

Retention stage (Indicative)

POE Variable
KPI
Points under Consideration
Department
Breadth
No of team members entering the solution
How many team members activate frequently on an account level?
Customer Success / Product Management
Depth
Which features are explored end-to-end
Which key features prevail over others, usage wise?
Sales/ Product Management
Efficiency
No of users reaching true value
Are onboarding flows encouraging users to explore the solution to its full extent?
Sales/ Product Management
Frequency
Have users created a habit out of the product?
What is the percent of users returning to the product autonomously?
Sales/ Product Management

Key Takeaways

Customer Success & Product Management

Both departments are the rightful owners of accounts retention and customer journey optimization.

Depth

The retention stage defines which features prevail over the others. It is ideal at this point internal teams to define when accounts reach satisfactory levels of adoption (depth) and when they show indications of expansion.

Frequency 

Whenever frequency of use increases so do the indications of those accounts reaching long term retention.

Expansion stage (Indicative)

POE Variable
KPI
Points under Consideration
Department
Frequency
No of users returning daily into the product
What makes users return daily into the product?
Customer Success / Product Management
Depth
No of users exploiting features to their full extent
Which adoption levels indicate expansion opportunities?
Sales/ Product Management
Breadth
No of users using the product to its full extent inside an account
Are all users realizing true value?
Sales/ Product Management
Efficiency
No of features users discover seamlessly
What upsells/crosells apply to accounts reaching targeted levels of adoption?
Sales/ Product Management

Key Takeaways

Expansion

Accounts reaching expansion meet all four POEs standards. This the point where all variables following the adoption loop take full effect.

Frequency of Use

Despite the importance usage frequency has on all customer lifecycle stages, renewals and expansion teams should also consider:

1. Exploring usage-based pricing when the majority of accounts have surpassed nominal expected levels.

2. Innovating features that would help increase upsell and cross-sell opportunities within a customer’s contracted term.

Conclusion

What should you do next? Whether you are deeply invested in providing data-driven experiences or not, at the end of the day your users perceive the end result as your company’s main value. A bad experience is still an experience, following customers’ growth or just preventing it.

Transforming your product experiences toa data-driven force is not optional anymore. It is the only option if you want to reach and surpass the growth levels your organization has set.

Product-Led practices may constitute a challenge, as they dictate alignment across all organization practices, but at the same time, they can leverage and map product experiences to the very end.  Take a closer look at the examples provided above and try to go out of the conventional ways you use so far. Experiment, iterate, repeat until you reached the point where you can proactively predict the anomalies following the customer journey.

Try to remember that while there is no panacea on evaluating customers’ behavior, capitalization solely on business & marketing metrics leads internal teams to miscalculations that compromise product delivery. Only, product performance and customer feedback in conjunction with product data analysis can give accurate estimates on product experiences.

We are convinced that Product-Led practices are the key to achieving company-wide alignment and product success and this is not possible if specific benchmarks driven by users’ actions are not in place.

Revisit your product delivery, by putting to work some of the practices discussed above when striving to evaluate user behavior and feel free to reach out if you want to continue the discussion.

We are always keen on taking your feedback, brainstorming and helping you deliver Product-Led experiences that can lead to measurable growth outcomes.