Measuring Impact
A well built Progressive Web App should increase engagement, speed up the page and drive conversions through a meaningful offline mode and installability. So what’s the best way to measure this and determine the impact?
As a PWA includes a few different techologies which can be used somewhat independently, it is advised to measure each feature on its own to ensure clear data and a complete picture on what to optimize further.
Measuring Install Rates
Making a Progressive Web App installable is a powerful feature, but success of it is also very dependent on the UX patterns used for promotion - the same as with install prompts for native apps.
Therefore, make sure to have the right analytics in place to track installation rates and engagement/conversion rates, so that the flow and UX can be optimized.
On Chrome and Microsoft Edge, the entire funnel can be tracked as follows
Step | What to measure | How to measure |
1 | # of users eligible to install | Listen to beforeinstallprompt event, which will fire for installable sites when the user meets an engagement heuristic |
2 | # of users who click on UI install prompt | This will depend on how exactly the prompt method is called, but it’ll probably be from a click event on a UI element in the page |
3 | # of users who accept / decline to install | After a user clicks the UI install prompt, the chrome install popup will trigger. Measure whether a user has accepted or rejected in the userChoice property |
4 | # of users who have installed | Successful installation can be measured with the appinstalled event |
For other browsers
For other browsers that support beforeinstallprompt event, similar analytics can be instrumented.
For those that don’t (like Safari), it is harder to track PWA installs. In these cases it is recommended to at least measure the usage for the installed PWA as described in the next section.
Measuring Installed Users
It is important to understand the behavior of users that have installed the Progressive Web App.
The first thing to track is how many users are opening the PWA from the homescreen (or in general the PWA icon). This can be done by adding a tracking parameter to the launch URL in the manifest, so that homescreen traffic can be drilled into within your analytics package.
The PWA can also be accessed when a user clicks on any link (see: deeplinks), such as from a search engine, or also via a web share target. Therefore, it is important to also detect dynamically through javascript if users are in standalone mode (thereby indicating usage of the installed PWA), and then use this as variable or dimension for your analytics tracking.
TWA Usage and Events
When a Progressive Web App is published in the Play Store using Trusted Web Activities, many relevant metrics regarding install rates are directly available from the Play Console.
Besides this default tracking there are several ways to pass information into a TWA via URL query parameters, referrer URL or request headers, and all of these can be used to enable additional tracking.
Usage within the Trusted Web Activity is recorded via the regular website analytics code, but make sure to add a tracking parameter to the default url of the TWA, to enable filtering down to TWA users in analytics.
App Campaign attribution
App Campaigns can be used to promote a PWA published in Play using Trusted Web Activities.
To attribute installs, use Google Play campaign attribution, or a native app analytics SDK e.g. Google Analytics for Firebase.
To attribute more than just the install to the campaign, it’s a little more complex, as what happens in the web part (the TWA) is separate from the native part (what’s installed). Here is a specific example on how this can be done with AppsFlyer by passing tracking parameters through URL or headers, combined with server-to-server tracking. Many App Analytics packages support similar server to server integration which can be used to track TWAs accurately.
Measuring Offline & Caching
Offline
Offline usage can be measured by recording analytics pings when offline, and then replaying them when connection comes back, as described here.
For Google Analytics this is readily available in a module for the Workbox library, and means all analytics events will still be tracked, even when the user goes offline.
It may be useful to add an explicit tracking for when the user goes offline or online, by using the corresponding browser events. This way allows measurement of how many users that went offline and could be recovered, and how much time is spent offline (by calculating the time between online and offline events and sending this towards your analytics). Keep in mind however, that there will only be a lot of offline activity if there is a good offline experience.
Caching & Conversions
A service worker can bring more uplift than just offline by handling unreliable network conditions.
Therefore, another way to measure impact from improved network resilience and offline is to compare conversion and engagement rates with and without service worker caching as it can be assumed that a more robust and resilient website will convert better. Results should be filtered down on carrier connection and mobile, to see the impact the strongest.
Here is one way to do such an A/B test, where only a subset of users get the service worker installed via cookie split.
Caching & Speed
Well defined caching strategies through a service worker can make a website faster on repeat loads. Speed-up through a service worker can be easily measured through common speed tools like Lighthouse or Webpagetest.org by measuring once with and once without service worker, and compare both runs. Keep in mind that the service worker is installed on first load, and typically has most impact on page loads after the first, so make sure to measure speed on repeat loads or with prewarmed cache.
For field data, use your analytics package of choice, filter on performance metrics for repeat loads or returning users, and compare data before and after service worker launch.
Another option here which doesn’t need additional setup is to just monitor field performance through the Chrome User Experience Report before and after launch (which contains first and repeat loads, and should therefore show speed uplift through service worker caching).
Measuring Push Notifications
There is a whole funnel to keep track of when measuring push notifications. Similarly to install (and basically most browser permission prompts) it is highly advised to not directly bring up the default permission prompt, but rather use custom UI ahead of time to give the user a real value proposition why they should allow this feature.
Tracking and measuring the whole funnel is especially important for notifications, as sub-optimal implementations can easily be perceived as spammy and annoying by users, and therefore may lead to complete abandonment of the Progressive Web App. It may therefore be advised to look closely for engagement metrics of users receiving notifications, to see if there is attrition or churn.
With this in mind, the flow to track looks like this:
Step | What to measure | How to measure |
1 | # of users eligible for push notifications | Detect if push notification is available in the browser through feature detection |
2 | # of users who click on custom notification UI prompt | This will depend on how exactly this feature is offered to the users, but it will probably be from a click event on a UI element in the page |
3 | # of users who accept / decline push notification permission | After a user accepts the custom UI notification prompt, the chrome notification permission will be triggered. This will give back if the user accepted or rejected, which can be tracked in analytics |
4 | # of users receiving the push notification | Whenever a push notification is sent from the backend, the push event is triggered in the service worker on the client side. Use this event handler to measure how many users received the push notification successfully |
5 | # of users who engage with the notification | Whenever a notification (or one of the actions within it) is clicked by the users, the notificationclick event is triggered in the service worker. In here, add analytics code to track user engagement. Similarly, there is also a notification close event to do the same for users dismissing the notification without interacting |
6 | Measure conversions and engagement afterwards | Most of the time when a notification is clicked, it will open up the PWA. The code to do this can be found here, and can be used to add tracking parameters to the URL to mark the session as started from a notification (e.g. via utm_source in Google Analytics). However, depending on the analytics package custom code in thenotification click event handler can be used instead of using url parameters |