Comparing two mobile applications that are used to help insomnia patients

The question

Our client was investigating digital solutions that would support insomnia patients utilising Cognitive Behavioural Therapy for Insomnia (CBTi). CBTi is a safe, clinically validated solution that addresses the underlying cause of insomnia, not the symptoms. There are no typical side effects and patients do not need a prescription.

During their research, they identified two already-existing, mobile solutions that they would like to test with insomnia sufferers in the U.S. Application one focused solely on the goal of improving one’s sleep, application two came with a wider scope in offerings, but less targeted to sleep. SUAZIO was asked to compare both applications, by means of a qualitative analysis.

The end goal

The end goal was quite clear: by testing the two mobile applications, each with their own accompanying wearable accessoires, we wanted to gain more understanding about which application was perceived as more favourable by our target audience.

The app that would prove to be the best would then be considered for acquisition and further development.

Our approach

We recruited a small target audience of insomnia patients with an Insomnia Severity Index of 15+. After a thorough screening, this group agreed to test each application for one week as well as take part in a 25-minute interview per application to summarise the successes and challenges associated with each product.

The question path

During the use of each application, the respondents were asked to fill out score lists.

During the exit interview, we asked some in depth questions about these scores.

Based on these answers, we’d find out which app scored best and therefore had the highest likelihood of purchase.

Our question path during the interview was divided up in six chapters:

Product evaluation: we talked about their assessment of the product, their likes and dislikes and asked about their ratings on key attributes such as ease of use, comfort and likelihood to recommend.
Purchase evaluation: we asked about their personal likelihood to purchase, their assessment of the identified price and listened to their recommendations to increase the likelihood to purchase.
Pain points: we talked about the duration of setup and the reporting/confirmation of issues.
Sleep impact: we did a reassessment of the Insomnia Severity Index and made an assessment of changes in sleep.
Appearance and app: we talked about the importance of appearance, their rating of the current appearance and their likes and dislikes of the mobile application use.
Product tournament: we ended the interview by asking them to make a choice between the two applications based on overall experience, ease of use, etc.

We deduced our conclusion based on following three characteristics.

Overall choice: at the end of the product testing phase, we told the respondents they could keep one of two applications. This helped us determine which of both products they were more likely to purchase.
Sleep improvement: which product delivered the best results in improving the respondents’ sleep (falling asleep, staying asleep, worrying about falling asleep, etc).
Key attribute review: feedback on appearance, comfort, ease of use, fulfilling needs, etc).

We delivered the results of our questionnaire as well as a brief summary with key takeaways to our client, so they could make a substantiated decision about which application (not) to acquire.