Foreword

Design thinking

My intention is to share my design thinking process for evaluating the user experience of virtual reality (VR) products.

This article collects my professional experience as a Lead UX, without any information about the projects I participated in.

NDA

I have signed a non-disclosure agreement with my employer.

This publication is a personal article. Any views or opinions represented in this publication are personal and belong solely to me. They do not represent my employer.

Virtual reality for pain relief?

Recent articles support the hypothesis that VR therapies can effectively distract patients with chronic pain (Virtual Reality as a Clinical Tool for Pain Management, PubMed.gov).

Indeed, the patient's immersion in virtual reality provides a distraction from painful stimuli and can reduce the perception of pain.

VR can also help reinforce the effectiveness of techniques such as physiotherapy, hypnosis and cognitive behavioural therapy to treat chronic pain (Virtual Reality as Therapy for Pain, New York Times).

An immersive, relaxing and interactive VR application can keep the brain so busy that it has no room to process pain sensations at the same time.

Evaluating the user experience of a VR product for pain relief

From a UX perspective, a VR product designed to relieve pain has two components:

1. The user experience

The person, beyond their condition as a patient, is an individual with their own mental model.

2. Experience with the therapeutic procedure

Stimuli specifically designed to change the perception of pain.

The user

As a UX Researcher, what interests me most is the pure "user experience" part.

The person, beyond their condition as a patient, is an individual with their own mental model: their technological culture, their emotions and their own cognitive abilities.

If the user experience is inadequate, it will be in conflict with the ultimate goals of the therapeutic procedure.

A good UX strategy must guarantee everything related to the interaction of the individual with the system.

UX evaluation tests will help identify areas for improvement. Once a satisfactory point has been reached, we can focus on the scientific part and work on the stimuli, their active principles and their validation under scientific study conditions.

UX references

Scientific and UX literature can help us ask the right questions and guide the research. Here is a selection of articles:

Structure of the UX evaluation phase

The evaluation varies according to the complexity of the project. Globally, we have these steps:

1. Research plan and test protocol

Identify each step and criteria. Document all UX work.

2. Definition of needs and goals

Define clear and measurable goals. Write the script for the user testing.

3. Testing

Test the prototype as soon as possible with a minimum of 5 target users.

4. Analysis and results.

Analyze the sessions and obtain clear and objective conclusions.

5. Presentation of results

Present results to stakeholders to assess feasibility and priorities.

Questionnaire for the evaluation of experience in a virtual environment

After testing the product, the user is asked to complete a user experience evaluation questionnaire.

The questionnaire is based on a set of criteria that collects the user's opinion and preferences on the virtual environment, as well as their state of possible cybersickness. It is completed by the participant immediately after testing the virtual reality application.

For each project I adapt the questionnaire, but my work is notably based on Katy Tcha-Tokey's thesis published in ResearchGate.

The questionnaire proposed by this publication is a unified questionnaire based on nine other existing questionnaires (PQ, ITQ, Flow4D16, CSE, AEQ, SUS, UTAUT, AttracDiff, SSQ).

L’un des critères qui m’intéresse est celui du flow. À l’état de flow (Flow on Wikipedia) is characterized by the complete absorption in what one does with a feeling of immersion and enjoyment.

More specifically in video gaming, the "game flow" (The Gamer's Brain, Hodent) is essential for successful engagement of the gamer or, in our case, the future patient, to finally succeed in the therapeutic treatment.

STEPS OF TESTING

1. First questionnaire

The participant completes a questionnaire on their demographic and technophile profile and a questionnaire on cybersickness.

2. Test of the VR application

The participant tests the VR application. Unlike UX standards, the "think aloud" method may not be advisable as it may break the immersion.

3. Observation grid

During the session (which is recorded) the moderator can complete an observational analysis grid.

4. Second questionnaire

After the test, the participant completes the experience evaluation questionnaire and once again the cybersickness questionnaire.

5. Nondirected interview

Qualitative interview led by the moderator with questions that do not lead or bias the answers.

6. Wrap-up

It formally completes the testing. We can thank the participant.

Ethics

The participant must feel listened to, respected and safe. Not only for ethical reasons, but also to ensure the quality, spontaneity and sincerity of their feedback.

The researcher must commit herself to ask only for information that is useful for her study and to treat it with anonymity and respect for the individual and the legislation.

No judgment should be made on the participant, quite the contrary, because the session is meant to evaluate the product, not the participant.

Cybersickness is a real problem in VR. Measuring and monitoring symptoms is mandatory and the participant must feel free to stop the test at any time.

Finally, nothing like a smile to stimulate mirror neurons 🙂. The researcher is the conductor of the orchestra: all she has to do is relax and enjoy these privileged moments of exchange.

How to integrate UX into a production pipeline?

Easier said than done... Indeed, the UX team is on a constant mission to empathize and pedagogically approach other members of the production team and stakeholders.

UX must be integrated into the planning from the beginning of the project.

In my experience, it's very important to involve stakeholders and listen to their opinions. It empowers the UX mindset of each team member and everyone can be a great source of proposals and creativity.

I invited my colleagues (game designer, 3D artist, sound designer, etc.) to attend the test sessions. Thanks to their involvement, they have a better understanding of the UX mindset and they can better understand the final user.

Finally, when the UX team effectively shows tangible, proven results like those of a test session, it's easier to align the entire team for the next iteration.

Conclusions

UX testing for a VR app of pain relief must be part of an overall strategy of user-centered design.

Conducting tests early in the production process is essential to effectively identify design problems. Detected later ("we don't have time, we'll do the UX after...!") the danger is that the product will end up being tested and evaluated too late, at a time when nothing can be done or only quick fixes can be applied.

Plan continuation bias

We should keep in mind the plan continuation bias (Sunken Fallacy), especially known in aviation safety.

According to this bias, humans tend to reject new options because they have spent a lot of time in the original plan.

Hence the interest of carrying out tests early in the production process.

Finally, it's all about never stop learning and seeing how a user, whether a is gamer or a patient, understands and interacts with our system without outside help. The UX researcher must fill the gaps between the mental model of the user and the mental model of the designers.