The emergence of General World Models (GWMs) is creating new possibilities for the audiovisual sector. As AI becomes increasingly capable of generating realistic worlds, interactive characters, and personalized experiences, success will depend not only on technical performance but also on how people interact with these systems.
Understanding how users respond to AI-driven feedback, when they seek support, and how different people engage with intelligent systems is becoming an important part of designing future AI experiences. Within the TETRA GWM project, part of our work therefore focuses on the human side of AI-supported and immersive technologies.
Imec-mict-UGhent’s research addresses fundamental questions highly relevant to future AI-generated environments:
- How do people react to adaptive feedback?
- How can intelligent systems recognize when support is needed?
And how can AI experiences become more responsive to individual users?
In this post, we will expand on two of our studies that aim to answer the above-listed questions.
Study 1: The Magic-Mirror
In this study, we explore whether seeing live information about your body’s signals can help you relax in virtual reality.
Imagine entering a calm virtual room with warm wooden surfaces, plants, and natural light. In front of you stands a mirror showing a virtual body that moves exactly as you move. When you raise your arm, the virtual body raises its arm too. This matching of real and virtual movements, known as visuo-motor synchrony, helps create a feeling of embodiment: the sense that the virtual body belongs to you.
An audio guide leads you through a relaxation exercise involving muscle relaxation and slow breathing. At the same time, a virtual heart and lungs appear on the mirror. These icons change colour in real time, becoming greener as heart rate and breathing slow down and redder as they increase. This process is known as biofeedback. By making otherwise invisible bodily signals visible, biofeedback may help people better regulate their physiological state during relaxation exercises.
But does it actually make relaxation more effective?
To answer this question, we invited 80 young adults to participate in two VR relaxation sessions, one week apart. In one session, participants received live biofeedback through the heart and lung icons. In the other session, they completed the same exercise without the feedback. The virtual body remained synchronized with participants’ movements in both conditions.
Our preliminary findings show that participants reported feeling less anxious and more relaxed after both sessions. However, adding biofeedback did not produce a clear additional improvement in subjective feelings of relaxation or anxiety. Interestingly, we did observe a measurable difference in breathing behaviour. Participants breathed approximately one breath per minute more slowly during the exercise when the biofeedback was visible.
This suggests that the visual feedback helped participants follow the breathing instructions more closely during the exercise. These findings suggest that biofeedback may influence behaviour even when participants do not consciously perceive greater relaxation.
Although this study was conducted in a relaxation context, it addresses a broader question that is highly relevant for future AI-generated experiences: how users respond to real-time personalized feedback. As GWMs make increasingly adaptive virtual environments possible, understanding when such feedback helps users and when it provides little added value will become an important design consideration.
Our discussions within the TETRA team focused on what these findings mean for future immersive experiences and how they can inform follow-up research. We are currently exploring participants’ interview responses and existing literature to identify promising next steps.
Study 2: The AI Tutor
An AI tutor can be very effective at answering questions. However, one challenge comes before the answer itself: knowing when someone actually needs help.
Traditional tutoring systems often wait until learners explicitly request assistance or make a mistake. But learners do not always ask when they need support, and timing matters. Intervening too early can disrupt productive learning, while stepping in too late may increase frustration. This led us to a different question:
Can we recognize that someone is about to ask for help before they actually do?
Can we predict help-seeking?
We explored this question in virtual reality. Twenty-one participants completed a rule-based assembly task in a virtual workshop. Across 120 trials, they assembled and sorted objects according to a set of rules that occasionally changed without warning. Whenever participants wanted to check the rules, they could consult a virtual instructor, who displayed the relevant information on a virtual board.
Throughout the task, the VR headset continuously recorded where participants looked and how their head and hands moved. This allowed us to examine behavioural patterns in the moments leading up to each help request.
Using these data, we trained a machine learning model to predict whether a help request would occur shortly afterwards. Importantly, the model used only non-verbal behavioural signals such as eye, head, and hand movements. It had no information about task performance, progress, or errors. To evaluate the model, we always tested it on participants it had never encountered before.
The model predicted upcoming help requests approximately 2.5 times better than chance and performed above chance for all 21 participants. This indicates that behavioural signals contain useful information about future help-seeking behaviour.
We also examined which behavioural patterns contributed to these predictions. Across the 44 behavioural features we analysed, 30 showed significant differences before help-seeking events. The strongest pattern indicated that participants tended to display more sustained gaze behaviour and slower head and hand movements before requesting assistance.
Importantly, these signals should not be interpreted as direct measures of uncertainty or confusion. Rather, they are behavioural patterns that tend to occur before a learner seeks help.
The model itself remains far from perfect. While it detected around 40% of upcoming requests, only 13% of its alerts were followed by a help request. These results are not yet sufficient for an AI tutor to intervene autonomously, but they demonstrate that anticipating help-seeking is a realistic possibility.
A particularly important finding was the extent of individual differences. Some participants requested help in only around 4% of trials, while others did so in more than 40% of trials. Although the general model performed above chance for everyone, prediction accuracy varied considerably between participants. For some individuals, performance was only slightly above chance. For others, it exceeded five times chance levels.
These findings suggest that behavioural signals alone are unlikely to be enough. Understanding the person behind the behaviour may be equally important.
Looking Ahead
Although these projects explore different domains, they share a common theme: understanding how people interact with intelligent and adaptive systems.
For the TETRA GWM project, this human-centred perspective is essential. Future GWM-powered applications are already generating worlds, characters, and interactions in real time, but their success largely depends on how well they respond to the people using them. Designing effective AI-driven experiences requires understanding how users react to feedback, when they need assistance, and why different individuals engage with technology in different ways.
By investigating these questions today, we help lay the groundwork for more adaptive, responsible, and human-centred applications of emerging GWM technologies in the audiovisual sector tomorrow.