EUR 1.8 million in funds for research into 3D avatars in telemedicine

German government funds AI and mixed reality project managed by University of Hamburg
25 April 2022
VR headset

The German Ministry of Education and Research has earmarked EUR 1.8 million in funds for the three-year Hybrid Interaction Systems for Maintaining Health Even in Exceptional Situations (HVIAM) managed by the University of Hamburg. The pandemic has highlighted the importance of remote medial care and the need to develop intelligent assistance systems for remote diagnostics using virtual 3D avatars. 

Artificial intelligence and mixed reality in remote diagnostics

During the project, the human intelligence of healthcare professionals will be combined with artificial intelligence (AI) and mixed reality (MR). AI-supported virtual 3D avatars of medical professionals will be generated, which can be displayed on mobile devices, 3D headsets or other MR applications. The avatars will use the sensors to record their environment as well as patient data such as blood pressure, blood glucose levels or weight in real time. Unlike previous voice assistants, the 3D avatars will be able to speak and show facial expressions and gestures to evoke familiarity among participants.

The AI-controlled avatars will be able to perform simple, routine tasks themselves. Medical experts will monitor the work of the virtual assistant later. If a patient wishes to speak to a doctor or nurse, they can arrange a real or virtual appointment through the assistant. "Patients can interact seamlessly with intelligent virtual assistants and avatars of doctors, therapists and nursing staff," said Prof. Dr. Frank Steinicke, a computer scientist at the University of Hamburg and spokesman for the joint project.

Strong Hanseatic participation

The participants include the Faculty of Computer Science at the University of Hamburg, the University Hospital Hamburg-Eppendorf, the Düsseldorf University of Applied Sciences and the Hamburg-based start-ups Sympatient and ApoqIar. The project volume amounts to EUR 2.12 million for the period 2022 to 2022, 85 per cent of which is being funded by the German government.