AI and VR help job center advisors practice difficult conversations

No, he is not prepared to move for a new job, says Michel Lichtenstein. No, it's not his fault that he can't find a job. It's the fault of the company, which has no interest in his talent, even though he has a perfect A-level in his pocket.

"Whew, that sounds like a long conversation," says Dominik Erdinger, case manager at the Friedrichshafen job center, as he takes off his VR glasses. He had used the goggles to put himself in a virtual office where he met the unemployed academic to talk to him about his job prospects. "But the customer is well met."

In the case of the unruly academic, well met means that the students of the ZU course "Virtual. Intelligent. Social? VR & AI in social skills training" created a precise psychological person profile and injected it into an artificial language intelligence. The person profile was to have certain character traits and bear the name Michel Lichtenstein.

Case manager Erdinger therefore spoke to an artificially generated person. And he finds that the conversation he had with this agent, with all the arguments that were exchanged in it, was quite realistic. People who react so stubbornly really do exist, he says. Very often academics behave as dismissively as this Michel Lichtenstein.

The director of the Friedrichshafen Job Center, Maria Gerard, visited Zeppelin University this Thursday with four of her case managers to watch the students' presentation. The course was a collaboration between three professors: Raphael Zender, Chair of Virtual Reality Systems, Mateusz Dolata, Chair of Artificial Intelligence, and Steffen Eckhard, Chair of Public Administration.

The students worked in different groups during the course: One group came up with the virtual office - with furniture, windows, a view, props and pictures on the walls - while the other group came up with three person profiles and integrated their character traits into the artificial voice intelligence.

The visitors from the job center were impressed after the presentation. What VR and AI make possible here is a conceivable training situation for case managers to prepare for difficult customers. They agree on that.

How can such tricky conversations actually be practiced? It would take a great deal of effort to simulate conversations as realistically as AI and VR make them possible - with actors, for example. But such an effort would hardly be financially viable. Of course, role-playing with colleagues would be conceivable. But they would also have to have time first. AI and VR always have time and - once set up - are comparatively cheap to maintain.

"What I'm still missing are facial expressions and gestures," says Yannic Fiegel after speaking to the artificial customer named Elke Specht. A woman who makes a great effort, but is almost completely resigned and frustrated by the many rejections. "Body language is very decisive for the outcome of a conversation," explains Fiegel. Incidental remarks or unconsciously shown feelings often allow us to look behind the façade and recognize influences such as domestic violence or drug addiction. The profile is coherent, but eye contact is missing.

"The technology isn't that good yet," explains Prof. Zender, "but of course it will be able to do that at some point." From now on, the systems will become more and more realistic and human-like. And it is only a matter of time before facial expressions and gestures can also be reproduced very realistically.

Zender also emphasizes that this is still just a prototype that students have developed. The project is therefore still in its infancy and can still take on board feedback from the case managers and develop it further. The students now have to adapt to this feedback. The valuable feedback from the case managers is part of the documentation of the project and therefore also part of the examination results.