A bartending robot that can engage in personalized interactions with humans

A widely discussed application of social robots that has so far been rarely tested in real-world settings is their use as bartenders in cafés, cocktail bars and restaurants. While many roboticists have been trying to develop systems that can effectively prepare drinks and serve them, so far very few have focused on artificially reproducing the social aspect of bartending.

Researchers at University of Naples Federico II in Italy have recently developed a new interactive robotic system called BRILLO, which is specifically designed for bartending. In a recent paper published in UMAP ’22 Adjunct: Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, they introduced a new approach that could allow their robot to have personalized interactions with regular customers.

“The bartending scenario is an extremely challenging one to tackle using robots, yet it is also very interesting from a research point of view,” Prof. Silvia Rossi, one of the researchers who carried out the study and the scientific coordinator of the project, told TechXplore. “In fact, this scenario combines the complexity of efficiently manipulating objects to make drinks with the need to interact with the users. Interestingly, however, all current applications of robotics for bartending scenarios ignore the interaction part entirely.”

Silvia Rossi and her colleagues Alessandra Rossi and Nitha Elizabeth John believe that to effectively take on the role of a bartender, a robot should not only be able to interact with humans, but it should also create a “profile” of users. This would allow it to personalize its interactions with regular customers, increasing the likelihood that they will like and continue using the robotic bartending service.

“In the end, we think a barman should not only be the one who remembers your tastes, but also your interests and your daily life. Just like human bartenders, they should sometimes act as close friends,” Rossi said.

BRILLO, the bartending robot used by Rossi, Rossi and John consists of a humanoid bust with two robotic arms, which allow the robot to make drinks, and a monitor-based face that can produce different facial expressions. The robot also features a microphone, speaker and camera that allow it to capture images of customers and pick up their body language, process what they are saying and answer back to them.

In contrast with other robotic systems developed in the past, however, BRILLO is designed to engage with customers in a personalized way, providing unique recommendations based on their preferences and processing both their verbal and non-verbal cues. In addition, the robot creates user profiles that allow it to adapt its topics of conversation for regular customers, for instance remembering their interests and whether they prefer to engage in serious or funny conversations.

“Personalization is achieved by the combination of a so-called ‘system of record’ (e.g., by analyzing and storing past interactions in terms of past purchases but also the topics of interaction and the type of interaction and a ‘system of engagement’ (e.g., by improving the experience via a run-time evaluation of the current engagement level),” Rossi explained.

The innovative system created by this team of researchers allows their robot to process what a human user is telling them and their non-verbal cues, to determine what mood they are in, how attentive they are and what types of drinks they prefer. This information is stored by the robot and used to guide its future interactions with returning customers, so that they also consider their personalities and personal stories, along with their drinking preferences.

“Of course, our aim is not to build robotic bartenders, but this scenario is a challenging testbed to investigate how a robot can maintain and update a profile of the user to continuously adapt its interaction with them according to a profile that continuously changes,” Rossi said. “We believe that this kind of personalization is the key to the long-term acceptance of robots in our life and with different service and assistive roles.”

So far, the researchers have tested a prototype of their robotic system in an experimental setting. Their initial results were promising, so they would now like to also assess their system’s performance in a real-world environment, where it will be interacting with many different human users.

“We need to conduct more experiments in real-world settings, allowing customers to interact for a long time with our bartending robot to effectively validates our hypothesis,” Rossi added.

More information: Nitha Elizabeth John et al, Personalized Human-Robot Interaction with a Robot Bartender, Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization (2022). DOI: 10.1145/3511047.3537686

© 2022 Science X Network