Towards Embedding Dynamic Personas in Interactive Robots:
Masquerading Animated Social Kinematics (MASK)

Jeongeun Park1, Taemoon Jeong1, Hyeonseong Kim1, Taehyun Byun1, Seungyoun Shin1, Keunjun Choi2, Jaewoon Kwon3, Taeyoon Lee3, Matthew Pan4, Sungjoon Choi1
Korea University, Korea1
Rainbow Robotics, Korea2
NAVER LABS, Korea3
Queens University, Canada4
Paper Paper (Short Version)

Supplementary Materials

User Subject Studies Protocols Additional Results Generated Behavior Database Prompts

News

Our paper got accepted in Robotics and Automation Letters (RA-L)!🥳🥳 .
We will be presenting our work at ICRA 2025.


Abstract

This paper presents the design and development of an innovative interactive robotic system to enhance audi- ence engagement using character-like personas. Built upon the foundations of persona-driven dialog agents, this work extends the agent’s application to the physical realm, employing robots to provide a more immersive and interactive experience. The proposed system, named the Masquerading Animated Social Kinematics (MASK), leverages an anthropomorphic robot which interacts with guests using non-verbal interactions, in- cluding facial expressions and gestures. A behaviour generation system based upon a finite-state machine structure effectively conditions robotic behaviour to convey distinct personas The MASK framework integrates a perception engine, a behaviour selection engine, and a comprehensive behaviour library to enable real-time, dynamic interactions with minimal human intervention in behaviour design. Throughout the user subject studies, we examined whether the users could recognize the intended character in both personality- and film-character- based persona conditions. We conclude by discussing the role of personas in interactive agents and the factors to consider for creating an engaging user experience.

Proposed method

Personality based Persona Demonstrations

Choose Persona

Extroverted and Cooperative

Character based Persona Demonstrations

Choose Persona

Minion

User Subject Studies Results

Hypohesis
We hypothesize that by embedding a character template into robotic behavior, interactive agents can convincingly embody distinct personas for users to engage with. In par- ticular, we aim to achieve this through a simple finite-state machine framework, which determines robotic actions from an observed user’s behavior, a current robot’s behavior, and a persona
Number of Participants in Phase 1
Number of Participants Extroverted Persona Introverted Persona Sum
Agreeable Persona Extroverted and Cooperative (13) Introverted and Cooperative (13) 26
Disagreeable Persona Extroverted and Uncooperative (13) Introverted and Uncooperative (13) 26
Sum 26 26 52
Number of Participants in Phase 2
Minions Cowardly Lion Scrooge Spock Total
27 27 27 27 108

Survey Phase 1 - Personality Based Persona (52 Participants)
During the first phase of the survey, the participants were exposed to personality-based persona robots. (between-subject studies)
The users were asked to score the extroversion (E-score) and agreeableness (A-score) of the robot.
We conducted a two-way repeated-measures MANOVA on the results obtained from user survey questions S1 and S2 with α = 0.05.
MY ALT TEXT
We observed that the users could recognize both extroversion and the agreeableness of the robot.
There was a significiant difference is measured E-score between interacted persona (extroverted - introverted).
In addition, there was a significiant difference is measured A-score between interacted persona (agreeable - disagreeable).

Survey Phase 2 - Film Character Based Persona (108 Participants)
During the first phase of the survey, the participants were exposed to character-based persona robots. (between-subject studies)
The users were asked to choose one of the six choices of the character and rate the matching score.
MY ALT TEXT
The classification accuracy was reported as 76.7%, the users were able to identify and classify the intended mimicing character. We observed that the users could successfully identify the Cowardly Lion from The Wizard of OZ, Minions, and Scrooge characters. However, participants appeared to misclassify the character of Spock, especially between Spock (0.44) and Scrooge (0.39).

For more details, please check
User Subject Studies Protocols or Additional Results

Persona Infuser

In the quest to embed robots with distinct personas, we introduce a persona infuser that constructs a behavior database. This database, acts as a blueprint for persona-driven behavior, encoding all possible combinations of states, observations, and transitions by large language models.


Generated Samples

Choose Persona


Graph Visualization

Extroverted and Cooperative

Selected States: look_around, wave_hand_big, attract_to_come_closer, wave_hand_small, small_bow, stand_still
Initial State: look_around
MY ALT TEXT

BibTeX

@ARTICLE{10643257,
  author={Park, Jeongeun and Jeong, Taemoon and Kim, Hyeonseong and Byun, Taehyun and Shin, Seungyoun and Choi, Keunjun and Kwon, Jaewoon and Lee, Taeyoon and Pan, Matthew and Choi, Sungjoon},
  journal={IEEE Robotics and Automation Letters}, 
  title={Towards Embedding Dynamic Personas in Interactive Robots: Masquerading Animated Social Kinematic (MASK)}, 
  year={2024},
  volume={9},
  number={10},
  pages={8826-8833},
  keywords={Robots;Engines;Behavioral sciences;Libraries;Databases;User experience;Kinematics;Social HRI;gesture;posture and facial expressions;design and human factors},
  doi={10.1109/LRA.2024.3447470}}