Socially-Assistive Robots for Dementia Care on Embodied Voice Assistant (Eva)



graph TD
  Mermaid --> Diagram

Eva platform is an open-source social robotics platform that provides you with the resources to build a social robot and its interactions cheaply, fast, and easily.



EVA-SIM, multimodal data, dementia, HCI, HRI, Embodied Voice Assistant (EVA)

Getting started

Welcome to EvaPlatform! We're thrilled that you've decided to join our community. Our goal is to provide you with an exceptional experience every time you use our platform. Don't hesitate to reach out if you have any questions or feedback.

Build your IoT prototypes faster with JavaScript and no servers here. You can find some interesting and useful notebooks on GitHub or full experiments on EvaStore. Test your amazing ideas with Evasim. Ready? Build your own Evabot following the instructions on Eva official repository.

Would you like to contribute to the project(s)?





Code mobility





Our main goal is to create an EVA platform to aid patients with dementia. To achieve this, we are currently focused on developing an EVA simulator using research from papers [5] and [6], as well as utilizing tools such as Blender, 3D printing, WebGL, and other relevant technologies. Because researchers strive for smart improvements, we bring out EvaSim [7], a simulator that enhances fast prototyping.

The development of the simulator will be followed by rigorous testing and evaluation of our system to ensure its efficacy in aiding dementia patients.


The software development process of this project is user-centered and RUP-based, this involves understanding the human beings who use our systems and their needs before writing a single line of code.

We’re going to find user needs by understanding the discourse of the domain, observing the user behavior in the current workflow, and prototyping insofar as we have uncertainty or doubts.

So, our goals are to understand the language from the available literature focused on HCI, Ambient Intelligence, and Social Assistance; print and build a robot model to identify improvement opportunities using the 3D printer and Blender; document the available software; write down the system requirements; develop the Eva-SIM [7] [8] —which has been built in Brazil; and work with other interested groups.

We propose the following work plan, with activities starting on 01-18-2023 and ending on 07-30-2023.

Work plan

Understand the language from the available literature focused on HCI, Ambient Intelligence, and Social Assistance
Print and build a robot model to identify improvement opportunities using the 3D printer and Blender
Document the available software
EVASIM prototypes and observations
Simulator requirements
Quarterly report writing
Final report writing
Software construction
Acceptance tests



Computer simulations

Robots as therapy

We use robots to assist people, we call such robots Socially Assistive Robots (SARs). Eva is a low-cost hardware SAR that assists dementia and geriatric cognitive disorder patients with non-pharmacological interventions. Nowadays, Evabot has become an Eva platform referring to the open-source social robotics platform that builds around it —EvaSim, Evabot, Eva Language, Evaweb, and so on.

Some of the multimodal interaction Evabot capabilities are Text-To-Speech (e.g. you choose vendors such as Google Cloud or IBM Watson), facial expression recognition, controlling light sensory effects, Wizard and Oz approach, and connecting IoT devices.

Serious games

The imitation game

The imitation game by Marcelo Rocha.

Game of Colors

The game of colors by Marcelo Rocha.

The game of emotions

The game of emotions by Marcelo Rocha.

Eva Language

Eva Language is a visual scripting language (VPL) built using Google Bloocky whose purpose is that non-technical people generate new interactions in the Eva language.

The language should include what is shown such that Eva Sim can show custom interfaces.

function NumericComparison({ A, B, OP }) {
    let value1 = parseInt(A);
    let value2 = parseInt(B);
    if (OP === 'EQ') {
        return value1 == value2;
    } else if (OP === 'NEQ') {
        return value1 != value2;
    } else if (OP === 'LT') {
        return value1 < value2;
    } else if (OP === 'LTE') {
        return value1 <= value2;
    } else if (OP === 'GT') {
        return value1 > value2;
    } else if (OP === 'GTE') {
        return value1 >= value2;

const ProcessCounterNode = function ({ ops, count, value, first, second }) {
    let temp = getCounter();
    if (ops === 'ADD') {
        temp[count] = getValue(first, temp) + getValue(second, temp);
    } else if (ops === 'MINUS') {
        temp[count] = getValue(first, temp) - getValue(second, temp);
    } else if (ops === 'MULTIPLY') {
        temp[count] = getValue(first, temp) * getValue(second, temp);
    } else if (ops === 'DIVIDE') {
        temp[count] = getValue(first, temp) / getValue;
    } else if (ops === 'DIVIDE') {
        temp[count] = getValue(first, temp) / getValue(second, temp);
    } else if (ops === 'POWER') {
        temp[count] = Math.pow(getValue(first, temp), getValue(second, temp));
    } else if (ops === 'assign'){
        temp[count] =  value;
    } else if (ops === 'random'){
        temp[count] =  generarNumeroRandom(parseInt(first), parseInt(second));

Interaction consists of code, script, and assets.


An example is “Pasalabra”

script/Rosco-Pasapalabra 20220404.json

We check out the Pasapalabra.json



VPL Blocks



Eva Sim

Due to the risk of damaging the assembled robot and a slow feedback loop, Marcelo Marques da Rocha proposed a simulator that interprets the Eva language and emulates the capabilities of Eva [7] [11]. This simulator is intended for use by developers, HRI researchers, and enthusiasts who need to safely and quickly test their real-world scenarios, enabling better training.

To summarize, the goal of the Eva Sim project, and therefore our goal, is to develop a simulator that allows Eva developers to prototype, learn, and iterate more quickly.

graph LR
  JSON["JSON (Eva Language)"] --> EvaSimParser["Eva Sim x Parser"]
  EvaSimParser["Eva Sim Parser"] --> XML
  XML --> EvaSimModule["EvaSim Execution Module"]
  EvaSimModule --> UserInterface

For example, on the simulator, you can run Serious games such as The Imitation Game (not the movie), Game of Colors, and Game of Emotions easily and with the same behavior as in the Evabot.

You can only visualize what is going on, but you cannot program on EvaSim.

TODO: Eva Sim user experience evaluation.

EvaSim Demo

EvaML (EVA Markup Language)

EvaML is an XML vocabulary in the Eva language domain that is obtained from another system by JSON. EvaML is deprecated.




Project vision

We would like to enhance EVA researchers in training, programming, and testing new interactions.


What is the current workflow for researchers to develop new interactions?

Without a server or broker, the current performance of the system is slow, leading to user frustration. To improve the user experience, faster prototyping is essential. Additionally, users interact more frequently with computer vision, computer voice, and gestures, making it necessary to optimize the system for these interactions.

Although the system could support multiple programming languages, we’ve chosen VPL due to its ease of use for developers.






Construction. Team versions.




Fluminense Federal University (UFF)



Universidad de Castilla-La Mancha




We are migrating the current version of EvaSim to the Unity platform on the Web. This will eliminate the need for researchers to go through an installation process, allowing for immediate use. Faster feedback from patients is crucial for research purposes.

Researchers require the ability to visualize the 3D robot and execute interactions in their preferred programming language. Additionally, they need to load customized environments, including keys for services such as Dialogflow, IBM Watson, ChatGPT, and others. EvaSim must support all features available on Evabot.

Researchers aim to connect to EvaSim through automation tasks such as "The Game of Emotions" and execute interactions automatically. However, due to cost and performance considerations, external servers are not allowed, and therefore EvaSim must run entirely on the client side. To meet these constraints, we propose using a Terminal with WebRTC and WebSockets, which enables EvaSim to operate without user interaction and run entirely on the client side.

Continuous integration


Testing and evaluations



[1] Dix, Alan J., Janet E. Finlay, et al. Human-Computer Interaction. 2nd ed. Prentice Hall, 1998. ISBN: 9780132398640.

[2] Olsen, Dan R. Developing User Interfaces (Interactive Technologies). Morgan Kaufmann, 1998. ISBN: 9781558604186.

[3] Cook, Albert M., and Jan Miller Polgar. Cook & Hussey’s Assistive Technologies: Principles and Practice. 3rd ed. Mosby Elsevier, 2007. ISBN: 9780323039079.

[4] Item 1007/3283 | Repositorio CICESE. (2020, August 06). Retrieved from https://cicese.repositorioinstitucional.mx/jspui/handle/1007/3283

[5] Raskin, Jef. The Humane Interface: New Directions for Designing Interactive Systems. Addison-Wesley Professional, 2000. ISBN: 9780201379372.

[6] D. Cruz-Sandoval and J. Favela, "A Conversational Robot to Conduct Therapeutic Interventions for Dementia," in IEEE Pervasive Computing, vol. 18, no. 2, pp. 10-19, 1 April-June 2019, doi: 10.1109/MPRV.2019.2907020.

[7] M. Marques da Rocha, D. Cruz-Sandoval, J. Favela and D. C. Muchaluat-Saade, "EvaSIM: a Software Simulator for the EVA Open-source Robotics Platform," 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Napoli, Italy, 2022, pp. 714-721, doi: 10.1109/RO-MAN53752.2022.9900561.

[8] EVA - An open-source social robotics platform. (2022, December 20). Retrieved from https://eva-social-robot.github.io

[9] google. "mediapipe." GitHub, 23 Jan. 2023, github.com/google/mediapipe.

[10] Technologies, Unity. "Unity - Manual: Building your WebGL application." 20 Jan. 2023, docs.unity3d.com/Manual/webgl-building.html.

[11] midiacom. "eva-robot." GitHub, 29 Jan. 2023, github.com/midiacom/eva-robot.

[12] Villa, L., Hervás, R., Dobrescu, C.C., Cruz-Sandoval, D., Favela, J. (2022). Incorporating Affective Proactive Behavior to a Social Companion Robot for Community Dwelling Older Adults. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2022 – Late Breaking Posters. HCII 2022. Communications in Computer and Information Science, vol 1655. Springer, Cham. https://doi.org/10.1007/978-3-031-19682-9_72