Multimodal Elderly Care Systems (MECS)

• Woll, Anita & Tørresen, Jim (2023). What is a Smart Hospital? A Review of the Literature. In Duffy, V.G.; Lehto, M.; Yih, Y. & Proctor, R.W. (Ed.), Human-Automation Interaction. Springer. ISSN 978-3-031-10780-1. p. 145–165.
• Baselizadeh, Adel; Khaksar, Weria; Uddin, Md Zia; Saplacan, Diana & Tørresen, Jim (2023). Privacy-Preserving User Pose Prediction for Safe and Efficient Human-Robot Interaction, Proceedings of the 2023 19th IEEE International Conference on Automation Science and Engineering (CASE) . IEEE conference proceedings. ISSN 979-8-3503-2070-1. doi: 10.1109/CASE56687.2023.10260298. Full text in Research Archive
• Tørresen, Jim; Saplacan, Diana; Baselizadeh, Adel & Mahler, Tobias (2023). Machine Excellence Tradeoffs to Ethical and Legal Perspectives, Proceedings of the 2023 IEEE Conference on Artificial Intelligence (IEEE CAI). IEEE conference proceedings. ISSN 979-8-3503-3984-0. p. 237–240. doi: 10.1109/CAI54212.2023.00109. Full text in Research Archive Show summary

  Abstract—We appreciate well-functioning technology being able to also personalize its services. However, to protect privacy and avoid a potential misuse of personal data, we are encouraged to limit the amount of personal data we share through apps and Internet services. While some services do not really need all the data they ask us to provide, others depend on it to provide the best possible performance of its service. That regards systems that apply data in machine learning for tasks like medical diagnostics. Especially deep learning algorithms perform better by using a large amount of data and are now able to benefit from the large amount as well with limited training time given access to high-performance computing resources. This paper address and discuss the tradeoffs like the one we have between data sharing minimalization for increased privacy and data maximization for machine learning systems. Perspectives related to ethics, legal, and social issues are considered in the paper. There is no single conclusion on the challenge, but attention to it can increase the awareness that the best balance differs depending on the application addressed.

• van Otterdijk, Marieke; Saplacan, Diana; Baselizadeh, Adel; Laeng, Bruno & Tørresen, Jim (2023). To Shake or Not to Shake: Intuitive Reactions of Senior Adults to a Robot Handshake in a Western Culture, Proceedings of the 32nd IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) 2023. IEEE conference proceedings. ISSN 979-8-3503-3671-9. p. 890–896. doi: 10.1109/RO-MAN57019.2023.10309608. Full text in Research Archive
• Saplacan, Diana; Schulz, Trenton; Tørresen, Jim & Pajalic, Zada (2023). Health Professionals’ Views on the Use of Social Robots with Vulnerable Users: A Scenario-Based Qualitative Study Using Story Dialogue Method , Proceedings of the 32nd IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) 2023. IEEE conference proceedings. ISSN 979-8-3503-3671-9. p. 421–428. doi: 10.1109/RO-MAN57019.2023.10309644. Full text in Research Archive Show summary

  We used the story dialog method (SDM) to gather the viewpoints of health professionals about the use of social ro-bots in the home and healthcare services with vulnerable users. SDM consists of participants bringing stories that are discussed together. The aim of the study was to address universal design and accessibility issues with robots in specific use situations. Three social robots were used for this purpose in four stories: TIAGo, Romibo and robot companions. SDM method was used in two workshops with eight participants (33 invited, 12 re-cruited, and 8 participated in the end). The participants uncov-ered issues regarding ethics, responsibility, use of data, infra-structure, design, and user concerns based on provided stories and own experiences. These issues provide important aspects that should be considered when using robots with vulnerable us-ers and ensuring that a robot is usable by as many people as pos-sible. The main contribution of the paper is introducing the SDM method to Human-Robot Interaction community, as well as find-ings around universal design and accessibility of robots, in order to create more inclusive social robots.

• Saplacan, Diana & Tørresen, Jim (2022). Robots as Welfare Technologies to Reduce Falls Amongst Older Adults: An Explorative Study from Norway, Human Aspects of IT for the Aged Population. Technology in Everyday Living. Lecture Notes in Computer Science. LNCS 13331. Springer Nature Switzerland AG. ISSN 978-3-031-05654-3. p. 88–106. doi: doi.org/10.1007/978-3-031-05654-3_6.
• Soma, Rebekka; Bratteteig, Tone; Saplacan, Diana; Schimmer, Robyn; Camparo, Erik & Verne, Guri Birgitte (2022). Strengthening Human Autonomy. In the Era of Autonomous Technology. Scandinavian Journal of Information Systems (SJIS). ISSN 0905-0167. 34(2).
• Bordvik, David Andreas; Hou, Jie; Noori, Farzan Majeed; Uddin, Md Zia & Tørresen, Jim (2022). Monitoring In-Home Emergency Situation and Preserve Privacy using Multi-modal Sensing and Deep Learning, 2022 International Conference on Electronics, Information, and Communication (ICEIC 2022) . IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665409353. doi: 10.1109/ICEIC54506.2022.9748829. Full text in Research Archive Show summary

  Videos and images are commonly used in home monitoring systems. However, detecting emergencies in-home while preserving privacy is a challenging task concerning Human Activity Recognition (HAR). In recent years, HAR combined with deep learning has drawn much attention from the general public. Besides that, relying entirely on a single sensor modal-ity is not promising. In this paper, depth images and radar presence data were used to investigate if such sensor data can tackle the challenge of a system's ability to detect abnormal and normal situations while preserving privacy. The recurrence plots and wavelet transformations were used to make a two-dimensional representation of the presence radar data. Moreover, we fused data from both sensors using data-level, feature-level, and decision-level fusions. The decision-level fusion showed its superiority over the other two techniques. For the decision-level fusion, a combination of the depth images and presence data recurrence plots trained first on convolutional neural networks (CNN). The output was fed into support vector machines, which yielded the best accuracy of 99.98%.

• Mantelli, Mathias; Pittol, Diego; Maffei, Renan; Tørresen, Jim; Prestes, Edson & Kolberg, Mariana (2021). Semantic Active Visual Search System Based on Text Information for Large and Unknown Environments. Journal of Intelligent and Robotic Systems. ISSN 0921-0296. 101(2). doi: 10.1007/s10846-020-01298-7.
• van Otterdijk, Marieke; Neggers, Margot; Tørresen, Jim & Barakova, Emilia Ivanova (2021). Preferences of Seniors for Robots Delivering a Message With Congruent Approaching Behavior, Proceedings of the 2021 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO) Virtual Conference, July 8-10, 2021. IEEE Robotics and Automation Society. ISSN 978-1-6654-4952-6. doi: 10.1109/ARSO51874.2021.9542833. Show summary

  Congruence of verbal and nonverbal behavior of a robot is essential to establish a seamless and natural interaction between humans and robots. We have investigated the preference and user experience of the senior participants that a robot approaches with expressive movement that is tailored or incongruent to the message the robot brings. Twelve elderly experienced three scenarios with different messages varying from good news, bad news, and functional news, accompanied by the corresponding approaching behavior and body posture. The user experience of these elderly was evaluated during these scenarios using a questionnaire and a semi-structured interview. The participants were interviewed about their experience, preference, and motivation for the different robot motion behaviors. The analysis showed no significant difference in user experiences in the good news and functional news scenarios. Also, no clear preference among the different robot behaviors was found. However, a significant preference for congruence in the sad news scenario was found with a clear preference for the sad approaching behavior.

• Khaksar, Weria; Neggers, Margot; Barakova, Emilia Ivanova & Tørresen, Jim (2021). Generation Differences in Perception of the Elderly Care Robot. In Caleb-Solly, Praminda; Ghafurian, Moojan; Johnson, Michelle & Wada, Kazuyoshi (Ed.), Proceedings of the 30th IEEE International Conference on Robot & Human Interactive Communication (ROMAN). IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-6654-0492-1. p. 551–558. doi: 10.1109/RO-MAN50785.2021.9515534.
• Saplacan, Diana; Khaksar, Weria & Tørresen, Jim (2021). On Ethical Challenges Raised by Care Robots: A Review of the Existing Regulatory-, Theoretical-, and Research Gaps, Proceedings of the 2021 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO) Virtual Conference, July 8-10, 2021. IEEE Robotics and Automation Society. ISSN 978-1-6654-4952-6. p. 219–226. doi: 10.1109/ARSO51874.2021.9542844. Show summary

  This paper presents some of the ethical challenges that current care robots raise in home- and healthcare services for senior adults (≥65 years). The paper is grounded in some of the state-of-the-art projects within the area of care robotics.Further, the paper identifies and discusses several central challenges raised by using robots as part of care services for the elderly people. The paper contributes to the ethical debate on the implications care robots may have for the practical context of healthcare. In addition, the paper summarizes the main lines of the EU legal approach to AI robotic technology, offering a comprehensive picture of the existing regulatory, theoretical and research gaps, compelling the need of an interdisciplinary ethical reflection on care robots. Finally, the discussion is then balanced by some of the opportunities the care robots may provide for the care services.

• Herstad, Jo; Schulz, Trenton Wade & Saplacan, Diana (2021). T-able: An Investigation of Habituating Moving Tables at Home . In Verma, Ira (Eds.), Universal Design 2021: From Special to Mainstream Solutions. IOS Press. ISSN 978-1-64368-190-0. p. 238–251. doi: 10.3233/SHTI210400. Show summary

  This paper investigates the philosophical and designerly questions of how the concepts of familiarity, orientation, and habituation can be used to understand a selfmoving (semi-autonomous robot) table at home. Tables are familiar habituated objects in domestic settings for people with various abilities. We explore the idea of a selfmoving table through the lens of universal design. Phenomenology is applied to get a grip on ways of orienting and being oriented by such habituated familiar objects. Specifically, we investigate how the t-able is used as a telephone table, where the telephone is always charged and in a fixed place on the table. This is an attempt to make the telephone easier to use and relate to at home. The paper aims to inform future robots' design for the independently living elderly by designing robots mainly from natural materials, such as wood. We also discuss similarities and differences between the universal design of the built environment and ICT environments with this paper.

• Noori, Farzan Majeed; Uddin, Md Zia & Tørresen, Jim (2021). Ultra-Wideband Radar-Based Activity Recognition Using Deep Learning. IEEE Access. ISSN 2169-3536. 9, p. 138132–138143. doi: 10.1109/ACCESS.2021.3117667. Full text in Research Archive Show summary

  With recent advances in the field of sensing, it has become possible to build better assistive technologies. This enables the strengthening of eldercare with regard to daily routines and the provision of personalised care to users. For instance, it is possible to detect a person’s behaviour based on wearable or ambient sensors; however, it is difficult for users to wear devices 24/7, as they would have to be recharged regularly because of their energy consumption. Similarly, although cameras have been widely used as ambient sensors, they carry the risk of breaching users’ privacy. This paper presents a novel sensing approach based on deep learning for human activity recognition using a non-wearable ultra-wideband (UWB) radar sensor. UWB sensors protect privacy better than RGB cameras because they do not collect visual data. In this study, UWB sensors were mounted on a mobile robot to monitor and observe subjects from a specific distance (namely, 1.5–2.0 m). Initially, data were collected in a lab environment for five different human activities. Subsequently, the data were used to train a model using the state-of-the-art deep learning approach, namely long short-term memory (LSTM). Conventional training approaches were also tested to validate the superiority of LSTM. As a UWB sensor collects many data points in a single frame, enhanced discriminant analysis was used to reduce the dimensions of the features through application of principal component analysis to the raw dataset, followed by linear discriminant analysis. The enhanced discriminant features were fed into the LSTMs. Finally, the trained model was tested using new inputs. The proposed LSTM-based activity recognition approach performed better than conventional approaches, with an accuracy of 99.6%. We applied 5-fold cross-validation to test our approach. We also validated our approach on publically available dataset. The proposed method can be applied in many prominent fields, including human–robot interaction for various practical applications, such as mobile robots for eldercare.

• Schulz, Trenton Wade; Soma, Rebekka & Holthaus, Patrick (2021). Movement acts in breakdown situations: How a robot's recovery procedure affects participants' opinions. Paladyn - Journal of Behavioral Robotics. ISSN 2080-9778. 12(1), p. 336–355. doi: 10.1515/pjbr-2021-0027. Full text in Research Archive Show summary

  Recovery procedures are targeted at correcting issues encountered by robots. What are people’s opinions of a robot during these recovery procedures? During an experiment that examined how a mobile robot moved, the robot would unexpectedly pause or rotate itself to recover from a navigation problem. The serendipity of the recovery procedure and people’s understanding of it became a case study to examine how future study designs could consider breakdowns better and look at suggestions for better robot behaviors in such situations. We present the original experiment with the recovery procedure. We then examine the responses from the participants in this experiment qualitatively to see how they interpreted the breakdown situation when it occurred. Responses could be grouped into themes of sentience, competence, and the robot’s forms. The themes indicate that the robot’s movement communicated different information to different participants. This leads us to introduce the concept of movement acts to help examine the explicit and implicit parts of communication in movement. Given that we developed the concept looking at an unexpected breakdown, we suggest that researchers should plan for the possibility of breakdowns in experiments and examine and report people’s experience around a robot breakdown to further explore unintended robot communication.

• Tørresen, Jim (2021). Undertaking Research with Humans within Artificial Intelligence and Robotics: Multimodal Elderly Care Systems. Technology Architecture + Design (TAD). ISSN 2475-1448. 5(2), p. 141–145. doi: 10.1080/24751448.2021.1967052.
• Saplacan, Diana; Herstad, Jo & Pajalic, Zada (2020). An Analysis of Independent Living Elderly’s Views on Robots: A Descriptive Study from the Norwegian Context. In Mauri, Lloret, James; Saplacan, Diana; Carcani, Klaudia; Ardiansyah, Prima Oky Dicky & Vasilache, Simona (Ed.), ACHI 2020, The Thirteenth International Conference on Advances in Computer-Human Interactions. International Academy, Research and Industry Association (IARIA). ISSN 978-1-61208-761-0. p. 199–208. Full text in Research Archive Show summary

  This study illustrates the independent living elderly’s (≥65 years) views on robots. The data was documented through audio recordings of interviews, photos, and written logs. The analysis was done through qualitative manifest and latent content analysis. The results of the analysis were sorted into three categories: aging during the technological renaissance, domestic robots, and the elderly’s expectations of robots. The overall resulted theme was: integrating robots in the elderly’s everyday life. The results were discussed through the lenses of the Sense-of-Coherence (SOC) theoretical construct and its belonging elements: comprehensibility, manageability, and meaningfulness. The relevance of this paper contributes to giving an understanding of the domestic robots’ requirements specifications and the elderly’s expectation of human-robot interaction.

• Noori, Farzan; Riegler, Michael; Uddin, Md Zia & Tørresen, Jim (2020). Human Activity Recognition from Multiple Sensors Data Using Multi-Fusion Representations and CNNs. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP). ISSN 1551-6857. 16(2). doi: 10.1145/3377882. Show summary

  With the emerging interest in the ubiquitous sensing field, it has become possible to build assistive technologies for persons during their daily life activities to provide personalized feedback and services. For instance, it is possible to detect an individual’s behavioral pattern (e.g., physical activity, location, and mood) by using sensors embedded in smart-watches and smartphones. The multi-sensor environments also come with some challenges, such as how to fuse and combine different sources of data. In this article, we explore several methods of fusion for multi-representations of data from sensors. Furthermore, multiple representations of sensor data were generated and then fused using data-level, feature-level, and decision-level fusions. The presented methods were evaluated using three publicly available human activity recognition (HAR) datasets. The presented approaches utilize Deep Convolutional Neural Networks (CNNs). A generic architecture for the fusion of different sensors is proposed. The proposed method shows promising performance, with the best results reaching an overall accuracy of 98.4% for the Context-Awareness via Wrist-Worn Motion Sensors (HANDY) dataset and 98.7% for the Wireless Sensor Data Mining (WISDM version 1.1) dataset. Both results outperform previous approaches.

• Saplacan, Diana; Herstad, Jo; Tørresen, Jim & Pajalic, Zada (2020). A Framework on Division of Work Tasks between Humans and Robots in the Home. Multimodal Technologies and Interaction. ISSN 2414-4088. 4(3). doi: 10.3390/mti4030044. Full text in Research Archive Show summary

  This paper analyzes work activity in the home, e.g., cleaning, performed by two actors, a human and a robot. Nowadays, there are attempts to automate this activity through the use of robots. However, the activity of cleaning, in and of itself, is not important; it is used instrumentally to understand if and how robots can be integrated within current and future homes. The theoretical framework of the paper is based on empirical work collected as part of the Multimodal Elderly Care Systems (MECS) project. The study proposes a framework for the division of work tasks between humans and robots. The framework is anchored within existing research and our empirical findings. Swim-lane diagrams are used to visualize the tasks performed (WHAT), by each of the two actors, to ascertain the tasks’ temporality (WHEN), and their distribution and transitioning from one actor to the other (WHERE). The study presents the framework of various dimensions of work tasks, such as the types of work tasks, but also the temporality and spatiality of tasks, illustrating linear, parallel, sequential, and distributed tasks in a shared or non-shared space. The study’s contribution lies in its foundation for analyzing work tasks that robots integrated into or used in the home may generate for humans, along with their multimodal interactions. Finally, the framework can be used to visualize, plan, and design work tasks for the human and for the robot, respectively, and their work division.

• Saplacan, Diana; Herstad, Jo & Schulz, Trenton Wade (2020). Situated Abilities within Universal Design – A Theoretical Exploration. International Journal on Advances in Intelligent Systems. ISSN 1942-2679. 13(34), p. 278–291. Full text in Research Archive Show summary

  This paper investigates Universal Design (UD) through the idea of designing for situated abilities, rather than focusing on designing for disabled users. This shift in perspective from disabilities to abilities is explored through the design of a domestic robot that integrates into our homes, in a familiar way. We explore the concept of designing for situated abilities through a proof-of-concept robotic wooden table, the T-ABLE, as an alternative design for domestic robots. Finally, the paper identifies four dimensions of situated abilities.

• Uddin, Md Zia; Noori, Farzan Majeed & Tørresen, Jim (2020). In-Home Emergency Detection Using an Ambient Ultra-Wideband Radar Sensor and Deep Learning. In Noori, Farzan Majeed (Eds.), IEEE 10th International Conference on Ultrawideband and Ultrashort Impulse Signals (UWBUSIS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-7281-7314-6. p. 1089–1093. doi: 10.1109/UkrMW49653.2020.9252708. Show summary

  Human behavior analysis in indoor environments from ambient sensor data is one of the most challenging human-machine interaction research topics. In this paper, we present a deep learning-based novel approach for human posture recognition and emergency detection using an ultra-wideband radar sensor. The strength of this sensor is that it collects less privacy-related information than a regular camera. The sensor is installed on a mobile robot to observe a subject from a short distance. At first, raw data is collected from the sensor. Then, the data is used to train a Recurrent Neural Network (RNN) for modeling of different human activities and conditions, including normal and emergency conditions. Finally, the trained model is used for testing the model on new inputs. The experiments were performed using two datasets recorded in lab environments, and the proposed approach produced better recognition performance than the conventional ones. The proposed method can be applied in many prominent research fields (e.g., human-robot interaction) for different practical applications such as mobile robots for eldercare.

• Woll, Anita & Tørresen, Jim (2020). Bridging the User Barriers of Home Telecare. Advances in Intelligent Systems and Computing. ISSN 2194-5357. 1069, p. 816–835. doi: 10.1007/978-3-030-32520-6_59.
• Bråthen, Heidi; Maartmann-Moe, Harald & Schulz, Trenton Wade (2019). The Role of Physical Prototyping in Participatory Design with Older Adults. In Kokil, Uttam & Ota, Tomoko (Ed.), ACHI 2019, The Twelfth International Conference on Advances in Computer-Human Interactions. International Academy, Research and Industry Association (IARIA). ISSN 978-1-61208-686-6. p. 141–146. Full text in Research Archive Show summary

  How can older adults actively participate in the design processes of assistive robots designed for their homes? We have organized workshops with a group of older adults who worked actively with materials and physical prototyping to design a fetch robot for the home. We present the basics of the workshop materials, how the workshop was performed, and findings on the role of physical prototyping from the workshops.

View all works in Cristin

• Saplacan, Diana (2023). Webpage for The Ethical, Legal and User Perspectives on Social and Assistive Robots (ELUPSAR) Workshop, at at 11th European Conference on Mobile Robots .
• Saplacan, Diana; Pajalic, Zada & Tørresen, Jim (2023). Should Social and Assistive Robots Integrated within Home- and Healthcare Services Be Universally Designed? Cambridge Handbook on Law, Policy, and Regulations for Human-Robot Interaction. Cambridge University Press. ISSN 000-0-000-00000-0. doi: ISBN%209781009386661.
• Saplacan, Diana (2023). Comment on "Vulnerability in Social and Health Robotics" - presentation by Rachele Carli (Univ. of Bologna and Univ. of Luxembourg), as part of the Vulnerability in Robot Society (VIROS) Annual Symposium, Session I on "Vulnerability and Cognitive Sovereignty".
• Saplacan, Diana (2023). Presentation of the paper "Health Professionals’ Views on the Use of Social Robots with Vulnerable Users: A Scenario-Based Qualitative Study Using Story Dialogue Method".
• Bergflødt, Anderson, Adrian; Søyseth, Vegard & Saplacan, Diana (2022). OPEN ZONE FOR EXPERIMENTAL INFORMATICS (ÅPEN SONE FOR EKSPERIMENTELL INFORMATIKK) as part of "Signals and noise in artificial intelligence" - SENT Festival at the Technical Museum, Oslo, Norway.
• Saplacan, Diana (2022). Presentation at The 2nd Workshop on Design-Centered HRI and Governance. Title: Making Care Robots Understandable: An Introduction to Universal Design Principles as Design and Ethical Guidelines of Social Assistive Robots.
• Saplacan, Diana (2022). Robots as Welfare Technologies to Reduce Falls Amongst Older Adults: An Explorative Study from Norway.
• Saplacan, Diana (2022). Tutorial presentation. Title: "Robots and Society: Challenges and Opportunities within social Human-Robot Interaction" at 31st IEEE International Conference on Robot & Human Interactive Communication.
• Cheung, Anne; Saplacan, Diana; Pietropaoli, Stefano & Weng, Yueh-Hsuan (2022). SORO – INTERDISCIPLINARY WORKSHOP ON THE GOVERNANCE FOR SOCIAL ROBOTS (HALF DAY).
• Saplacan, Diana (2022). Ethics and Social Robots: A Universal Design Perspective - Presentation during Interdisciplinary Workshop on the Governance for Social Robots (SORO) at JURIX 2022 (The 35th International Conference on Legal Knowledge and Information Systems).
• Saplacan, Diana (2022). Ongoing research with Social and Assistive Robots - Presentation for the National Commitee in Research Ethics in Science and Technology (NENT).
• Saplacan, Diana (2022). Guest Lecture: Ongoing research with Social and Assistive Robots: Research projects, empirical examples, and theory.
• Saplacan, Diana (2022). TIAGo wishes you welcome to a debate on the theme: Should a robot be involved in care tasks? (Norwegian title: TIAGo ønsker velkommen til en debatt med tema: Bør en robot involveres i omsorgsoppgaver?).
• Saplacan, Diana; Baselizadeh, Adel & Tørresen, Jim (2022). Robot Demonstration: Meet TIAGo the robot! The robot showcases several tasks: brushing hair, putting the lipstick on, using a (plastic) knife, moving an object, carrying a bag.
• Saplacan, Diana (2022). What do we talk about when we talk about social robots vs. robot sociomorphism ? Empirical examples from previous and ongoing research.
• Saplacan, Diana (2022). Ethical challenges and opportunities with (care) robots - Universal Design as an ethical charter for inclusive robot design and interaction.
• Brinck, Ingar; Paiva, Ana; Saplacan, Diana & Bensch, Suna (2022). Debate during the symposium: "The Future of Human-Robot Interaction Science" at the Trandisciplinary AI for Good of All (TAIGA) Inauguration Conference, leaded by Prof. Thomas Hellström and Ass. Prof. Niclas Kaiser leading TAIGAS focus area on Embodied interactive AI, at Umeå University, Sweden .
• Tørresen, Jim; Atsushi, Nazawa; Saplacan, Diana; Prestes, Edson; Mahler, Tobias & Weng, Yueh-Hsuan (2022). Tutorial at IROS 2022 - Ethical, Legal and User Perspectives on Robots and Systems (ELAUPORAS) – Assessments and Potential Measures.
• Saplacan, Diana (2022). Ethics: Autonomous Technologies for All (ATA)?
• Saplacan, Diana (2022). Ethics: Research Ethics and Autonomous Technologies for User Diversity.
• Tørresen, Jim (2022). Sensing, acting and adapting in the real world.
• Tørresen, Jim (2022). Ethical Perspectives of Robotics and AI – How to develop preferable system? .
• Tørresen, Jim (2022). Research on health-related treatment and care technology – A technical and ethical view.
• Tørresen, Jim (2022). Tutorial: Ethical challenges for Autonomous and Multiagent Systems.
• Saplacan, Diana; Tørresen, Jim; Weng, Yueh-Hsuan & Li, Phoebe (2022). Tutorial: Robots and Society (RO-SO 2022) / Ethical perspectives and technical challenges and opportunities with care robots.
• Tørresen, Jim & Nakasawa, Atsushi (2022). Tutorial: Ethical Considerations in User Modeling and Personalization (ECUMAP).
• Tørresen, Jim (2022). Research Ethics in AI and Robotics.
• Tørresen, Jim (2022). Keynote: Multi-Modal Sensing for Care Robots for Older People.
• Tørresen, Jim (2022). Tutorial: Ethical Challenges in Computational Intelligence Research.
• Saplacan, Diana (2022). Ethical Issues in Mobile Robots - Guest Lecture for Norwegian University of Life Sciences (NMBU), at the Faculty of Science and Technology, part of Human-Robot Interaction Course, Advanced Topics in Mobile robots.
• Saplacan, Diana (2022). Participatory Design and Human-Robot Interaction - An ethical and inclusive perspective on contemporary technologies.
• Dahl, Heidi Elisabeth Iuell; Mehmandarov, Rustam Karim; Harkestad, Inge & Saplacan, Diana (2022). AI and Ethics workshop.
• Saplacan, Diana (2022). Healthcare Professionals’ Attitudes towards the Organization of Care Services and the Adoption of Welfare Robots in Norway.
• Saplacan, Diana (2022). The importance of standarization within new technological areas.
• Saplacan, Diana (2021). AI, Robots and Their Social Impact: An Introduction to AI, ML, and Robots.
• Saplacan, Diana (2021). Challenges and Opportunities with AI-based Care Robots: An Ethical and Inclusive Approach.
• Tørresen, Jim (2021). Tutorial: Explainability, Trust and Ethics for Robots and Autonomous Systems.
• Tørresen, Jim (2021). INTROMAT: INtroducing personalized TReatment Of Mental health problems using Adaptive Technology.
• Tørresen, Jim (2021). Tutorial: Ethical Considerations in Robotics and Automation.
• Tørresen, Jim (2021). Tutorial: Ethical Considerations in User Modeling and Personalization.
• Tørresen, Jim; Saplacan, Diana; Lintvedt, Mona Naomi; Mahler, Tobias & Fosch-Villaronga, Eduard (2021). Tutorial: Ethical and Legal Assessments Related to Robots and Systems .
• Tørresen, Jim (2021). Kunstig intelligens – allsidig i metoder og anvendelser.
• Tørresen, Jim (2021). Tutorial: Intelligent System Research – AI Ethical Challenges and Opportunities.
• Saplacan, Diana; Khaksar, Weria & Tørresen, Jim (2021). On Ethical Challenges Raised by Care Robots: A Review of the Existing Regulatory-, Theoretical-, and Research Gaps.
• Saplacan, Diana (2021). Some experiences about analysis and writing it up - On qualitative data analysis methods .
• Noori, Farzan Majeed (2021). Recognition of Human Activities using UWB Radar and Deep Learning. Show summary

  The population of older adults are increasing every day, and the trend is expected to continue. With the help of advanced assistive technologies, we can provide them better healthcare. In this work, we provide a novel sensing approach based on Ultra-wideband (UWB) sensors to recognize human activities and forecast future events. Previously, researchers focused on wearable or video sensors to detect a person’s behaviour. However, it is challenging for older people to wear devices 24/7. Similarly, vision-based sensors always carry privacy concerns. In contrast, a non-contact ambient sensor with no such privacy issues is XeThru ultra-wideband (UWB) radar. We collected the data using multiple modalities, such as UWB, depth images, thermal images, and actigraphy device, i.e. to collect heart rate (HR) for ground truth. The dataset comprised of the participants sitting on the sofa in normal situation. Afterwards, we recommended participants do some exercise for the sake of increasing HR. When the HR increased by more than 140 BPM, the users lie down on the floor in front of sensors until their HR reached a normal level. In this research, we classify normal vs abnormal situations using CNNs and LSTMs. Accuracy, precision, and recall are used as performance measures. We got 95% and 98% accuracies using CNNs and LSTMs, respectively. Furthermore, seven classes were introduced based on HR levels, as shown in Table1. We got promising results with LSTMs. A confusion matrix is shown in Figure 1. It was the first step towards classifying activities using UWB radar. In the future, we are planning to predict the future HR based only on UWB data, as it would be challenging for older people to wear actigraphy devices. Moreover, we will include other modalities which we did not include in our preliminary analysis.

• Saplacan, Diana (2021). Ethical Challenges and Opportunities with Care Robots Universal Design as An Ethical Charter for Inclusive Robot Design and Interaction.
• Saplacan, Diana (2021). Ethics and technology: Research With --, About --, and for Users.
• Saplacan, Diana (2021). Ethical challenges and opportunities with care robots (Talk at RO-MAN 2021, in PANEL I on Introduction on Ethics and Care Robots).
• Saplacan, Diana; Tørresen, Jim; Mahler, Tobias & Fosch-Villaronga, Eduard (2021). Robots and Society: Ethical, Legal, and Technical Perspectives on Integrating Robots in the Home- and Healthcare Systems and Services (RO-SO). Show summary

  The RO-SO tutorial is a continuation of the previous tutorials regarding robots, ethics, legal, and technical aspects. The tutorial is part of the 30th IEEE International Conference on Robot and Human Interactive Communication (ROMAN 2021). It focuses on providing an overview and discussing the ethical, legal, and technical perspectives, including challenges, dilemmas, and advantages and opportunities of integrating robots as part of home- and healthcare systems and services. The following two questions will guide the overall objective for this tutorial, but these will not limit the talks: How does the meaning of good care change when we integrate robots as part of the home- and healthcare systems and services? What does good care mean when we integrate assistive robots as part of the home- and healthcare services? What are some of the ethical, legal, and technical challenges and opportunities when integrating assistive care robots in the home- and healthcare services?

• Saplacan, Diana (2021). "How can I trust that you care about my care, RO-BOT?" - A talk about ethics in the context of care robots.
• Saplacan, Diana (2021). Care Robots and Universal Design - An Ethical Charter for Designing Care Robots and Human-Robot Interaction.
• Saplacan, Diana (2021). Ethics and Technology.
• Tørresen, Jim (2021). How to achieve ethical artificial intelligence (AI) research and development?
• van Otterdijk, Marieke (2021). De Kennis van Nu” (The knowledge of now). [TV]. Dutch national TV (NPO).
• Tørresen, Jim (2021). Introducing a Multimodal Elderly Care Safety Alarm Robot.
• Tørresen, Jim (2021). Tutorial: Ethical Considerations in the Development and Use of Neural Information Processing Systems.
• Tørresen, Jim (2020). Ethical Considerations in User Modeling and Personalization: ACM UMAP 2020 Tutorial. In Kuflik, Tsvi; Torre, Ilaria & Burke, Robin (Ed.), Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery (ACM). ISSN 978-1-4503-6861-2. p. 390–391.
• Tørresen, Jim (2020). Addressing ethical challenges within evolutionary computation applications: GECCO 2020 tutorial. In Coello Coello, Carlos A. (Eds.), GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 978-1-4503-7127-8. p. 1206–1223. doi: https:/doi-org.ezproxy.uio.no/10.1145/3377929.3389894.
• Tørresen, Jim (2020). Tutorial: Ethical Challenges and Opportunities within Computational Intelligence System Development.
• Tørresen, Jim (2020). Ethical Consideration in Robotics and Intelligent Systems Research.
• Tørresen, Jim (2020). Addressing Ethical Challenges for Care Giving Robots.
• Tørresen, Jim (2020). Tutorial: Ethical Considerations in User Modeling and Personalization.
• Tørresen, Jim; Kurazume, Ryo & Prestes, Edson (2020). Guest Editorial: Special Issue on Elderly Care Robotics – Technology and Ethics . Journal of Intelligent and Robotic Systems. ISSN 0921-0296. 98, p. 3–4. doi: 10.1007/s10846-020-01148-6.
• Tørresen, Jim (2020). Tutorial: Addressing Ethical Challenges within Evolutionary Computation Applications.
• Tørresen, Jim (2020). AI Ethics: How to achieve ethically good artificial intelligence research and development.
• Tørresen, Jim (2020). Intelligent Robots and Systems for Foundational and Applied Healthcare Research.
• Tørresen, Jim (2020). Sensing Human State with Application in Older People Care and Mental Health Treatment.
• Tørresen, Jim (2020). Interdisciplinary Research Collaboration in Rhythm, Time and Motion.
• Noori, Farzan Majeed; Uddin, Md Zia & Tørresen, Jim (2020). Presentation on In-Home Emergency Detection Using an Ambient Ultra-Wideband Radar Sensor and Deep Learning.
• Schulz, Trenton Wade; Holthaus, Patrick; Amirabdollahian, Farshid & Koay, Kheng Lee (2019). Humans’ Perception of a Robot Moving Using a Slow in and Slow Out Velocity Profile. In Kim, Jung H.; Tapus, Adriana; Sirkin, David; Jung, Malte & Kwak, Sonya S. (Ed.), 14th Annual ACM/IEEE International Conference on Human Robot Interaction – HRI’19. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8555-6. p. 594–595. Show summary

  Humans need to understand and trust the robots they are working with. We hypothesize that how a robot moves can impact people’s perception and their trust. We present a methodology for a study to explore people’s perception of a robot using the animation principle of slow in, slow out—to change the robot’s velocity profile versus a robot moving using a linear velocity profile. Study participants will interact with the robot within a home context to complete a task while the robot moves around the house. The participants’ perceptions of the robot will be recorded using the Godspeed Questionnaire. A pilot study shows that pilot participants notice the difference between the linear and the slow in, slow out velocity profiles, so the full experiment planned with participants will allow us to compare their perceptions based on the two observable behaviors.

• Noori, Farzan Majeed; Tørresen, Jim; Uddin, Md Zia & Riegler, Michael A. (2023). Multimodal Deep Learning Approaches for Human Activity recognition. Universitetet i Oslo. ISSN 1501-7710. Full text in Research Archive Show summary

  Smart homes may be beneficial for people of all ages, but this is especially true for those with care needs, such as the elderly. To assist, monitor for emergencies, and provide companionship for the elderly, a substantial amount of research on human activity recognition systems has been conducted. Several algorithms for activity recognition and prediction of future events have been reported in the scientific literature. However, the majority of published research does not address privacy concerns or employ a variety of ambient sensors. The objective of this thesis is to contribute to the progress in research relevant to activity recognition systems that use sensors that collect less privacy-related information. The following tasks are included in the work: assessment of sensors while keeping privacy concerns in mind, selection of cutting-edge classification methods, and how to fuse the data from multiple sensors. This thesis contributes to making progress on systems for analyzing human activity and state—or vital signs—for application in a mobile robot. This dissertation examines two topics. First, it examines the privacy concerns associated with having a robot in the home. On a robot, an ultra-wideband (UWB) radar-based sensor and an RGB camera (for ground truth) were installed. An actigraphy device was also worn by the users for heart rate monitoring. The UWB sensor was selected to maintain privacy while monitoring human activities. Considering different ways to represent data from a single sensor is the second topic under investigation. That is, how data from multiple representations can be combined. For this purpose, we investigate various data representations from a single sensor’s data and analysis using cutting-edge deep learning algorithms. The contributions provide considerations for equipping a mobile home robot with activity recognition abilities while reducing the amount of privacy-sensitive sensor data. The work also concerns examining the potential privacy restrictions that must be established for the analyzing systems. The thesis contains new methods for combining data from multiple information sources. To achieve our objective, convolutional neural networks and recurrent neural networks were applied and validated using conventional methods. The conclusion of the thesis is that we can achieve good accuracy with limited sensors while maintaining privacy. It is, however, likely adequate for assisting healthcare personnel and caregivers in their work by indicating current activity status and measuring activity levels, providing alerts about abnormal activities. The results can hopefully contribute to older people being able to live alone in their homes with a larger chance of any unwanted events being quickly detected and notified to the caregivers and providers.

• Soma, Rebekka; Saplacan, Diana & Sikora, Magdalena Claudia (2022). Exploring personality and behavior with a robot table - a qualitative case study. Universitetet i Oslo.
• Orten, Mathias Kramer & Tørresen, Jim (2021). Machine Ethics for Autonomous systems – Learning verifiable rules from the environment. Universitetet i Oslo.
• Wang, Lina; Tørresen, Jim & Noori, Farzan Majeed (2021). Emergency Monitoring of Older People Using Multiple Sensors with Privacy in Consideration. Universitetet i Oslo.
• Saplacan, Diana (2020). Situated Abilities: Understanding Everyday Use of ICTs. Reprosentralen, University of Oslo. ISSN 1501-7710.
• Schulz, Trenton Wade (2020). Exploration of Moving Things in the Home. Universitetet i Oslo. ISSN 1501-7710. Show summary

  One question we ask ourselves as we age is, “who will help us when we get older and need extra help?” A natural answer to this question has, in the past, been younger members of the family or nurses and other members of the healthcare system. Future demographics of at least Europe and North America, however, highlight a trend that there will be a larger proportion of older, retired people than younger who can take care of them. This could mean that when an older person is in need of help, there is nobody to provide it. One possible solution is to use information and communication technology to help older people maintain their independence and live at home longer. There are many ways this can be achieved. This Ph.D. dissertation focuses on having a mobile robot in the home that can monitor the vital signs of a person and potentially contact experts in event of a problem. A robot in the home opens many areas of research. This dissertation, however, focuses on two areas. The first area we examine is the privacy issues of a robot in the home. Many of the technology solutions require collecting and processing data about the home residents. How can we examine and discuss the privacy issues related to a robot in the home? What trade-offs must we take into consideration when a robot is in the home environment? The other area we examine is robot movement in the home, how a robot should move, and how it affects people’s interaction with a robot. Can other disciplines, such as film animation, help make a robot move in ways that will lead to a better interaction? Investigations into these aspects resulted in the four papers that are presented in this dissertation. It also resulted in the following additional contributions of: (1) a framework, with sample dilemmas, for examining privacy issues in a home environment with a robot. (2) a review of the use of animation techniques in human-robot interaction user studies, (3) an examination of one of the principles of animation and how it can be applied to a robot, (4) a way of examining and categorizing movement between a human and a robot in the home, and (5) an evaluation of how applying this principle to a robot’s movement affects people’s perception of the robot. The contributions provide items that should be considered when one is creating a robot for the home. Examining the potential privacy boundaries that must be negotiated when a robot is in the home can lead to privacy-preserving robots. In addition, using animation techniques to move a robot may help in people feeling safer around a robot, and this can make robots easier to interact with in the home or anywhere we encounter them. These contributions can lead to safe and trustworthy humanrobot interaction with older people in the home.

View all works in Cristin