BioResonant interfaces : tangible, subliminal biofeedback to regulate physiological states

BioResonant Interfaces:

Tangible, Subliminal Biofeedback to Regulate Physiological States

by

Kyung Yun Choi

B.Sc. Mechanical Engineering, Yonsei University, Seoul, South Korea (2013)

M.S. Aerospace Engineering, University of Illinois at Urbana-Champaign (2017)

Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning,

in partial fulfillment of the requirements for the degree of

Master of Science in Media Arts and Sciences

at the

Massachusetts Institute of Technology

June 2019

MASSAHSES INSTITUTE

OFEhCHNOLOGY

JUL. 2 62019

LIBRARIES

ARCHIVES

© Massachusetts Institute of Technology, 2019. All rights reserved

Signature redacted

Author ...

Kyung Yun Choi

Program in Media Arts and Sciences

May 10. 2019

Signature redacted

Certified by...

Hiroshi

Ishii

Jerome B. Wiesner Professor of Media

A

ts and Sciences

SqI nature reactedar

_gI

Accepted by ...

L/,

Tod Machover

BioResonant Interfaces:

Tangible, Subliminal Biofeedback to Regulate Physiological States

by

Kyung Yun Choi

Submitted to the Program of Media Arts and Sciences,

School of Architecture and Planning

on May 10, 2019, in partial fulfillment of the

requirements for the degree of

Master of Science in Media Arts and Sciences

Abstract

This thesis introduces BioResonant interfaces, which bidirectionally communicate with the

user's body to provide information on their physiological state. This facilitates regulation of

their stress levels and ultimately

a

more mindful daily life. Based on the principle of

respira-tory sinus arrhythmia (RSA) and physiological synchronization, the BioResonant interface

is designed to stimulate at the subconscious level using subliminal tactile biofeedback.

Through this process, the user's physiological state can be aligned with their desired state.

I

present three different forms of the BioResonant Interface,

a

wearable device,

a

cushion,

and

a

kinetic display, which utilize either heart rate (HR) or breathing rate (BR), or both.

I

present their fabrication, design, and evaluation method in the following interaction scale

order: local interaction with the skin; general interaction with the body; and embodied

in-teraction with the environment. First, I

introduce

a

mobile heart rate rhythmic

regulator-ambienBeat-which provides subtle tactile stimulus. This wearable device in the form of

a

watch monitors heart rate and interacts with the skin in the wrist area. Second,

I

explore

using both types of real-time biometric data-HR and BR-to regulate physiological state.

somaPneu, an actuating cushion the size of an adult's

upper body,

provides two different

forms of tactile stimulus simultaneously by changing its volume and generating

a

pulse. In

contrast to ambienBeat, somaPneu interacts with the broader somatosensory system of

the user's body. Lastly, I

present

a

kinetic display-reSpire-which regulates users'

breath-ing patterns through embodied tangible interactions with their inner state and with people

around them. It encourages them to perform dynamic body motions to interact with the

shape-changing fabrics that represent their own BR. This thesis closes by providing my

vi-sion for the BioResonant interface, which encourages

a

better understanding of ourselves

for richer intrapersonal and interpersonal interactions.

Thesis Supervisor: Hiroshi

Ishii

BioResonant Interfaces:

Tangible, Subliminal Biofeedback to Regulate Physiological States

by

Kyung Yun

Choi

This thesis has been reviewed and approved by the following committee members

Hiroshi Ishii

Signatureredacted

Jerome B. Wiesner Professor of Media Arts and Sciences Tangible Media Group, MIT Media Lab

Rosalind W. Picard

Signature redacted

Professor of Media Arts and Sciences

Affective

Computing

Group, MIT Media Lab

Canan

Dagdeviren

Signature redacted

...

Ce

Assistant Professor of Media Arts and Sciences

Conformable Decoders Group, MIT Media Lab

"A purely disembodied human emotion is a nonentity,"

Contents

Acknowledgements 14

1 Introduction 16

2 Background 20

2.1 Interoception . . . . 20

2.2 Respiratory Sinus Arrhythmia (RSA) and Mindfulness . . . . 21

2.3 Physiological Synchronization via Touch . . . . 22

2.4 Am bient M edia . . . . 24

3 Related Work 25 4 ambienBeat: HRV Regulation via Skin 28 4.1 Design Requirements . . . . 29

4.2 Hardware System . . . . 30

4.3 Noise-less Soft Actuator Mechanism . . . . 32

4.4 HR Regulation System . . . . 33

4.5 Evaluation . . . . 34

4.5.1 M ethod . . . . 34

4.5.2 Experimental Conditions . . . . 35

4.6 Data A nalysis . . . . 36

4.7 Results and Discussion . . . . 37

4.7.1 H i: Boosting Up . . . . 39 4.7.2 H2: Calming Down . . . . 40 4.7.3 H3: Disturbance Level . . . . 40 4.7.4 Sum m ary . . . . 42 4.7.5 Rem arks . . . . 44 5

4.8 Use-cases . . . .. . . . .... . . . . . . 4.8.1 Self-awareness . . . . 4.8.2 Boost Up HRV . . . . 4.8.3 Sleep Assist and Monitoring . . . . 4.8.4 Multimodal Feedback by Voice-coil . . . 4.8.5 Immersive Experiences . . . . 4.8.6 Ambient Presence . . . . 4.9 Future W ork . . . . 4.9.1 Subliminal Tactile Feedback on Different 4.9.2 Effect of Low Frequency Magnetic Fields 4.9.3 How subtle is subtle enough? . . . .

5 somaPneu: HR and BR Regulation 5.1 Design Requirements . 5.2 Hardware System . . . . 5.3 Actuation Mechanism . . . . 5.3.1 Pulse Generation 5.3.2 Volume-change . 5.4 Use-cases... .... 5.4.1 Personal Device . . . 5.4.2 Interpersonal Device 5.5 Future Work . . . . Body Parts via Body

6 reSpire: BR Regulation via Motion

6.1 Kinetic Shape-changing Display . . . .

6.2 Inspiration . . . .

6.3 Interactions . . . .

6.3.1 Personal Connectedness: Mindfulness and Inside Out

6.4 6.5

6.3.2 Interpersonal Connectedness: Physiological and Behavioral Sync. HardwareandSystemDesign . . . . Com putationalInsight . . . . 7 Conclusion . . . . 45 45 46 46 47 47 48 48 48 49 50 51 52 52 53 53 54 55 55 56 57 58 58 59 61 62 63 66 68 70

8 Vision of BioResonant Interfaces: eavesdropping on the body-brain conversation 72

References 75

A Appendix 87

A.1 HR (bpm) change overtime after/during different tasks . . . . 87

A .2 V ideos . . . . 91

A.2.1 Video-am bienBeat: . . . . 91

A.2.2 Video-som aPneu: . . . . 91

A .2.3 Video-reSpire: . . . . 91

A .3 Bills of M aterials . . . . 91

A.3.1 Bills of Material-ambienBeat: . . . . 91

A.3.2 Bills of Material-somaPneu: . . . . 91

A.3.3 Bills of M aterial-reSpire: . . . . 91

List of Figures

1-1 Overview of the BioResonant Interface concept and following projects . . . 17 2-1 RSA. ECG is shortened during inhalation and prolonged during exhalation. . 21 4-1 a smaller version of ambienBeats in different shapes (a) Dome-shapd

ambi-enBeat on wrist (b)Bean-shaped amebiambi-enBeat (c) Unfastened ambiambi-enBeats (d) Fastened ambienBeats . . . . 28 4-2 Regular size of ambienBeat (a) Bottom view of ambienBeat. It has a soft

pressure sensor pad on the surface which contacts with the dorsal part of forearm. (b) Disconnected state of ambienBeat. (c, d) ambienBeat when its two straps are connected. The HRV sensor is integrated on the inner surface

of the black strap. . . . . 29

4-3 System diagram of ambienBeat and its components. It has three major parts; strap-a, main actuation part, strap-b . . . . . 30 4-4 Electronic components of ambienBeat. (a) Inside of the soft actuator.

(a-1) Silicone membrane with a neodymium magnet. (a-2) TIP120 Mosfet for controlling the current through the voice coil (a-3). (a-4) RGB LED for turn ON/OFF, Run indication. (b) Li-po batteries (c) ON/OFF Switch. (d) Microcon-troller (e) Female button electrode for pulse sensor data transmission from the strap-b. (f) Two conductive thread strips that hold conductive Velcro for the power connection of the pulse sensor from the strap-b. . . . . 31 4-5 Displacement of soft membrane of the voice-coil actuator (side view) based

on the activation state of the coil . . . . 32 4-6 Simplified illustration to show the ultimate goal of ambienBeat's HR

regula-tion m echanism . . . . 33 4-7 Mechanism of HRV regulation by subliminal tactile feedback . . . . 33

4-8 Biofeedback tactile control diagram of ambienBeat, and its desired states. . 34 4-9 Experimental procedure. All of the tasks and stimulus proceed in random

order for each subject. . . . . 35 4-10 Preference ranking of tactile, auditory, and visual stimulation. Tactile

stimu-lation was ranked first, followed by visual, and auditory. . . . . 37 4-11 SDNN under different conditions for testing boosting up and calming down

effects. (a)Sitting still task NS: No stimulation, TS: getting tactile stimulation, AS: getting auditory stimulation, VS: getting visual stimulation. (b)Sitting still afterjumping jack task NJ: no stimulation, TJ: getting tactile stimulation, AJ: getting auditory stimulation, VJ: getting visual stimulation) . . . . 37 4-12 One example of the instantaneous HR (bpm) from R-R interval over time after/during

dif-ferent tasks. It represents well the overall of all subject's HR variance over time. All of the graphs from the twelve participants'HR data can be found in A.1A.1 . . . . 38

4-13 Average HR in bpm during three different conditions (a) Sitting still task dur-ing gettdur-ing 120 bpm stimulation NS: no stimulation, TS: gettdur-ing tactile stim-ulation, AS: getting auditory stimstim-ulation, VS: getting visual stimulation. (b) Sitting still after jumping jack task during getting 60 bpm stimulation NS: no stimulation, NJ: base line, TJ: getting tactile stimulation, AJ: getting au-ditory stimulation, VJ: getting visual stimulation.) (c) Drawing task during getting 60 bpm stimulation (ND: base line during the task, TD: getting tactile stimulation, AD: getting auditory stimulation, VD: getting visual stimulation. 38 4-14 Slope of the linear regression line of the instantaneous heart rate from R-R

intervals (bpm) over time under different conditions for testing boosting up (a) and calming down (b) effect. (a) Sitting still task NS: no stimulation, TS: getting tactile stimulation, AS: getting auditory stimulation, VS: getting visual stimulation. (b) Sitting still after jumping jack task NJ: no stimulation, TJ: getting tactile stimulation, AJ: getting auditory stimulation, VJ: getting visual stim ulation . . . . 39 4-15 Post-survey result about participants' feeling of disturbance level for each

modality during the drawing task . . . . 40

4-16 (a) SDNN result during the drawing task ND: no stimulation, TD: getting tac-tile stimulation, AD: getting auditory stimulation, VD: getting visual stimula-tion., (b) post-survey result about subjects' feeling of stress level for each m o d a lity . . . . 4 1 4-17 (a, b) ambienBeat can be worn in various circumstances with minimal effect

on task engagement (c) It can be also worn during relaxing activities (e.g. reading a book, writing a journal) to keep a stable pace of heart rate. (d) M editation assistant . . . . 45 4-18 (a)When a user gets stressed during a difficult task, the R-R interval can be

decreased. In this case, ambienBeat can regulate the user's HR to a more calming state. (b) Regulating HRwhiledriving a car. When a drivergets drowsy, which can be indicated by an increased R-R interval, ambienBeat can boost the bpm by providing short R-R interval feedback. . . . . 46 4-19 When a user watches a movie, for example, by providing a certain character's

physiological state variation through ambienBeat, the user's physiological state can be more attuned to that character's state. . . . . 47 4-20 (a) ambienBeat as a brooch form. (b) Pin it on cloth around neck area. (c)

Pin it on the back, aligned with the neck. . . . . 49 5-1 somaPneu, an actuated cushion that helps people calm down and reduce

their stress-level using two different tactile stimuli. . . . . 51 5-2 System diagram of somaPneu . . . . 53 5-3 Flow chart of the tactile feedback system of somaPneu . . . . 54 5-4 Heart beat pulse generation mechanism. (a)Side view, (b)Top view, (c)Assembled

with breathing motion mechanism . . . . 55 5-5 Heart beat pulse generation patterns based on different voltage supply . . . 55 5-6 Volume-change mechanism. (a)Side view, (b)Top view, (c)Bottom view of the

breathing motion mechanism . . . . 56 5-7 (a)Personal device for self-awareness and assisting the individual to control

stress level (b) Interpersonal device as a distributed synchronized physical d evice . . . . . 56

6-1 reSpire is a shape-changing fabric display that allows audiences to interact with their physiological states in tangible way. . . . . 58 6-2 Breathing projected to water surface. The audience can interact with it

with-out casting shadows on the display. . . . . 59 6-3 reSpire brings tangibility to their invisible physiological state through

shape-changing fabrics (Left: Chiffon, Right: Lycra) . . . . 60 6-4 (a) Personal interaction for mindfulness (b) Interpersonal interactions for

positive resonance. . . . . 62 6-5 Deformation of fabric in different locations and its interaction with a viewer's

hands ... 63

6-6 Different deformations of fabric controlled by two iris nozzle modules. For two-people interactions, it amplifies the fabric deformation when the positive

resonance happens ... 64

6-7 a) Supporting the self-awareness as an ambient biofeedback display (b) Am-bient display (furniture, interior decor) for physiological synchronization of people in an intimate relationship (c) Assistive device for a mental therapy session (d), (e), (f) Interactive art installation . . . . 65 6-8 Interactions with reSpire. It connects the participants respiration pattern to

the deform ation of fabric . . . . 66 6-9 Iris nozzle and 6DOF platform module. The platform enables the nozzle change

its outlet direction and distance between the fabric on top. . . . . 67 6-10 Breathing projected to galaxy. . . . . 67 6-11 Computational fluid dynamics and finite element analysis of fabric

mem-brane with circular boundary constraints in reaction to the different orien-tation of airflow. unit[m /s] . . . . 68 6-12 Air velocity vectors resulted from the computational fluid dynamics analysis.

It shows a cross-sectional view of airflow vortex created below the fabric. The nozzle was placed on the middle in the simulation. unit[m/s] . . . . 69

8-1 Three different probable cases of physiological state regulation. The gray dotted line represents conventional regulation, and the orange dotted line represents regulation by a BioResonant Interface. (a) When people get stressed while completing a complex task. (b) When people are in a neutral state re-garding stress-level, but have low engagement with the given task. (c) When people get so stressed that they feel like they do not want to do anything and g ive u p . . . . 72

List of Tables

3.1 Related w orks . . . . 27

4.1 User study result summary. The bold-text with colors highlights the most significant result by the color-coded stimulation. (Orange: tactile, yellow: au-ditory, green: visual feedback). Black bold values under the combination of the no stimulation stimulation and H1 task, show the base line of the all

eval-uation param eters. . . . . 42 4.2 Summary of p-values from paired t-tests (Df = 11). (a) P-values from the

paired t-test of SDNNs. (b) P-values from the paired t-test of average HR (c) P-values from the paired t-test of the slopes of the linear regression of the HR change over time. The bold text in a yellow cell highlights the condition that rejects the null hypothesis (P-value < 0.05). Only the bold text presents the condition that has a considerable trend toward significance. . . . . 43

Acknowledgements

I

would like to express my deepest gratitude to my adviser, Prof. Hiroshi Ishii, for his thought-ful, insightful guidance and encouragement. Having research discussions with him have always been a great learning opportunity. He has motivated me to be ambitious, passion-ate, and to have a strong vision. Thanks to his encouragement and generosity, I have been able to express my thoughts clearly, generate creative ideas, and make them real through fun explorations. Without his supportand advice, myworkwould nothave been completed.

Also, I would like to thank Deema Qashat. Without her kind support and help, I would not have been able to begin prototyping my ideas. I would also like to extend my appreciation to the former and current members of the Tangible Media Group: Jifei Ou, Ken Nakagaki, Niko-laos Vlavianos, Takatoshi Yoshida, Joanne Leong, Hila Mor, Joao Wilbert, Koichi Yoshino, Paula Aguilera, Jonathan Williams, Amos Golan, Daniel Levine, and Valentina Sumini. They have always supported each other and made my days at the lab fun, enjoyable, and mem-orable. I was privileged and grateful to have been part of Tangible Media Group. I am so thankful to my awesome teacher and friend, Kerianne Panos, and Jon Edwards for their insightful feedback and support.

I am also very thankful to my thesis readers, Prof. Rosalind Picard and Prof. Canan Dagdeviren, for their support and thoughtful feedback on my research. It has been a great honor to have them as my M.S committee. I would also like to thank Dr. Hae Won Park and Sooyeon Jeong from Personal Robots Group, who helped me to design and conduct user studies.

Educa-tional Foundation in Korea for two years of the M.S program in the MIT Media Lab.

I am deeply grateful to my parents, my brother, and my friend, Seong Soon Jo at MIT, and to my friends in Korea, who have always supported me in pursuing my dream and who have given me endless love. Lastly, I would like to send big love and gratitude to my grand-mother, who was always sending me beautiful smiles, hugs, support, and love. I am sure she would be happy to see my work and would be proud of me.

.7?k

`

j

-

[3LA[21FLIE4f

Introduction

The definition of 'Health' is not merely the absence of disease or infirmity, but also mental and social well-being [73]. A lack of our mental well-being resulting from, for example, emo-tional changes, depression, and loneliness, is a greater risk for heart disease than lack of exercise, smoking, excessive alcohol consumption, and obesity [65, 62].

To address these issues, researchers in the field of Human-computer Interaction (HCI) have improved ways of supporting our physiological state using various bio-monitoring technologies. In addition to such technology, the practice of mindfulness encourages peo-ple to pay attention to their inner body state [59], which has been enhanced by technol-ogy [90]. Commercial products, such as a b Breath [12], Embr [57] and Qiu [10] help users to focus on their breathing by stimulating different perceptions using, for example, light,

temperature, and vibrations. However devices or smartphone applications for practicing

meditation, such as Headspace [42], require full cognitive attention and a dedicated space to help users to focus on the instructions. BrightBeat [35] is effective in regulating stress-levels with effortless mindful breathing guidance via visual and auditory feedback. Also, EmotionCheck [22] demonstrated the effects of false tactile feedback on heart beat rate by providing vibrations on the wrist. However, interfaces that communicate with the user about their physiological state bidirectionally, which lead to a behavior change in the user without requiring cognitive effort, are missing. This stems from the fact that combining the functions of sensing and actuation for feedback into one device is challenging.

In this thesis, I introduce the concept of the BioResonant Interface (Fig. 1-1), which uti-lizes physiological information from the users'bodyto guide their physiological state to

res-Synchronization

Vital Signs H Sublimil BR

BiofeTactc Suion

H

- - -

H

tiultinsE=ode

Input:-Pulse rate, Threshold sensitive to pressureon wrist Output: Stimulation duration, Stimulation Intensity Input. Pulse rate, Breathing rate, Output: Volumechange, Pulse Breathing rate, Hand gesture Out-ut Fabric deformation, Deformation movement r A

Figure 1-1: Overview of the BioResonant Interface concept and following projects

onate with the biofeeback stimulus produced by the interface. It aims at regulating user's physiological states without disturbing their foreground tasks, to ultimately reach mindful awareness in daily life. The BioResonant Interface connects the dialogue between the phys-ical body and the inner emotional and intellectual state.

I present three different forms of the BioResonant Interface: utilizing heart rate (HR) in the form of a wearable, breathing rate (BR) in the form of a cushion, or HR and BR in the form of a kinetic shape-changing display. These are presented in the following order of interaction scale: local interaction with the skin, general interaction with the body, and embodied interaction with the environment.

In Chapter 4, I present ambienBeat, a mobile heart rate rhythmic regulator with

sublim-17

I

I

I

inal tactile stimulus. This wearable device in the form of a watch monitors heart rate and interacts with the skin on the wrist. In Chapter 5, 1 explore using both types of biometric data - HR and BR- to regulate physiological state via somaPneu, an actuating cushion the size of an adult's upper body. It monitors both HR and BR, and provides two different forms of tactile stimulus simultaneously by changing its volume and generating a pulse. These inter-actions occure at a broader somatosensory level than those of ambienBeat. Lastly, Chapter 6 presents a kinetic shape-changing display, reSpire, which regulates an audience's breath-ing patterns via their dynamic motions, which then are expressed as tangible interactions using shape-changing fabrics which represent their own BR.

Contributions of my work are as follows:

1. I present an overview of current technologies related to supporting self-awareness and regulating physiological states through biofeedback, using respiratory sinus ar-rhythmia (RSA) and physiological synchronization.

2. I show how these studies can be used to design a subliminal biofeedback system to guide the user's body to regulate their own physiological state.

3. I describe and evaluate the development of three different BioResonant Interfaces that produce closed-loop biofeedback, which integrate the capability of sensing and actuation for non-invasive regulation of users's HR/BR. Also, I introduce fabrication

methods to develop those interfaces meeting design requirements.

4. I evaluate three different ambient biofeedback modalities: auditory, visual, and tac-tile feedback, through the monitoring of biometric information to verify their effec-tiveness in regulating HR. Also, I present remarks from the user study showing the strength of subliminal tactile stimulation compared to other modalities.

5. I demonstrate that tactile feedback has a significant impact on lowering the user's heart rate in a non-interruptive way compared to other methods, and which can also be used in raising the heart rate. Lastly, I introduce the exploration of a new method

for physiological state regulation by combining tactile and auditory biofeedback.

6. I describe the design implications of this work, including promising directions for the development of closed-loop subliminal biofeedback systems and their potential to

support athletic training, as a mental-health therapeutic device, for sleep monitoring and as a feedback device, as well as an interface for ambient presence and interper-sonal connectedness.

Furthermore, I envision my research to be a general platform which provides access to the dialogue between brain and body so as to assist people in voluntarily, but effortlessly, regulating their emotional state, and possibly customizing their own perceptions. Based on the mechanism of how emotions work in our brain and how the physiological state of the body affects our feelings, I hope to find a way to re-render the same emotions felt during certain memorable moments whenever people feel like accessing them as easily as we now take, store, and share photos using a camera.

Background

2.1

Interoception

As humans, we are able to sense our emotional and physiological states. The perception of internal events-known as interoception-contributes to the regulation of physiological integrity (homeostasis) and associated affective feelings, drives, and emotions. Selfhood is built, in part, through the elaboration of interoceptive representations and their integra-tion with exteroceptive signals [881.

A study by Aspell et al. [4] demonstrated that exteroceptive and interoceptive signals are potent modulators of bodily self-consciousness, and proposed that signals from the inside and the outside of the human body build bodily self-consciousness. The primary method for assessing interoception has been heart beat perception or detection [14].

The feeling of touch acts as self-specifying sensory information [9] since tactile stimuli necessarily provide information on one's own body, unlike visual signals. Interoceptive sig-nals are also self-specifying sensory sigsig-nals and, as such, are afferent sigsig-nals, which arise from the organism's own visceral motor-control processes rather than from the external environment [20, 4].

Haptic interfaces fall into two categories depending on the feedback they transfer: kinesthetic (proprioceptive) ortactile (exteroceptive) feedback [39]. However, there is alack of studies exploring the ways to provide haptic feedback which could externally transfer in-formation present inside the body. As we are not only conscious of external inputs but also

of ourselves as

a

coherent whole [1], developing ways to understand and utilize information

on these internal body functions will enhance human interactions through haptic feedback.

2.2 Respiratory Sinus Arrhythmia (RSA) and Mindfulness

Exhale

Inhale

Z

44h

Figure 2-1: RSA. ECG is shortened during inhalation and prolonged during exhalation.

By reading the distance between the peaks of the heart beat pulse (R-R interval), we can continuously estimate the changing HR, and derive a great deal of information about our cardiovascular state, mood, stress and levels and anxiety. For this reason, one technique to regulate mental state is tightly coupled with cardiovascular afferents, which have numer-ous connections to multiple areas of the brain, including the thalamus, hypothalamus, and amygdala, and play an important role in determining emotional experience [1]. Controlling breathing rhythm has been well known as an indirect yet efficient way to regulate HR and its variability, which is known as respiratory sinus arrhythmia (RSA).

The R-R interval on an electrocardiogram (ECG) is shortened during inspiration and pro-longed during expiration (Fig. 2-1). This phenomena known as, RSA, has been used as an index of cardiac vagal function, which represents the contribution of the parasympathetic nervous system (PNS) to cardiac regulation [94].

Meditation is a form of mental training that has been practiced for thousands of years.

It can be considered to be part of the family of complex emotional and attentional regu-latory training regimens developed for various ends, including the cultivation of well-being and emotional balance [51]. The key factor for meditation is knowing how to control one's breathing rate, which is an effective way to indirectly control one's heartbeat through the principle of RSA. Research with clinically depressed individuals supports this idea by show-ing that RSA biofeedback trainshow-ing may facilitate an increase in HRV amplitude and a de-crease in depressive symptoms [95, 49].

Practicing meditation incorporates attention to internal body sensations. The most com-monly attended body sensations include the breath, the position of the limbs (propriocep-tion), the degree of muscle tension, and the heartbeat [51]. Although attention to internal body sensations is most commonly practiced under conditions of rest, the subjective ex-perience of these interoceptive body sensations is also routinely modulated through ma-nipulations of the breath and musculoskeletal posture, as seen during the practice of yoga exercises [3, 51. Practicing focusing on interoceptive sensations brings enhanced aware-ness of internal body sensations and events, such as the ongoing experience of thoughts and emotions.

Mindfulness [59], which refers to a mental state of being conscious or aware of one's present moment, and accepting feelings, thoughts, and bodily sensations, has been called the "heart" of Buddhist meditation [82, 47, 81]. Over the past 20 years, researchers have focused on evaluating the effects of mindfulness on stress reduction [48] through clinical tests and validated its working mechanisms [81].

2.3 Physiological Synchronization via Touch

Interpersonal touch holds significant social and affective value [36, 70, 34, 13]. Skin-to-skin touch contributes to the healthy development of infants [30], regulates their stress

responses, provides comfort and emotional well-being [67] and can relieve pain [83].

Physi-ologically, it also increases the coupling of electrodermal activity (EDA) and heart rate vari-ability(HRV) [17, 44], and modulates blood pressure reactivity to stress as well as reactivity to distress [37].

This physiological coupling through touch helps people to have empathy [33] and strength-ens social attachment between group members [93], increases conformity [28], builds trust and cooperation among team members [68, 71], and improves physical and mental health [17, 44, 36].

The physiological synchronization between people has been explained by various theo-ries. Durkheim stated [29], "Once the individuals are gathered together, a sort of electricity is generated from their closeness and quickly launches them to an extraordinary height of exaltation" [69]. A study by McCraty et al. [65] showed that information reflecting one's emotional state can be represented by the patterns of the HRV but is also contained in the heart's electromagnetic field radiated into the environment [64, 66]. When an individual is in a heart resonant state, the heart's magnetic field also has a more coherent structure. Researchers have found that a direct mathematical relationship between the HRV patterns and the spectral information is encoded in the magnetic field [66].

Koban et al. [54] argue that predictive coding [52] and the Free Energy Principle could explain interpersonal coordination dynamics. Each brain minimizes coding costs by reduc-ing the mismatch between sensory input and one's own motor behavior. Continuous mu-tual prediction and alignment result in an overall minimization of free energy, thus forming a stable attractor state. Hasson et al. [41] proposed that environmental stimuli, including faces, emit different forms of mechanical, chemical and electromagnetic energy. The sen-sory receptors convert these elements into electrical impulses that the brain uses to gather environmental information and to coordinate an appropriate action. Thus, via the transmis-sion of a signal through the environment, the neural processes in one brain can synchronize to the neural processes in another brain [40, 78].

All these theories support the importance of physically proximity and closeness for im-proved mental wellness and for emotionally rich and supportive communication. These

studies also emphasize the value of tangible interaction using the body. Based on this

the-oretical background, one of the goals of this thesis is to demonstrate the efficacy of tactile feedback, via the medium of touch, to regulate one's mental and/or emotional state via

physiological synchronization.

2.4 Ambient Media

"Once we are located in a world, the door is opened to social interactions among shared things in that world. As we learn to design calm technology, we will enrich not only our space of artifacts, but also our opportunities for being with other people. Thus the design of calm technology come to play a central role in a more humanly empowered twenty-first century."

- Mark Weiser and John Seely Brown, Designing Calm Technology [91]

Even though researchers have focused primarily on exteroceptive sources of informa-tion about the body (i.e., vision, moinforma-tion, and touch), the brain's representainforma-tions of internal bodily states (i.e. interoceptive processes) are equally or even more important for the self [23, 26, 4], and also affect the individual's emotional perception through interactions with others.

The unconscious sense of one's inner state can be likened to ambient media which

sub-tly triggers our unconsciousness while not overloading our cognition. It can be seen as

in-strumental to the approach behind Calm Technology [91]. The example of an office window, given by Weiser et al. in their article [91], illustrates a fundamental property of motion be-tween center and periphery; it allows us freely to attune to the foreground and background even by very small cues that are sometimes unnoticeable depending on the engagement level of the main task. ambientRoom [461, and the water lamp and pinwheels by Dahley et al. [25] are key examples of ambient media as a technology that calms and informs users.

For this thesis and especially for the ambienBeat project introduced in Chapter 4, this principle of calm technology has an important role since the major design requirement of the BioResonant interface is to regulate the physiological state by providing subtle stimu-lation at sub/unconscious cognitive levels to avoid disturbance of task engagement.

Related Work

Stress can cause not only physiological changes but also it can contribute to psychological illnesses [89]. Mindfulness [51, 90] has been identified as one effective way-besides med-ication, sports, social interaction, sleep hygine and others-to deal with stress and mental illness. Mindfulness meditation encourages people to focus on their breathing, indirectly regulating their HR, which in turn can affect their emotional states. There are already many well-known breathing techniques that help people to calm down, and yoga and meditation are some systems of practice. Many studies on the relationship between breathing regu-lation and stress level have proven that biofeedback based on people's breathing pattern helps them to relieve stress and anxiety [87, 49, 60, 55, 43, 95]. Based on these findings, there are studies and commercial products providing different types of biofeedback to help

people to practice self-awareness.

Table. 3.1 shows several related works focusing on regulating stress level and

support-ing mindfulness. Products like Spire Stone [84] and the research project, CalmMeNow [74] mainly focuses on notifying or alerting users to be aware of their current states through

dif-ferent methods of intervention. Also, a wearable personal thermostat, Embr [57], lacks of a

sensing ability and requires the user to control the temperature setting and running time.

Smartphone applications like Headspace [42] and Calm [15] require the user's full-time

at-tention to follow the meditation instructions, as well as a quiet space to focus on the prac-tice. Also, EmotionCheck [22] presented the effect of false tactile feedback of heart beat rate by providing vibrations on the wrist. However, it has a lack of the sensing component to provide a closed-loop biofeedback that can reflect on the user's current physiological state in real-time. All these methods put demands on the users'cognitive load and sometimes

terrupt their engagement with an on-going foreground task. Other Common ways of leading to behavior change of users involve providing feedback in various modalities, such as vibro-tactile, heat, auditory, and visual stimulation. However, devices/interfaces using the sound and visual effect also require the user's attentions and are mostly effective when users are in certain circumstances, such as during a dedicated break time [12], meditation [10], sleep [18, 77], or driving a car [6, 75]. BrightBeat [35] presented about its effectiveness for regulating stress levels with effortless mindful breathing guidance via visual and auditory feedback and presented a vision of the biofeedback, adapting its rate to the user. However, its current version still requires manual effort to combining the functions of sensing and actuation for feedback into a mobile device that can be used in everyday environments

To improve the practice of mindfulness in a non-disruptive way which also enhances the bidirectional communication between users and their physiological states, in the following chapters, I introduce BioResonant Interfaces in three different interaction scales, as also shown in the first three rows of the Table. 3.1.

Table 3.1: Related works

form

Type Name

Research ambienBeat HRV (PPG, PulseSensor) on wrist, 0 HRV pattern guidance Subliminal tactile by oscillation of soft membrane Watch Pressure sensing

tactile stimulation by volume changeCuho

Research somaPneu HRV (ECG), BR sensor on chest X (separated) HRV, BR pattern guidance and pulse generation volumechangeCushion

BR pattern sensing _{Embodied tangible interaction}

Research reSpire with microphone, X (separated) BR pattern guidance by encouraging body movement Kinetic display Kinect gesture sensing

Research Breathing Bear[21 X X Breathing pattern for sleep, calm Inflating/deflating the doll by DC pump Teddy Bear Research EmotionCheck[22] X X BPM False vibrotactile feedback by DC motor Watch Research BrightBeat[35] BR on (BioHarness wearable X (separated) Breathing for calm Light (Brightness) of display, Sound by headset Laptop, Headset

sensor on chest area)

Research Mimo[18] HRV (PPG) on parents'finger X (separated) HRV of parents for sleep, calm Vibrotactile by DC motor box for sensing Research CalmMeNow [741 HR, HRV (ECG), GSR (EDA) X Notification, alerts Vibrotactile by DC motor, SMS message, game Smartphone App., Watch

-only for user test

Research Just Breathe 1751 X X BR rate regulation for destressing Vibrotactile by DC motor array Carseat Research Breath Booster![61 X X Boosting BR rate for drivers Vibrotactile by DC motor array Car seat

Volume changing by

Research Relaxushion[7 Accelerometer for motor control 0 BR regulation for calming motorized rack and gear, smartphone App. Cushion. for visualization

Product Qiu[10] HRV (PPG) on ear X (separated) HRV for meditation Light (color, brightness) Hand-sized ball Product Somneo[77 X X Breathing for sleep Light, (Color, brightness) Sound Lamp, App.

Wearable stone-shaped Product Spire Stone[84] BR, PPG, Accelerometer 0 Notification Smartphone notification, Data visualization device on belt, App. Product Embr Wave[57] X X Body temperature Cooling and heating body Watch

Volume changing for breathing regulation, Pillow,Cushion Product Somnox[77] X X Breathing for sleep Soothing sound Pillow,_Cushion

Product b Breathe[12 BR (pressure sensor) 0 Breathing for calm Light, Vibrotactile Cigarrat

App Headspace[421 X X Breathing Sound (narrative voice), Animation Smartphone app

App Calm[?] X X Breathing Sound (narative voice), Animation Smartphone app

Sensing with Actuation? Guidance Method

ambienBeat: HRV Regulation via Skin

c

dl

Figure 4-1: a smaller version of ambienBeats in different shapes (a) Dome-shapd ambi-enBeat on wrist (b)Bean-shaped amebiambi-enBeat (c) Unfastened ambiambi-enBeats (d) Fastened

ambienBeats

The goal of this work is to develop a mobile heart rate regulator that provides biofeed-back via subliminal tactile stimulus to assist people in regulating their mental stress levels while maintaining their task engagement. Here, I present a mobile heart rate rhythmic reg-ulator, ambienBeat (Fig. 4-1), which provides closed-loop ambient biofeedback via a subtle tactile stimulus based on the user's HR.

4.1

Design Requirements

c

d

Figure 4-2: Regular size of ambienBeat (a) Bottom view of ambienBeat. It has a soft pres-sure sensor pad on the surface which contacts with the dorsal part of forearm. (b) Discon-nected state of ambienBeat. (c, d) ambienBeat when its two straps are conDiscon-nected. The HRV sensor is integrated on the inner surface of the black strap.

For this project in particular, I was interested in providing tactile biofeedback close to user's body but still in non-interruptive way. To achieve this, I set the following design re-quirements.

1. The interface should be wearable and make contact with glabrous skin.

2. To meet the first requirement, the interface should be mobile and lightweight so as to avoid causing the user any uncomfortable feelings.

3. The interface should be able to provide various patterns of tactile stimulus similar to the feeling of a heartbeat pulse. Also, those stimuli should be acoustically noiseless

so as to not disrupt any of the user's foreground tasks.

4. The tactile stimulus provided by the interface should be under the threshold of pres-29

sure sensitivity of the wearer's skin to render subtle sensations.

5. The interface should integrate the functions of both biosensing and biofeedback in a mobile form.

6. All of the feedback provided by the interface should be adjustable in real-time based on the current input values from the user.

Considering all of the design requirements, I developed an ambienBeat that has a watch form that is worn around the wrist area and has an acoustically noiseless soft actuator.

Hardware and system details are described in the following sections.

4.2 Hardware System

STRAP-a-lv9 E_/ STRAP-a MAIN Conductive Velcro ... ConductiveVelcro p Controller LED PC Pressure ON/OFF _Sensor Amplitude Feedback SV Regulator TIP120 3.7V BAT -1 3.7V BAT -2 CI

Figure 4-3: System diagram of ambienBeat and its components. It has three major parts; strap-a, main actuation part, strap-b.

As illustrated in Fig. 4-3, ambienBeat has three main components; two wrist straps (strap-a, b), and a dome-shaped soft actuator part (main). A microcontroller (Teensy 3.2, PJRC), two li-po batteries (220 mAh), a system on/off switch, and a 5V step-up voltage reg-ulator (U3V12F5, Pololu) are embedded inside strap-a, which is made of white faux leather.

On the surface of strap-a, two strips of conductive Velcro and a female snap button-type electrode are placed for the HR sensor's power and data wire connection on strap-b. On the surface of strap-b (black faux leather),the HR sensor (PulseSensor) is fixed and wires for ground, power, and data are embedded inside of the strap. Also, those wires are connected to conductive Velcro and two male snap button-type electrodes (Fig. 4-4(e)) which were sewn on the other surface of the strap-b.

Figure 4-4: Electronic components of ambienBeat. (a) Inside of the soft actuator. (a-1) Sili-cone membrane with a neodymium magnet. (a-2) TIP120 Mosfet for controlling the current through the voice coil (a-3). (a-4) RGB LED for turn ON/OFF, Run indication. (b) Li-po bat-teries (c) ON/OFF Switch. (d) Microcontroller (e) Female button electrode for pulse sensor data transmission from the strap-b. (f) Two conductive thread strips that hold conductive Velcro for the power connection of the pulse sensor from the strap-b.

To achieve the goal of providing non-disruptive tactile feedback, I fabricated an elec-tromagnetic field-driven soft actuator (Fig. 4-4(a), 4-5) using a customized voice-coil and molded a silicone membrane with a neodymium magnet (NdFeB, Grade N52,5233 Gauss Disc magnet) embedded, as this does not produce unwanted acoustic noise. I was inspired bythe work of Nemitz et al. [72], who developed a soft actuator driven by an electromagnetic

A

field for a modular soft wormbot. Also, an electromagnetic haptic display, BubbleWrap [8], used a copper thread to oscillate textile. I coiled a coated 33 AWG copper wire for 250 turns around a hollow aluminum cylinder (permeability k =1.46) that had a diameter of15 mm and height of 50 mm. To fabricate a magnet embedded silicone membrane, I 3D printed a plas-tic negative dome-shaped mold and a cavity to mold a silicone (EcoFlex 00-30, Smooth On).

A barometric MEMS pressure sensor (MPL115A2, NXP) was embedded in the silicone as introduced by Choi et al. [19] for a touch sensitive pressure sensor. I placed it on the bottom part of the device. It is used for getting feedback of the force produced by the voice-coil soft actuator. To provide soft texture for the skin contact area, I molded Ecoflex 00-50 (Smooth On.) in 3D printed molds to make the pressure sensing pad. The main actuation part of the regular size has a diameter of 48 mm and a height of 25 mm. The smaller ambienBeat has a soft actuator with a diameter of 37.9 mm and a height of 15 mm. The wrist strap-a has a fixed length of 156 mm, and strap-b has an adjustable length of between 95 and 110 mm. The total weight of the regular size ambienBeat is 67.8 g. As for the smaller one, it weights 39.5g.

4.3 Noise-less Soft Actuator Mechanism

-- ---

--- --

---

---

---

--

6 m

m

Repulsion

Stable (Off)

Attraction

Figure 4-5: Displacement of soft membrane of the voice-coil actuator (side view) based on the activation state of the coil

For a technical evaluation, I measured the range of impulse exerted by the soft voice-coil actuator, and its power consumption. It was measured by the pressure sensor I embedded on the bottom side of the main part while it was worn on the wrist area. As shown in Fig. 4-5, the maximum displacement of the soft membrane was 2 mm by magnetic repulsion and

6 mm by attraction force. For the purpose of this study, I only used the attraction force to oscillate the membrane in order to render the rhythmic subliminal tactile stimulus. The magnetic field produced by the customized voice-coil was 2.27 mT. The power

consump-tion of the voice-coil was 1.85 W. The full operating time of ambienBeat was 54 min when it was producing a maximum pulse amplitude per second (500 ms On, 500 ms Off).

4.4 HR Regulation System

ambienBeat subliminal tactile feedback

Y t

Figure 4-6: Simplified illustration to show the ultimate goal of ambienBeat's HR regulation

mechanism.

By using the ambienBeat, the main goal is to influence the users' HR pattern to follow the provided tactile stimulation pattern as shown in Fig. 4-6.

Magnet

Silicone membrane

- NL -Voice-coil

Pressure sensor

Heart rate sensor

Figure 4-7: Mechanism of HRV regulation by subliminal tactile feedback

As illustrated in Fig. 4-8, ambienBeat is provided with two control input variables; pulse amplitude (ui), and pulse width (U2). To set the desired output of the pulse amplitude, the system goes through a process ofvinitial calibration. During the process, the ambienBeat produces a range of pulse amplitudes mapped as numbers between 0 and 255 (i). It starts by producing 100, then increases by 27.5 until the user starts to recognize the tactile stim-ulation. During this process, we take the values from the pressure sensor (p) to define the threshold (th) as a function of i. Through this process the threshold system generates the

thWpOi] Tt, ui(i) I+ PW 'YdP(L)] IYi-til-0 IU2 -Y21-0

Syden Cobv

~

SORCU~W U22w-r:Czlmng

th[p] - y.,p] t0 u₂!5r : sfvsshng I [P ,, r(t) Sensor -- --- --- ----+ R-R u₂(t) I Vao~oI JY2(t)Y Soft Abmtor _ _

Figure 4-8: Biofeedback tactile control diagram of ambienBeat, and its desired states.

desired ui. I used a PID controller for the voice-coil soft actuator force. The output force (y) exerted by the coil is measured by the pressure sensor for force feedback. For the pulse width control, I use R-R (r) from a HR sensor and compare with the current output pulse width (y2). Those two values feed through the R-R system for the comparison and it produces different2based on the user's needs for either a calming or a boosting effect. If they want to be calm, the R-R system inputsu2greater than r, and in the case of a boosting

effect, it produces the opposite.

4.5 Evaluation

4.5.1 Method

To evaluate the effectiveness of subliminal tactile stimulation on the increment and decre-ments of HR and compare it with auditory and visual stimulation, I conducted an in-lab experiment. All of the user study procedures were pre-approved by the MIT Institutional Review Board. Twelve subjects between the ages of 18 and 60 (8 females, 4 males) partic-ipated, and they received $20 Amazon gift cards for their participation. None of them were from the Tangible Media Group. They were students and employees from MIT, and all of them were new to the ambienBeat. They were not informed of the purpose of the ambien-Beat and who developed it until they finished their post-survey. I conducted a 55-min study that had 12 conditions for all subjects in a randomized order in each of three sessions that had 1 min breaks in between.

3 min 12m min i k min Inbrek-12mi

{Stimulus

S, : N(Nostimuli), T(Tactile), A(Auditory), V(Visual)

Task T, : S (Sit still), J (Sit still after 30 sec. Jumping Jack), D (Drawing given keywords using amouse)

+ e.x) TJ: Condition when a subject sits still and gets 60 bpm of tactile stimulus after 30 sec. jumping jack.

Figure 4-9: Experimental procedure. All of the tasks and stimulus proceed in random order for each subject.

4.5.2 Experimental Conditions

As described in Fig. 4-9, the study was conducted in three sessions (12 min each); Each session consisted of four different conditions (3 min each) drawn from the combination of 4 different stimuli (Si) and 3 tasks. The stimuli were; N getting no stimuli to have a base

line for comparison, T getting tactile stimuli, A getting auditory stimuli, and V getting

vi-sual feedback, 3 tasks (T). The tasks were; S sit still, J sit still after jumping jacks for 30 sec., and D draw a figure of given keywords with a mouse. During the task S, I provided a 120 bpm (2 Hz) stimulus with a constant R-R interval to examine whether those effects can also boost heartbeat rate. During the other two tasks J, D, I provided 60 bpm (1 Hz) stimulation. However, when the task J or D was paired with the non-stimulation N, I did not provide any feedback. The twelve combinations of stimulus and tasks were presented in randomized order in each session for each participant to minimize the hysteresis for data analysis. Participants were asked to wear the ambienBeat on the wrist area of their

dominant hand, with the HR sensor facing to the palmar side of forearm (glabrous skin). For the visual stimulation, I changed the background light of the computer display to vary smoothly between dark and bright white, as Ghandeharioun et al. [35] introduced for

am-bient light stimulation except I used 1 Hz or 2 Hz while they used slower breathing rate

for stimulus. For the auditory stimulation, I asked participants to put on a noise-cancelling headset, and I played the sound of a heartbeat at 60 bpm and 120 bpm. The measurement

matrices were R-R interval, and HR variation. I used a E4 wristband (empatica) on the wrist of the user's non-dominant hand to collect real-time HR during the study, and also an ex-tra PPG sensor (PulseSensor) on the subject's index fingertip of the non-dominant hand. All of the data were logged throughout. Also, I had participants fill in a post-study survey.

They were asked about their subjective experience of getting stimuli from different modali-ties. Questions included: "How much did you find that the dimming light on the screen was disturbing?", "Were you able to feel the tactile stimulation? If so, how intense was it?" and participants were asked to choose a score from 1 (Not at all) to 10 (Very much). Before starting to run the study, I asked subjects to set the intensity of tactile stimulation so that they find it very subtle or barely sensible. Throughout the user study, I had three hypotheses:

Hi: During the sitting-still task S, the 120 bpm tactile, auditory, and visual stimulation will increase the participant's HR (lower R-R interval). The speed of HR and R-R variance will increase the most under the tactile condition TS.

H2: During the sitting-still period after 30 sec. of jumping jack J, the 60 bpm tactile stim-uli will lower the HR with an increased, constant R-R interval. The lowering will be greatest with the tactile condition TJ.

H3: During drawing with a mouse task D, the disturbance level of the tactile stimuli will

be the lowest. Also, all of the stimuli given at 60 bpm will reduce participants' increased HR with alonger R-R compared to condition ND.

4.6 Data Analysis

I

was able to collect 12 data sets from each subject (4 stimuli with 3 tasks). Each data set contains measurement information of HR, and R-R interval variation. To estimate the stress level variance, I did a time-domain analysis of the HR and R-R interval variation data, which involves calculations of mean normal-to-normal (NN) intervals and the variance between NN intervals [53]. I could derive the standard deviation of the NN interval (SDNN) [80] which is an index of physiological resilience against stress [53]. When HRV is large and irregular, the SDNN value increases. To see how the HR changes overtime under different conditions, the instantaneous HR was obtained by using R-R interval timing (HR = 60000/R-R interval).

Also, to validate the speed of HR change while getting different stimuli, I calculated a slope

I analyzed the data using the single factor ANOVA and the paired T-test to compare effectiveness of each modality in terms of speed, for HR regulation compared to the con-trol group, and also for the comparison of each modality's disturbance level as reported by the subject via survey. Also, I combined the post-survey results with reports about the

participants' experiences and any feedback given during the experiment process.

Preference Ranking of Tactile, Auditory, Visual Stimulation

3.5 2t to 3 2.5 2 1.5

0.5 _{Tactile Auditory Visual}

Figure 4-10: Preference ranking of tactile, auditory, and visual stimulation. Tactile

stimula-tion was ranked first, followed by visual, and auditory.

4.7 Results and Discussion

[H1: Boosting Up] SDNN during Getting120 bpn Stimulation

NS TS AS VS

Mi

120 100 s0 160 40 20

[H2:Calmng Down] SDNN during Getting 60 bpmStimulationafter Jumping Jack

I!!'

NJ TJ AJ VJ

Figure 4-11: SDNN under different conditions for testing boosting up and calming down effects. (a)Sitting still task NS: No stimulation, TS: getting tactile stimulation, AS: getting auditory stimulation, VS: getting visual stimulation. (b)Sitting still after jumping jack task NJ: no stimulation, TJ: getting tactile stimulation, AJ: getting auditory stimulation, VJ: get-ting visual stimulation)

37 - Maximum -- Third quartile X - Mean Median First quartile Minimum 70 0 70 30 250

10

30

I

asked people to rank their preference between the three different forms of stimulation

they have experienced during tasks. As shown in Fig. 4-10, I found that people

preferred

to use the ambienBeat, followed by visual guidance, and auditory guidance. Even though

the analysis result from their biometric data shows that the effective stimulation method

was auditory stimulation over visual, they preferred the visual stimulation over sound.

Us-ing sensors for measurUs-ing biometric data led to noisy data, especially durUs-ing tasks which

required bodily movements (e.g. jumping jack, drawing using

a

mouse).

[HI:BoostingUp] [H2:CalmngDown]St StHandGetting 60 bpm

SltstaandGettingt2Obpmsalmutlon stmulanonarterumpingJacks

0090

I

during different tasks. It represents well the overall of all subject's HR variance over time.

All of the graphs from the twelve participants' HR data can be found in A.1

*Avg. HR (bpm) during Sitting still

U

Aw.HRtbpm)anerhlumpiglack

*

AVg.HR(bpm)duringDraing

110 -T110 S -VS

-100 0 55

NS TS AS VS MS NJ TJ AJ VJ NS ND TD AD V

00 5 5 2050 2 45 5 45 5 g us 1S t

Figure 4-13: Average HR inbpm duringthree different conditions (a) Sitting still taskduring

getting 120 bpm stimulation NS: no stimulation, TS: getting tactile stimulation, AS:

get-ting auditoryestimulation, VS:getget-tingvisual stimulation. (b) Sitget-tingstill after

jumping jack

task during getting 60bpm stimulationNS:nostimulation, NJ:baseline,TJ:getting

tac-tile stimulation, AJ: getting auditory stimulation, VJ: getting visual stimulation.) (c) Drawing

task during getting 60 bpm stimulation (ND: base line during the task, TD: getting tactile

stimulation, AD: getting auditory stimulation, VD: getting visual stimulation.

4.7.1

H1: Boosting Up

a [Hi:BoostingUp] [H2: Calming Down]

HR (bpm)lInearrereson sope HR (bpm)linear regression slope during getting

during getting 120 bpmsmulaton 60 bpm stimulation afterjumping jack

0.3 0.4 0.3 0.2 02 0.1 0.1 00 -0.1 -. -0.2 -0.2 NS TS AS VS NJ TJ AJ VJ

Figure 4-14: Slope of the linear regression line of the instantaneous heart rate from R-R intervals (bpm) overtime under different conditions for testing boosting up (a) and calming down (b) effect. (a) Sitting still task NS: no stimulation, TS: getting tactile stimulation, AS: getting auditory stimulation, VS: getting visual stimulation. (b) Sitting still after jumping jacktask NJ: no stimulation, TJ: getting tactile stimulation, AJ: getting auditory stimulation, VJ: getting visual stimulation

The condition of sitting still was intended to examine the efficacy of boosting up heart rates by different modalities. R-R interval was used to estimate the instantaneous HR change over time. Based on hypothesis H1, as the subject gets stimuli for boosting up (120 bpm), it is expected they will have an increased HR. Fig. 4-12(a) shows an example of these mea-surements.

The graph in Fig. 4-12(a) shows one example of the efficacy of the different regulation methods. Since the E4 wristband sensor took time to get a cleaner signal depending on the participants, the graph plots the instantaneous HR values starting from 5 sec. Auditory stimulation was the most effective way to increase bpm, followed by tactile stimulation (all of the graphs from the 12 participants' HR data can be found in A.1). The graphs in Fig. 4-14 present the slope of linear regression line (dependent variable: time (sec), inde-pendent variable: instantaneous HR (bpm)) of the instantaneous HR (bpm) overtime (sec), which represents how fast HR values varied. The graph in Fig. 4-14(a) shows that the

ditory stimulation was the most efficient in terms of time to boost up the HR. The p-value between the two methods was 0.26 > 0.05, and Fcrit = 4.60 > F = 0.49, so there was no significant difference between AJ and TJ The visual stimulation was the least effective method to increase heart rate.

4.7.2 H2: Calming Down

To evaluate the calming effect of the three different regulation methods, Ilooked at the data obtained under the task of doing jumping jacks (J). On average, after thejumping jacks, bpm was increased by 4.1% from their base line (NS) (Table 4.1). Based on the hypothesis H2, it was expected that subjects would have a decreased mean value of HR. As shown in the graph in Fig. 4-14(b), when tactile stimulation was provided to the participants, the slope of the linear regression line was biggest in terms of decreasing the bpm, while the visual stimulation showed the lowest impact on slowing down the heart rate.

4.7.3 H3: Disturbance Level

Feeling of Disturbinglevel 9 8 7 6 5 4 3 2 ln

0 Tactile Auditory Visual

Figure 4-15: Post-survey result about participants' feeling of disturbance level for each modality during the drawing task

From reviewing the post-survey, participants'most common comment about their expe-riences with different stimulation was that they hardly noticed they were getting feedback during the drawing task (D), unlike when they had to sit still, whether after the jumping jack task or not. Even if the stimulation they were getting was intended as a subtle ambient