This study was supported by the Philosophy and Social Science Research Project of Jiangsu Provincial Department of Education (2018SJA1089), Jiangsu Government Scholarship for Overseas Studies (JS-2018-262), the Natural Science Foundation of Zhejiang Province (LY19G020018) and the National Natural Science Foundation of China (NSFC) (72001096).

This study was conducted in accordance with the principles of the Helsinki Declaration. All participants were informed of the purpose and safety of the experiment and signed the informed consent form before participation. This study was approved by the institutional review board of Jiangsu University of Science and Technology.
1. Experiment procedure
<ol>
	<li>Preparation for the experiment
	<ol>
		<li>Explain informed consent to the participants and ask them to sign the consent form.</li>
		<li>Perform a color vision test on the participants to confirm that they have normal color discrimination.</li>
		<li>Present 30 mineral water brands to participants and ask them to pick the brands that they are familiar with, to ensure that they are unfamiliar with the brands of mineral water used in the experiment.</li>
		<li>Introduce the procedure of the experiment to the participants.</li>
		<li>Conduct a pre-experiment on the participants using mineral water brands other than the ones used in the study, and ensure that they are familiar with the AR and website operations.</li>
		<li>Direct each participant from Group A to perform the AR experiment first, and then to perform the website experiment. Direct each participant from Group B to perform the website experiment first, and then to perform the AR experiment.</li>
	</ol>
	</li>
	<li>Wearing instruments
	<ol>
		<li>fNIRS probes
		<ol>
			<li>Clean the forehead skin of the participants with skin preparation gel (see Table of Materials).</li>
			<li>Wrap the probes (see Table of Materials) in a plastic wrap for sweaty circumstances. Fix the probes on the FP1 and FP2 position according to the international 10-20 system with a black headband45. Use a black bandana to cover the probes to protect them against ambient light and improve signal quality.</li>
			<li>Clean the transmitter and receiver of the probes using a 70% isopropyl alcohol pad after finishing the experiment.</li>
		</ol>
		</li>
		<li>Eye tracker: Capture eye movement in real-world environments using eye tracking glasses (see Table of Materials). Fit prescription lenses on the head unit of the eye tracking glasses magnetically (if required) and ensure that the participants can walk around freely with a corrected sight. 
		&#8203;NOTE: Because the eye tracking glasses are not designed to work in conjunction with standard eye glasses, participants who wear eye glasses may still be included in the study by using optional prescription lenses for correction of either short-sightedness or long-sightedness. Participants can also wear standard contact lenses because although they may slightly increase noise, they do not normally introduce errors in the data. Participants cannot use colored or other lenses that change the appearance of the pupil or iris.</li>
	</ol>
	</li>
</ol>
2. Measures
<ol>
	<li>fNIRS
	<ol>
		<li>Open the recording software (see Table of Materials). Connect the probes to a laptop via a Bluetooth adapter, and then record the changes of concentration of oxygenated hemoglobin (O2Hb) in the participants&#39; prefrontal cortex through the laptop, with a sampling frequency of 10 Hz.</li>
		<li>Check the received light intensity and tissue saturation index (TSI) quality to control data quality. Ensure that the received signal is located between 1% and 95%.</li>
		<li>Ensure that the participants sit in a comfortable position on a chair and maintain the resting state for 2 min to collect baseline data before the real experiment.</li>
		<li>Click on the Start button on the software interface to record the fNIRS data.</li>
	</ol>
	</li>
	<li>Eye tracking
	<ol>
		<li>Set up the hardware for the eye tracking glasses. Connect the USB plug of the eye tracking glasses to a USB port on the laptop. Open the recording software (see Table of Materials) and set the sampling frequency to 120 Hz.</li>
		<li>Calibration: Perform a one-point-calibration. Ask the participant to focus on a clearly identifiable object in their field of view at 0.6 m. Move the crosshair cursor to the object, where the participant is focused on in the scene video and click on the object.</li>
		<li>Press the Record button on the software interface to start recording.</li>
	</ol>
	</li>
	<li>Questionnaire and scale: Present the usability questionnaire and NASA-TLX scale to the participants after they complete the tasks of the AR/website.</li>
</ol>
3. Data analysis
<ol>
	<li>fNIRS data processing
	<ol>
		<li>Convert the optical density values obtained from the fNIRS recording software into concentrations (&#956;mol) according to the modified Beer-Lambert Law46.</li>
		<li>Filter the raw data at low-pass 0.5 Hz to remove systematic noises such as heartbeats and respiration.</li>
		<li>Check and correct the data for motion artifacts by removing the data segments that exceeded three standard deviations above the entire time-series47.</li>
		<li>Export the mean and maximum fNIRS data in the AR and website conditions, and then subtract them from the baseline data.</li>
	</ol>
	</li>
	<li>Eye tracking data processing
	<ol>
		<li>Export the fixation frequency (count/s), total fixation time (ms), average fixation time (ms), saccade frequency (count/s), average saccade time (ms), and average scan path length (px/s) of the participants.</li>
	</ol>
	</li>
	<li>Statistical analysis
	<ol>
		<li>Perform a two-tailed test at a significance level of 0.05. Check the data normality using the Shapiro-Wilk test and perform a difference test. Perform multiple comparison corrections for p-values using the false discovery rate (FDR) method. 
		NOTE: While performing the difference test, the data that followed a normal distribution were analyzed using a paired samples t-test, and the data that did not follow a normal distribution were analyzed using Wilcoxon signed rank test.</li>
	</ol>
	</li>
</ol>

The authors have nothing to disclose.

Critical steps within the protocol 
During the experiment, several steps were considered to ensure reliability of the results. First, participants who are familiar with the brands of mineral water used in the experiment were excluded, because these participants would have performed the task based on their knowledge of the brand. Second, the participants completed a pre-experiment using other brands of mineral water, which was employed to ensure that the participants were familiar with AR and website...

Six frontier technologies that interact with consumers, typically represented by augmented reality, virtual reality, artificial intelligence, wearable technology, robotics, and big data, are reshaping many theoretical models of consumer behavior1. Augmented Reality (AR) is a new technology that could enhance consumer experience and improve consumer satisfaction. It superimposes textual information, images, videos, and other virtual items onto real scenarios to fuse virtuality and reality, thus enhancing information in the real world through explanation, guidance, evaluation, and prediction2. AR provides a new kind of human-computer interaction, creating an immersive shopping experience for consumers, and has led to the development of many applications3,4. However, the consumer acceptance of AR services is still minimal, and many companies are thus cautious about adopting AR technology5,6. The technology acceptance model (TAM) has been widely used to explain and predict the adoption behavior of new information technologies7,8. According to the TAM, the adoption intention for a new technology largely depends on its usability9. Therefore, a possible explanation for the slow consumer acceptance of AR services from the TAM perspective may relate to the usability of the new techniques, which highlights the need to evaluate the usability of AR while shopping10,11.
Usability is defined as the effectiveness, efficiency, and satisfaction of achieving specified goals in a specified context by specified users12. Currently, there are two main methods for evaluating usability: subjective and objective evaluations13. Subjective evaluations rely mainly on self-report methods using questionnaires and scales. Following this line of research, the questionnaire used in this study included five features associated with the information search mode to achieve a goal: (1) efficiency, (2) ease of use, (3) memorability (easy to remember), (4) satisfaction (the information search mode is comfortable and pleasant), and (5) generalizability to other objects14,15,16. In addition, cognitive load, representing the load while performing a particular task on the cognitive system of a learner17, is another core indicator of usability18,19. Thus, this study additionally used the NASA Task Load Index (NASA-TLX)13,20 as a subjective metric to measure the cognitive load while shopping using AR versus shopping using website services. It is noteworthy that self-report methods rely on the ability and willingness of individuals to accurately report their attitudes and/or prior behaviors21, leaving open the possibility for mis-reporting, under-reporting, or bias. Thus, objective measures could be a valuable complement to traditional subjective methods22.
Neuro-Information-Systems (NeuroIS) methods are used for objective evaluation of AR usability. NeuroIS, coined by Dimoka et al. at the 2007 ICIS conference, is attracting increasing attention in the field of information systems (IS)23. NeuroIS uses theories and tools of cognitive neuroscience to better understand the development, adoption, and impact of IS technologies24,25. To date, cognitive neuroscience tools, such as functional magnetic resonance imaging (fMRI), electroencephalogram (EEG), positron emission computed tomography, magnetoencephalography (MEG), and functional near-infrared spectroscopy (fNIRS), are commonly used in NeuroIS studies26,27. For instance, Dimoka and Davis used fMRI to measure the subjects&#39; activations when they interacted with the website, and revealed that perceived ease of use influenced activation in the prefrontal cortex (PFC)28. Similarly, using EEG, Moridis et al. found that frontal asymmetry was closely associated with usefulness29. These results indicate that the PFC may play a key role in usability.
Although achievements have been made in previous NeuroIS studies, the paradigms used in these studies had limited body movements of subjects with low ecological validity, limiting their theoretical and practical contributions. Interacting with technologies such as AR while shopping requires free body movements, and subject constraints largely impair consumer experience as discussed in He et al.22. Thus, brain-imaging tools with high ecological validity are needed for a usability test of information systems. In this regard, fNIRS has unique technical advantages: during fNIRS experiments, subjects can move freely30 to some extent. For instance, previous studies have measured subjects&#39; brain activations during several outdoor activities such as cycling using portable fNIRS31. In addition, fNIRS is low-cost and enables the measurement of brain activations for long periods of time32. In this study, fNIRS was used to objectively measure subjects&#39; level of cognitive load while using the shopping services of AR versus a website.
Eye tracking has been a valuable psychophysiological technique for detecting users&#39; visual attention during a usability test in recent years33 and has also been widely used in NeuroIS studies34. The technique relies on the eye-mind hypothesis, which assumes that the observer&#39;s focus goes where the attention is directed, that visual attention represents the mental process, and that patterns of visual attention reflect human cognitive strategies35,36,37. In the area of AR research, Yang et al. used eye tracking to find that AR advertisement improved consumers&#39; attitudes toward the advertisement by increasing their curiosity and attention38. In the current study, eye tracking was used to measure subjects&#39; attention, including parameters such as total fixation duration, average fixation duration, fixation frequency, saccade frequency, average saccade duration, and average scan path length.
In summary, this study proposes a usability test method that combines subjective and objective evaluations with AR applications as an example. A usability questionnaire and a NASA-TLX scale were used for subjective evaluation, and multimodal measures combining fNIRS and eye tracking were used for objective evaluation39,40.
Experimental design 
Experimental materials: To simulate a real-life shopping context, a product shelf was built in a laboratory, and two different brands of mineral water were placed on the shelf as experimental materials. As essential goods, mineral water was selected because participants would not have bias in subjective evaluations on the basis of their occupational background, gender, and purchasing ability. The price, capacity, and familiarity of the brands were controlled (see Table of Materials) to eliminate the interference of irrelevant variables.
The usability test included two conditions: a smartphone-based AR application (Supplemental Figure 1)&#160;and a website (Supplemental Figure 2). The AR application was programmed based on an AR engine. The website was developed using Python, based on Bootstrap for the front-end and Flask for the back-end. The AR application and website were run and browsed on a smartphone. Among the two different brands of mineral water, one was used as the experimental material in the AR condition, and the other was used in the website condition.
Experimental tasks: Participants were asked to perform four information search tasks that derived from IoT application contexts: the quality of water, the storage temperature, the matching diet, and the price per liter. These four information items are what consumers normally pay attention to when they buy mineral water. There was no time constraint for participants to complete the tasks.
Quality of water: The quality of mineral water commonly includes two indicators: the total dissolved solids (TDS) and the pH value. The TDS reflects the mineral content, and the pH value describes the acidity/alkalinity of the water. These two indicators are related to trace elements contained in the mineral water and influence taste. For instance, Bruvold and Ongerth divided the sensory quality of water into five grades according to its TDS content41. Marcussen et al. found that water has good sensory qualities in the range of 100-400 mg/L TDS42. The TDS and pH value of the two brands of mineral water used in this study were measured using TDS and pH meters, respectively, and then marked on the AR application and the website. While performing the task, participants were required to report the TDS and pH values of the mineral water and confirm whether these values were within the nominal range. In the AR condition, participants could acquire this information by scanning the bottle of water. In the website condition, participants were required to perform four steps: (1) finding a numeric code on the back of the bottle of mineral water, (2) entering the numeric code into a query box to obtain the TDS and pH values for mineral water, (3) searching the nominal range for mineral water on the website, and (4) verbally reporting whether TDS and pH value are within the nominal range for the product.
Storage temperature: The quality of mineral water may decrease during transportation and storage owing to changes in temperature. Experiments have shown that the appropriate temperature for mineral water is between 5 &#176;C and 25 &#176;C during transport and storage. In this temperature range, water does not have a bad odor43. In the present experiment, the storage temperature of the two types of mineral water in different places was marked on the AR application and the website. While performing the task, participants were required to report the storage location and corresponding temperature of the water. In the AR condition, participants could acquire this information by scanning the bottle of water. In the website condition, participants could acquire this information by entering the numeric code into a query box.
Matching diet: Different brands of mineral water are suitable for different menus due to their unique mineral composition and bubble content44. In the present experiment, dietary recommendations for the two mineral waters were marked on the AR application and website. While performing the task, participants were required to report how the mineral water matches the food in the menu. In the AR condition, participants could acquire this information by scanning the water bottle. In the website condition, participants could search for this information on the website.
Price per liter: Currently, the labels on the mineral water bottles in China do not display the price per liter information. This makes it difficult for consumers to distinguish the difference in unit prices of different types of mineral water. Therefore, the present experiment required participants to report the price per liter. In the AR application, participants could acquire the price per liter directly by scanning the bottle of water. In the website condition, the information could be calculated from the unit price and volume on the label.
This study used a within-participant design, with participant inclusion and exclusion criteria as described in Table 1. A total of 40 participants completed the experiment (20 males and 20 females, mean age = 21.31 &#177; 1.16 years). All participants were undergraduates of Jiangsu University of Science and Technology and were randomly arranged into two groups (A and B). In order to avoid the order effect, the experimental order was counterbalanced across the two groups (A/B). Specifically, one group performed the AR condition first and then the website condition, while the other group performed the website first and then the AR condition. The participants were required to complete preparation for the experiment, wear the instruments, and perform the experimental tasks. The inter-experiment interval was set to 10 s to allow cortical activation to return to the baseline level, avoiding cross-influence in the subsequent task. At the end of the AR/website experiment, participants were required to complete the usability questionnaire and the NASA-TLX scale. The experimental flowchart is shown in Figure 1. A photograph of the experimental setup is presented in Figure 2.
Table 1: Inclusion and exclusion criteria for the study. <a href="https://jove.unicartagenaproxy.elogim.com/files/ftp_upload/64667/Table 1 Inclusion and exclusion criteria.xlsx" target="_blank">Please click here to download this Table.</a>
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/64667/64667fig1v2.jpg" /> 
Figure 1: Experimental flowchart. Each experiment lasted ~45 min, with a rest period of 10 s between the tasks. <a href="https://jove.unicartagenaproxy.elogim.com/files/ftp_upload/64667/64667fig1v2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/64667/64667fig02.jpg" /> 
Figure 2: Example setup of the experimental scene. The experimental materials, the participant, and the equipment are shown. <a href="https://jove.unicartagenaproxy.elogim.com/files/ftp_upload/64667/64667fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>AR Engine</td>
			<td>Unity Technologies</td>
			<td>2020.3.1</td>
			<td>AR development platform</td>
		</tr>
		<tr>
			<td>AR SDK</td>
			<td>PTC</td>
			<td>Vuforia Engine 9.8.5</td>
			<td>AR development kit</td>
		</tr>
		<tr>
			<td>Eye Tracker (eye tracking glasses)</td>
			<td>SMI, Germany</td>
			<td>SMI ETG</td>
			<td>Head-mounted eye tracking 
			system</td>
		</tr>
		<tr>
			<td>Eye Tracker Recording software</td>
			<td>SMI, Germany</td>
			<td>iViewETG Software</td>
			<td>Eye Tracker Recording software</td>
		</tr>
		<tr>
			<td>fNIRS probes</td>
			<td>Artinis Medical Systems BV, Netherlands</td>
			<td>Artinis PortaLite</td>
			<td>Light source: Light emitting diodes 
			Wavelengths: Standard nominal 760 and 850 nm</td>
		</tr>
		<tr>
			<td>fNIRS software</td>
			<td>Artinis Medical Systems BV, Netherlands</td>
			<td>OxySoft 3.2.70</td>
			<td>fNIRS data recording and analysis software</td>
		</tr>
		<tr>
			<td>Mineral Water</td>
			<td>Groupe Danone</td>
			<td>Badoit&#160;</td>
			<td>Experimental material in the AR condition&#160;&#160; Capacity: 330ml 
			Price: <img alt="Equation 1" src="/files/ftp_upload/64667/64667eq01.jpg" style="margin: auto;" />6</td>
		</tr>
		<tr>
			<td>Mineral Water</td>
			<td>Nestl&#233;</td>
			<td>Acqua Panna</td>
			<td>Experimental material in the website condition Capacity: 250ml 
			Price: <img alt="Equation 1" src="/files/ftp_upload/64667/64667eq01.jpg" style="margin: auto;" />5.4</td>
		</tr>
		<tr>
			<td>Skin Preparation Gel</td>
			<td>Weaver and Company</td>
			<td>Nuprep</td>
			<td>Clean the forehead skin of the participants</td>
		</tr>
		<tr>
			<td>Smartphone</td>
			<td>Xiaomi</td>
			<td>Redmi K30 Ultra</td>
			<td>Smartphone-based AR application and website</td>
		</tr>
	</tbody>
</table>

usability evaluation of augmented reality: a neuro-information-systems study

This study introduces an experimental paradigm for a usability test of emerging technologies in a management information system (MIS). The usability test included both subjective and objective evaluations. For the subjective evaluation, a usability questionnaire and a NASA-TLX scale were adopted. For the objective evaluation, methods of Neuro-Information-Systems (NeuroIS) were used. From a NeuroIS perspective, this study used mobile fNIRS and eye tracking glasses for multimodal measurements, which solved the problem of ecological validity of cognitive neuroscience tools used in real-world behavior experiments. This study used Augmented Reality (AR) integrated into the Internet of Things (IoT) as an experimental object. Comparing the differences in the neuroimaging data, the physiological data, the usability questionnaire, and the NASA-TLX scale data between the two information-search modes (AR versus a website), information search with AR had a higher efficiency and a lower cognitive load compared with information search with a website during the process of consumption decision-making. The usability experiment results demonstrate that AR, as an emerging technology in retail, can effectively enhance consumer experiences and increase their purchase intention. The experimental paradigm, combining both subjective and objective evaluations in this study, could be applied to a usability test for emerging technologies, such as augmented reality, virtual reality, artificial intelligence, wearable technology, robotics, and big data. It provides a practical experimental solution for the user experience in human-computer-interactions with the adoption of emerging technologies.

The representative results of this study include the usability questionnaire results, eye tracking data analysis, NASA-TLX scale data, fNIRS data analysis, and dynamic cognitive load changes. For the usability questionnaire results, eye tracking data analysis, NASA-TLX scale data and fNIRS data analysis, normality tests, and differences tests were conducted. For dynamic cognitive load changes, this study selected fNIRS and eye tracking data from a single participant to demonstrate the validity of the multimodal measureme...

Watch this Scientific Journal Video about Usability Evaluation of Augmented Reality: A Neuro-Information-Systems Study at JoVE.com

Usability Evaluation of Augmented Reality: A Neuro-Information-Systems Study

This study presents an experimental paradigm for&#160;a&#160;usability test combining subjective and objective evaluations. The objective evaluation adopted Neuro-Information-Systems (NeuroIS) methods, and the subjective evaluation adopted a usability questionnaire and a NASA-Task Load Index (NASA-TLX) scale.

This study introduces an experimental paradigm for a usability test of emerging technologies in a management information system (MIS). The usability test ...

usability-evaluation-augmented-reality-neuro-information-systems

Universidad de Cartagena UDEC

Research

JoVE Journal

Behavior

2.2K Views.  Jiangsu University of Science and Technology. This study presents an experimental paradigm for a usability test combining subjective and objective evaluations. The objective evaluation adopted Neuro-Information-Systems (NeuroIS) methods, and the subjective evaluation adopted a usability questionnaire and a NASA-Task Load Index (NASA-TLX) scale.

Video: Usability Evaluation of Augmented Reality: A Neuro-Information-Systems Study

Assessment and Evaluation of the High Risk Neonate: The NICU Network Neurobehavioral Scale

There has been a long-standing interest in the assessment of the neurobehavioral integrity of the newborn infant. The NICU Network Neurobehavioral Scale (NNNS) was developed as an assessment for the at-risk infant. These are infants who are at increased risk for poor developmental outcome because of insults during prenatal development, such as substance exposure or prematurity or factors such as poverty, poor nutrition or lack of prenatal care that can have adverse effects on the intrauterine environment and affect the developing fetus. The NNNS assesses the full range of infant neurobehavioral performance including neurological integrity, behavioral functioning, and signs of stress/abstinence. The NNNS is a noninvasive neonatal assessment tool with demonstrated validity as a predictor, not only of medical outcomes such as cerebral palsy diagnosis, neurological abnormalities, and diseases with risks to the brain, but also of developmental outcomes such as mental and motor functioning, behavior problems, school readiness, and IQ. The NNNS can identify infants at high risk for abnormal developmental outcome and is an important clinical tool that enables medical researchers and health practitioners to identify these infants and develop intervention programs to optimize the development of these infants as early as possible. The video shows the NNNS procedures, shows examples of normal and abnormal performance and the various clinical populations in which the exam can be used.

The NICU Network Neurobehavioral Scale (NNNS) was developed as an assessment for the at-risk infant. The purpose of this article is to describe the NNNS, provide video examples of the NNNS procedures and discuss the ways in which the exam has been used.

There has been a long-standing interest in the assessment of the neurobehavioral integrity of the newborn infant.  The NICU Network Neurobehavioral Scale ...

Two-photon Calcium Imaging in Mice Navigating a Virtual Reality Environment

In recent years, two-photon imaging has become an invaluable tool in neuroscience, as it allows for chronic measurement of the activity of genetically identified cells during behavior1-6. Here we describe methods to perform two-photon imaging in mouse cortex while the animal navigates a virtual reality environment. We focus on the aspects of the experimental procedures that are key to imaging in a behaving animal in a brightly lit virtual environment. The key problems that arise in this experimental setup that we here address are: minimizing brain motion related artifacts, minimizing light leak from the virtual reality projection system, and minimizing laser induced tissue damage. We also provide sample software to control the virtual reality environment and to do pupil tracking. With these procedures and resources it should be possible to convert a conventional two-photon microscope for use in behaving mice.

Here we describe the experimental procedures involved in two-photon imaging of mouse cortex during behavior in a virtual reality environment.

In recent years, two-photon imaging has become an invaluable tool in neuroscience, as it allows for chronic measurement of the activity of genetically ...

Development of a Virtual Reality Assessment of Everyday Living Skills

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.&#160; Treatment studies aimed at cognitive impairment in patients with schizophrenia not only require demonstration of improvements on cognitive tests, but also evidence that any cognitive changes lead to clinically meaningful improvements.&#160; Measures of &#8220;functional capacity&#8221; index the extent to which individuals have the potential to perform skills required for real world functioning. &#160;Current data do not support the recommendation of any single instrument for measurement of functional capacity.&#160; The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is a novel, interactive gaming based measure of functional capacity that uses a realistic simulated environment to recreate routine activities of daily living. Studies are currently underway to evaluate and establish the VRFCAT&#8217;s sensitivity, reliability, validity, and practicality.&#160;This new measure of functional capacity is practical, relevant, easy to use, and has several features that improve validity and sensitivity of measurement of function in clinical trials of patients with CNS disorders.

A challenge for proving treatment efficacy for cognitive impairments in schizophrenia is finding the optimizing measurement of skills related to everyday functioning.&#160; The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is an interactive gaming based computerized measure aimed at skills associated with everyday functioning, including baseline impairments and treatment related changes.

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.&#160; ...

Evaluation of a Smartphone-based Human Activity Recognition System in a Daily Living Environment

An evaluation method that includes continuous activities in a daily-living environment was developed for Wearable Mobility Monitoring Systems (WMMS) that attempt to recognize user activities. Participants performed a pre-determined set of daily living actions within a continuous test circuit that included mobility activities (walking, standing, sitting, lying, ascending/descending stairs), daily living tasks (combing hair, brushing teeth, preparing food, eating, washing dishes), and subtle environment changes (opening doors, using an elevator, walking on inclines, traversing staircase landings, walking outdoors). 
To evaluate WMMS performance on this circuit, fifteen able-bodied participants completed the tasks while wearing a smartphone at their right front pelvis. The WMMS application used smartphone accelerometer and gyroscope signals to classify activity states. A gold standard comparison data set was created by video-recording each trial and manually logging activity onset times. Gold standard and WMMS data were analyzed offline. Three classification sets were calculated for each circuit: (i) mobility or immobility, ii) sit, stand, lie, or walking, and (iii) sit, stand, lie, walking, climbing stairs, or small standing movement. Sensitivities, specificities, and F-Scores for activity categorization and changes-of-state were calculated.
The mobile versus immobile classification set had a sensitivity of 86.30% &#177; 7.2% and specificity of 98.96% &#177; 0.6%, while the second prediction set had a sensitivity of 88.35% &#177; 7.80% and specificity of 98.51% &#177; 0.62%. For the third classification set, sensitivity was 84.92% &#177; 6.38% and specificity was 98.17 &#177; 0.62. F1 scores for the first, second and third classification sets were 86.17 &#177; 6.3, 80.19 &#177; 6.36, and 78.42 &#177; 5.96, respectively. This demonstrates that WMMS performance depends on the evaluation protocol in addition to the algorithms. The demonstrated protocol can be used and tailored for evaluating human activity recognition systems in rehabilitation medicine where mobility monitoring may be beneficial in clinical decision-making.

A standardized evaluation method was developed for Wearable Mobility Monitoring Systems (WMMS) that includes continuous activities in a realistic daily living environment. Testing with a series of daily living activities can decrease activity recognition sensitivity; therefore, realistic testing circuits are encouraged for valid evaluation of WMMS performance.

An evaluation method that includes continuous activities in a daily-living environment was developed for Wearable Mobility Monitoring Systems (WMMS) that ...

Force and Position Control in Humans - The Role of Augmented Feedback

During motor behaviour, humans interact with the environment by for example manipulating objects and this is only possible because sensory feedback is constantly integrated into the central nervous system and these sensory inputs need to be weighted in order meet the task specific goals. Additional feedback presented as augmented feedback was shown to have an impact on motor control and motor learning. A number of studies investigated whether force or position feedback has an influence on motor control and neural activation. However, as in the previous studies the presentation of the force and position feedback was always identical, a recent study assessed whether not only the content but also the interpretation of the feedback has an influence on the time to fatigue of a sustained submaximal contraction and the (inhibitory) activity of the primary motor cortex using subthreshold transcranial magnetic stimulation. This paper describes one possible way to investigate the influence of the interpretation of feedback on motor behaviour by investigating the time to fatigue of submaximal sustained contractions together with the neuromuscular adaptations that can be investigated using surface EMG. Furthermore, the current protocol also describes how motor cortical (inhibitory) activity can be investigated using subthreshold TMS, a method known to act solely on the cortical level. The results show that when participants interpret the feedback as position feedback, they display a significantly shorter time to fatigue of a submaximal sustained contraction. Furthermore, subjects also displayed an increased inhibitory activity of the primary cortex when they believed to receive position feedback compared when they believed to receive force feedback. Accordingly, the results show that interpretation of feedback results in differences on a behavioural level (time to fatigue) that is also reflected in interpretation-specific differences in the amount of inhibitory M1 activity.

Controlling an identical movement with position or force feedback results in different neural activation and motor behavior. This protocol describes how to investigate behavioral changes by looking at neuromuscular fatigue and how to evaluate motor cortical (inhibitory) activity using subthreshold TMS with respect to the interpretation of augmented feedback.

During motor behaviour, humans interact with the environment by for example manipulating objects and this is only possible because sensory feedback is ...

Mobile Game-based Virtual Reality Program for Upper Extremity Stroke Rehabilitation

Stroke rehabilitation requires repetitive, intensive, goal-oriented therapy. Virtual reality (VR) has the potential to satisfy these requirements. Game-based therapy can promote patients&#39; engagement in rehabilitation therapy as a more interesting and a motivating tool. Mobile devices such as smartphones and tablet PCs can provide personalized home-based therapy with interactive communication between patients and clinicians. In this study, a mobile VR upper extremity rehabilitation program using game applications was developed. The findings from the study show that the mobile game-based VR program effectively promotes upper extremity recovery in patients with stroke. In addition, patients completed two weeks of treatment using the program without adverse effects and were generally satisfied with the program. This mobile game-based VR upper extremity rehabilitation program can substitute for some parts of the conventional therapy that are delivered one-on-one by an occupational therapist. This time-efficient, easy to implement, and clinically effective program would be a good candidate tool for tele-rehabilitation for upper extremity recovery in patients with stroke. Patients and therapists can collaborate remotely through these e-health rehabilitation programs while reducing economic and social costs.

Here, we present a protocol to develop and apply a mobile game-based virtual reality program for the recovery of upper limb dysfunction in patients with stroke. The present study shows that the mobile program is feasible and effectively promotes upper limb recovery in stroke patients.

Stroke rehabilitation requires repetitive, intensive, goal-oriented therapy. Virtual reality (VR) has the potential to satisfy these requirements. ...

Tickling, a Technique for Inducing Positive Affect When Handling Rats

Handling small animals such as rats can lead to several adverse effects. These include the fear of humans, resistance to handling, increased injury risk for both the animals and the hands of their handlers, decreased animal welfare, and less valid research data. To minimize negative effects on experimental results and human-animal relationships, research animals are often habituated to being handled. However, the methods of habituation are highly variable and often of limited effectiveness. More potently, it is possible for humans to mimic aspects of the animals&#39; playful rough-and-tumble behavior during handling. When applied to laboratory rats in a systematic manner, this playful handling, referred to as tickling, consistently gives rise to positive behavioral responses. This article provides a detailed description of a standardized rat tickling technique. This method can contribute to future investigations into positive affective states in animals, make it easier to handle rats for common husbandry activities such as cage changing or medical/research procedures such as injection, and be implemented as a source of social enrichment. It is concluded that this method can be used to efficiently and practicably reduce rats&#39; fearfulness of humans and improve their welfare, as well as reliably model positive affective states.

This article demonstrates the standardized application of playful handling, a tickling technique designed to mimic rat rough-and-tumble play. This technique is effective at reducing fearful reactions to humans and generating positive affect when rats are handled for common husbandry activities and medical and research procedures such as injection.

Handling small animals such as rats can lead to several adverse effects. These include the fear of humans, resistance to handling, increased injury risk ...

A Networked Desktop Virtual Reality Setup for Decision Science and Navigation Experiments with Multiple Participants

Investigating the interactions among multiple participants is a challenge for researchers from various disciplines, including the decision sciences and spatial cognition. With a local area network and dedicated software platform, experimenters can efficiently monitor the behavior of the participants that are simultaneously immersed in a desktop virtual environment and digitalize the collected data. These capabilities allow for experimental designs in spatial cognition and navigation research that would be difficult (if not impossible) to conduct in the real world. Possible experimental variations include stress during an evacuation, cooperative and competitive search tasks, and other contextual factors that may influence emergent crowd behavior. However, such a laboratory requires maintenance and strict protocols for data collection in a controlled setting. While the external validity of laboratory studies with human participants is sometimes questioned, a number of recent papers suggest that the correspondence between real and virtual environments may be sufficient for studying social behavior in terms of trajectories, hesitations, and spatial decisions. In this article, we describe a method for conducting experiments on decision-making and navigation with up to 36 participants in a networked desktop virtual reality setup (i.e., the Decision Science Laboratory or DeSciL). This experiment protocol can be adapted and applied by other researchers in order to set up a networked desktop virtual reality laboratory.

This paper describes a method for conducting multi-user experiments on decision-making and navigation using a networked computer laboratory.

Investigating the interactions among multiple participants is a challenge for researchers from various disciplines, including the decision sciences and ...

Virtual Reality Experiments with Physiological Measures

Virtual reality (VR) experiments are increasingly employed because of their internal and external validity compared to real-world observation and laboratory experiments, respectively. VR is especially useful for geographic visualizations and investigations of spatial behavior. In spatial behavior research, VR provides a platform for studying the relationship between navigation and physiological measures (e.g., skin conductance, heart rate, blood pressure). Specifically, physiological measures allow researchers to address novel questions and constrain previous theories of spatial abilities, strategies, and performance. For example, individual differences in navigation performance may be explained by the extent to which changes in arousal mediate the effects of task difficulty. However, the complexities in the design and implementation of VR experiments can distract experimenters from their primary research goals and introduce irregularities in data collection and analysis. To address these challenges, the Experiments in Virtual Environments (EVE) framework includes standardized modules such as participant training with the control interface, data collection using questionnaires, the synchronization of physiological measurements, and data storage. EVE also provides the necessary infrastructure for data management, visualization, and evaluation. The present paper describes a protocol that employs the EVE framework to conduct navigation experiments in VR with physiological sensors. The protocol lists the steps necessary for recruiting participants, attaching the physiological sensors, administering the experiment using EVE, and assessing the collected data with EVE evaluation tools. Overall, this protocol will facilitate future research by streamlining the design and implementation of VR experiments with physiological sensors.

Virtual reality (VR) experiments can be difficult to implement and require meticulous planning. This protocol describes a method for the design and implementation of VR experiments that collect physiological data from human participants. The Experiments in Virtual Environments (EVE) framework is employed to accelerate this process.

Virtual reality (VR) experiments are increasingly employed because of their internal and external validity compared to real-world observation and ...

Assessing the Coherence of Parents' Short Narratives Regarding their Child Using the Five-Minute Speech Sample Procedure

Valid and efficient measures for assessing the quality of parent-child relationships are needed to facilitate research and evidence-based practice with parents. This paper focuses on such a method, namely Five-Minute Speech Sample-Coherence (FMSS-Coherence). In this method, a parent is asked to speak for five uninterrupted minutes about her/his child and their relationship. The resulted narrative is coded for coherence, namely the extent to which the parent provides&#160;in the narrative&#160;a clear, consistent, multidimensional and well-supported portrayal of the child. FMSS-Coherence is based on attachment research that shows that the coherence of parents&#8217; narratives is indicative of the quality of the parent-child relationship and child adjustment. It overcomes the limitations of attachment narrative measures which are typically labor intensive. FMSS-Coherence is less nuanced than extant attachment narrative measures. Yet, studies of families from different cultural backgrounds, across different child ages and in the context of typically developing children as well as children with special needs suggest that coherence can be reliably evaluated using the FMSS procedure. Furthermore, parents&#8217; FMSS-Coherence is associated with parenting quality and children&#8217;s socio-emotional adjustment. Thus, it holds promise for researchers and practitioners who seek a relatively time- and cost-effective method for assessing the coherence of parents&#8217; narratives regarding their child.

This paper introduces a method for assessing the coherence of parents' short narratives regarding their child and their relationship using the five-minute speech sample procedure.

Valid and efficient measures for assessing the quality of parent-child relationships are needed to facilitate research and evidence-based practice with ...