The authors acknowledge Evan Beisheim, Nicole Capela, Andrew Herbert-Copley for technical and data collection assistance. Project funding was received from the Natural Sciences and Engineering Research Council of Canada (NSERC) and BlackBerry Ltd., including smartphones used in the study.

This protocol was approved by the Ottawa Health Science Network Research Ethics Board.
1. Preparation
<ol>
	<li>Provide participants with an outline of the research, answer any questions, and obtain informed consent. Record participant characteristics (e.g., age, gender, height, weight, waist girth, leg height from the anterior superior iliac spine to the medial malleolus), identification code, and date on a data sheet. Ensure that the second smartphone that is used to capture video is set to at least a 30 frames per second capture rate.</li>
	<li>Securely attach a phone holster to the participant&#39;s front right belt or pant waist. Start the smartphone application that will be used to collect the sensor data (i.e., data logging or WMMS application) on the mobility measurement smartphone and ensure that the application is running appropriately. Place the smartphone in the holster, with back of the device (rear camera) facing outward.</li>
	<li>Start digital video recording on a second smartphone. For anonymity, record the comparison video without showing the person&#39;s face, but ensure to record all activity transitions. The phone can be handheld.</li>
</ol>
2. Activity Circuit
<ol>
	<li>Follow the participant and video their actions, on the second smartphone, while they perform the following actions, spoken by the investigator:
	<ol>
		<li>From a standing position, shake the smartphone to indicate the start of the trial.</li>
		<li>Continue standing for at least 10 sec. This standing phase can be used for phone orientation calibration 14.</li>
		<li>Walk to a nearby chair and sit down.</li>
		<li>Stand up and walk 60 meters to an elevator.</li>
		<li>Stand and wait for the elevator and then walk into the elevator.</li>
		<li>Take the elevator to the second floor.</li>
		<li>Turn and walk into the home environment.</li>
		<li>Walk into the bathroom and simulate brushing teeth.</li>
		<li>Simulate combing hair.</li>
		<li>Simulate washing hands.</li>
		<li>Dry hands using a towel.</li>
		<li>Walk to the kitchen.</li>
		<li>Take dishes from a rack and place them on the counter.</li>
		<li>Fill a kettle with water from the kitchen sink.</li>
		<li>Place the kettle on the stove element.</li>
		<li>Place bread in a toaster.</li>
		<li>Walk to the dining room.</li>
		<li>Sit at a dining room table.</li>
		<li>Simulate eating a meal at the table.</li>
		<li>Stand and walk back to the kitchen sink.</li>
		<li>Rinse off the dishes and place them in a rack.</li>
		<li>Walk from the kitchen back to the elevator.</li>
		<li>Stand and wait for the elevator and then walk into the elevator.</li>
		<li>Take the elevator to the first floor.</li>
		<li>Walk 50 meters to a stairwell.</li>
		<li>Open the door and enter the stairwell.</li>
		<li>Walk up stairs (13 steps, around landing, 13 steps).</li>
		<li>Open the stairwell door into the hallway.</li>
		<li>Turn right and walk down the hall for 15 meters.</li>
		<li>Turn around and walk 15 meters back to the stairwell.</li>
		<li>Open the door and enter the stairwell.</li>
		<li>Walk down stairs (13 steps, around landing, 13 steps).</li>
		<li>Exit the stairwell and walk into a room.</li>
		<li>Lie on a bed.</li>
		<li>Get up and walk 10 meters to a ramp.</li>
		<li>Walk up the ramp, turn around, then down the ramp (20 meters).</li>
		<li>Continue walking into the hall and open the door to outside.</li>
		<li>Walk 100 meters on the paved pathway.</li>
		<li>Turn around and walk back to the room.</li>
		<li>Walk into the room and stand at the starting point.</li>
		<li>Continue standing, and then shake the smartphone to indicate the end of trial.</li>
	</ol>
	</li>
</ol>
3. Trial Completion
<ol>
	<li>Stop the video recording smartphone and ask the participant to remove and return the smartphone and holster. Stop the data logging or WMMS application on the smartphone. Copy the acquired motion data files and the video file from both phones to a computer for post-processing.</li>
</ol>
4. Post-processing
<ol>
	<li>Synchronize timing between the video and the raw sensor data by determining the time when the shake action started. This shaking movement corresponds to a distinct accelerometer signal and video frame. Check for synchronization error by subtracting the end shake time from the start shake time, for sensor and video data sources. Time differences should be similar between the two data sets.</li>
	<li>Determine actual change-of-state times from the gold-standard video by recording the time difference from the start shake time to the video frame at the transition between activities. Use video editing software to obtain timing to within 0.033 sec (i.e., 30 frames per second video rate). Use WMMS software to generate comparable changes-of-state from the sensor data.</li>
	<li>Generate two data sets, one with true activities and the second with predicted activities, by labeling the activity for each video frame (based on the change of state timing) and then calculating the predicted activity at each video frame time from the WMMS output. For WMMS performance evaluation, calculate true positives, false negatives, true negatives, false positives between the gold-standard activity and WMMS predicted activity. Use these parameters to calculate sensitivity, specificity, and F-score outcomes measures. 
	Note: A tolerance setting of 3 data windows on either side of the window being analyzed can be used for determining change-of-state outcomes, and 2 data windows for classification outcomes. For example, since 1 second data windows were used for the WMMS in this study, 3 sec before and after the current window were examined so that consecutive changes within this tolerance are ignored. The consideration was that changes of state that happen in less than 3 sec can be ignored for gross human movement analysis since these states would be considered transitory.</li>
</ol>

<ol>
	<li>Lara, O. D., Labrador, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+Survey+on+Human+Activity+Recognition+using+Wearable+Sensors.">A Survey on Human Activity Recognition using Wearable Sensors.</a> IEEE Communications Surveys Tutorials. 15 (3), 1192-1209 (2013).</li><li>Foerster, F., Smeja, M., Fahrenberg, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detection+of+posture+and+motion+by+accelerometry:+a+validation+study+in+ambulatory+monitoring.">Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring.</a> Computers in Human Behavior. 15 (5), 571-583 (1999).</li><li>Elsmore, T. F., Naitoh, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Monitoring Activity With a Wrist-Worn Actigraph: Effects of Amplifier Passband and threshold Variations. , (1993).</li><li>Kripke, D. F., Webster, J. B., Mullaney, D. J., Messin, S., Mason, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Measuring sleep by wrist actigraph. , (1981).</li><li>Lockhart, J. W., Pulickal, T., Weiss, G. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Applications+of+mobile+activity+recognition.">Applications of mobile activity recognition.</a> Proceedings of the 2012 ACM Conference on Ubiquitous Computing. , 1054-1058 (2012).</li><li>Incel, O. D., Kose, M., Ersoy, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+Review+and+Taxonomy+of+Activity+Recognition+on+Mobile+Phones.">A Review and Taxonomy of Activity Recognition on Mobile Phones.</a> BioNanoScience. 3 (2), 145-171 (2013).</li><li>Yang, C. C., Hsu, Y. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+Review+of+Accelerometry-Based+Wearable+Motion+Detectors+for+Physical+Activity+Monitoring.">A Review of Accelerometry-Based Wearable Motion Detectors for Physical Activity Monitoring.</a> Sensors. 10 (8), 7772-7788 (2010).</li><li>He, Y., Li, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Physical+Activity+Recognition+Utilizing+the+Built-In+Kinematic+Sensors+of+a+Smartphone.">Physical Activity Recognition Utilizing the Built-In Kinematic Sensors of a Smartphone.</a> International Journal of Distributed Sensor Networks. 2013, (2013).</li><li>Vo, Q. V., Hoang, M. T., Choi, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Personalization+in+Mobile+Activity+Recognition+System+using+K-medoids+Clustering+Algorithm.">Personalization in Mobile Activity Recognition System using K-medoids Clustering Algorithm.</a> International Journal of Distributed Sensor Networks. 2013, (2013).</li><li>Dernbach, S., Das, B., Krishnan, N. C., Thomas, B. L., Cook, D. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Simple+and+Complex+Activity+Recognition+through+Smart+Phones.">Simple and Complex Activity Recognition through Smart Phones.</a> , 214-221 (2012).</li><li>Hache, G., Lemaire, E. D., Baddour, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mobility+change-of-state+detection+using+a+smartphone-based+approach.">Mobility change-of-state detection using a smartphone-based approach.</a> IEEE International Workshop on Medical Measurements and Applications Proceedings (MeMeA). , 43-46 (2010).</li><li>Wu, H. H., Lemaire, E. D., Baddour, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Change-of-state+determination+to+recognize+mobility+activities+using+a+BlackBerry+smartphone.">Change-of-state determination to recognize mobility activities using a BlackBerry smartphone.</a> , 5252-5255 (2011).</li><li>Wu, H., Lemaire, E. D., Baddour, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activity+Change-of-state+Identification+Using+a+Blackberry+Smartphone.">Activity Change-of-state Identification Using a Blackberry Smartphone.</a> Journal of Medical and Biological Engineering. 32 (4), 265-271 (2012).</li><li>Tundo, M. D., Lemaire, E., Baddour, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Correcting+Smartphone+orientation+for+accelerometer-based+analysis.">Correcting Smartphone orientation for accelerometer-based analysis.</a> IEEE International Symposium on Medical Measurements and Applications Proceedings (MeMeA). , 58-62 (2013).</li><li>Tundo, M. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Development of a human activity recognition system using inertial measurement unit sensors on a smartphone. , (2014).</li></ol>

The authors declare that they have no competing financial interests.

Human activity recognition with a wearable mobility monitoring system has received more attention in recent years due to the technical advances in wearable computing and smartphones and systematic needs for quantitative outcome measures that help with clinical decision-making and health intervention evaluation. The methodology described in this paper was effective for evaluating WMMS development since activity classification errors were found that would not have been present if a broad range of activities of daily living...

Ubiquitous sensing has become an engaging research area due to increasingly powerful, small, low cost computing and sensing equipment&#160;1. Mobility monitoring using wearable sensors has generated a great deal of interest since consumer-level microelectronics are capable of detecting motion characteristics with high accuracy&#160;1. Human activity recognition (HAR) using wearable sensors is a recent area of research, with preliminary studies performed in the 1980s and 1990s 2-4.
Modern smartphones contain the necessary sensors and real-time computation capability for mobility activity recognition. Real-time analysis on the device permits activity classification and data upload without user or investigator intervention. A smartphone with mobility analysis software could provide fitness tracking, health monitoring, fall detection, home or work automation, and self-managing exercise programs 5. Smartphones can be considered inertial measurement platforms for detecting mobile activities and mobile patterns in humans, using generated mathematical signal features calculated with onboard sensor outputs 6. Common feature generation methods include heuristic, time-domain, frequency-domain, and wavelet analysis-based approaches&#160;7.
Modern smartphone HAR systems have shown high prediction accuracies when detecting specified activities&#160;1,5,6,7. These studies vary in evaluation methodology as well as accuracy since most studies have their own training set, environmental setup, and data collection protocol. Sensitivity, specificity, accuracy, recall, precision, and F-Score are commonly used to describe prediction quality. However, little to no information is available on methods for &#34;concurrent activity&#34; recognition and evaluation of the ability to detect activity changes in real-time 1, for HAR systems that attempt to categorize several activities. Assessment methods for HAR system accuracy vary substantially between studies. Regardless of the classification algorithm or applied features, descriptions of gold standard evaluation methods are vague for most HAR research.
Activity recognition in a daily living environment has not been extensively researched. Most smartphone-based activity recognition systems are evaluated in a controlled manner, leading to an evaluation protocol that may be advantageous to the algorithm rather than realistic to a real-world environment. Within their evaluation scheme, participants often perform only the actions intended for prediction, rather than applying a large range of realistic activities for the participant to perform consecutively, mimicking real-life events.
Some smartphone HAR studies 8,9 group similar activities together, such as stairs and walking, but exclude other activities from the data set. Prediction accuracy is then determined by how well the algorithm identified the target activities. Dernbach et al. 9 had participants write the activity they were about to execute before moving, interrupting continuous change-of-state transitions. HAR system evaluations should assess the algorithm while the participant performs natural actions in a daily living setting. This would permit a real-life evaluation that replicates daily use of the application. A realistic circuit includes many changes-of-state as well as a mix of actions not predicable by the system. An investigator can then assess the algorithm&#39;s response to these additional movements, thus evaluating the algorithm&#39;s robustness to anomalous movements.
This paper presents a Wearable Mobility Monitoring System (WMMS) evaluation protocol that uses a controlled course that reflects real-life daily living environments. WMMS evaluation can then be made under controlled but realistic conditions. In this protocol, we use a third-generation WMMS that was developed at the University of Ottawa and Ottawa Hospital Research Institute 11-15. The WMMS was designed for smartphones with a tri-axis accelerometer and gyroscope. The mobility algorithm accounts for user variability, provides a reduction in the number of false positives for changes-of-state identification, and increases sensitivity in activity categorization. Minimizing false positives is important since the WMMS triggers short video clip recording when activity changes of state are detected, for context-sensitive activity evaluation that further improves WMMS classification. Unnecessary video recording creates inefficiencies in storage and battery use. The WMMS algorithm is structured as a low-computational learning model and evaluated using different prediction levels, where an increase in prediction level signifies an increase in the amount of recognizable actions.

<table><tbody><tr><td>Smartphone or wearable measurement device</td><td>Blackberry</td><td>Z10</td><td></td></tr><tr><td>Smartphone for video recording</td><td>Blackberry</td><td>Z10 or 9800</td><td></td></tr><tr><td>Phone holster</td><td>Any</td><td></td><td></td></tr><tr><td>Data logger application for the smartphone</td><td>BlackBerry World - TOHRC Data Logger for BlackBerry 10</td><td></td><td>&lt;a href=&quot;http://appworld.blackberry.com/webstore/content/32013891/?countrycode=CA&quot;&gt;http://appworld.blackberry.com/webstore/content/32013891/?countrycode=CA&lt;/a&gt;</td></tr><tr><td>Wearable mobility measurement</td><td>Custom Blackberry 10 and Matlab software for mobility monitoring</td><td></td><td>&lt;a href=&quot;http://www.irrd.ca/cag/smartphone/&quot;&gt;http://www.irrd.ca/cag/smartphone/&lt;/a&gt;</td></tr><tr><td>Video editing or analysis software</td><td>Motion Analysis Tools</td><td></td><td>&lt;a href=&quot;http://www.irrd.ca/cag/mat/&quot;&gt;http://www.irrd.ca/cag/mat/&lt;/a&gt;</td></tr></tbody></table>

evaluation of a smartphone-based human activity recognition system in a daily living environment

An evaluation method that includes continuous activities in a daily-living environment was developed for Wearable Mobility Monitoring Systems (WMMS) that attempt to recognize user activities. Participants performed a pre-determined set of daily living actions within a continuous test circuit that included mobility activities (walking, standing, sitting, lying, ascending/descending stairs), daily living tasks (combing hair, brushing teeth, preparing food, eating, washing dishes), and subtle environment changes (opening doors, using an elevator, walking on inclines, traversing staircase landings, walking outdoors). 
To evaluate WMMS performance on this circuit, fifteen able-bodied participants completed the tasks while wearing a smartphone at their right front pelvis. The WMMS application used smartphone accelerometer and gyroscope signals to classify activity states. A gold standard comparison data set was created by video-recording each trial and manually logging activity onset times. Gold standard and WMMS data were analyzed offline. Three classification sets were calculated for each circuit: (i) mobility or immobility, ii) sit, stand, lie, or walking, and (iii) sit, stand, lie, walking, climbing stairs, or small standing movement. Sensitivities, specificities, and F-Scores for activity categorization and changes-of-state were calculated.
The mobile versus immobile classification set had a sensitivity of 86.30% &#177; 7.2% and specificity of 98.96% &#177; 0.6%, while the second prediction set had a sensitivity of 88.35% &#177; 7.80% and specificity of 98.51% &#177; 0.62%. For the third classification set, sensitivity was 84.92% &#177; 6.38% and specificity was 98.17 &#177; 0.62. F1 scores for the first, second and third classification sets were 86.17 &#177; 6.3, 80.19 &#177; 6.36, and 78.42 &#177; 5.96, respectively. This demonstrates that WMMS performance depends on the evaluation protocol in addition to the algorithms. The demonstrated protocol can be used and tailored for evaluating human activity recognition systems in rehabilitation medicine where mobility monitoring may be beneficial in clinical decision-making.

The study protocol was conducted with a convenience sample of fifteen able-bodied participants whose average weight was 68.9 (&#177; 11.1) kg, height was 173.9 (&#177; 11.4) cm, and age was 26 (&#177; 9) years, recruited from The Ottawa Hospital and University of Ottawa staff and students. A smartphone captured sensor data at a variable 40-50 Hz rate. Sample rate variations are typical for smartphone sensor sampling. A second smartphone was used to record digital video at 1280x720 (720p) resolution.
<p class="jove_co...

Watch this Scientific Journal Video about Evaluation of a Smartphone-based Human Activity Recognition System in a Daily Living Environment at JoVE.com

Evaluation of a Smartphone-based Human Activity Recognition System in a Daily Living Environment

A standardized evaluation method was developed for Wearable Mobility Monitoring Systems (WMMS) that includes continuous activities in a realistic daily living environment. Testing with a series of daily living activities can decrease activity recognition sensitivity; therefore, realistic testing circuits are encouraged for valid evaluation of WMMS performance.

An evaluation method that includes continuous activities in a daily-living environment was developed for Wearable Mobility Monitoring Systems (WMMS) that ...

evaluation-smartphone-based-human-activity-recognition-system-daily

Universidad de Cartagena UDEC

Research

JoVE Journal

Behavior

8.8K Views.  University of Ottawa. A standardized evaluation method was developed for Wearable Mobility Monitoring Systems (WMMS) that includes continuous activities in a realistic daily living environment. Testing with a series of daily living activities can decrease activity recognition sensitivity; therefore, realistic testing circuits are encouraged for valid evaluation of WMMS performance.

Video: Evaluation of a Smartphone-based Human Activity Recognition System in a Daily Living Environment

Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data

An end-to-end suite of technologies has been established for the unobtrusive and continuous monitoring of health and activity changes occurring in the daily life of older adults over extended periods of time. The technology is aggregated into a system that incorporates the principles of being minimally obtrusive, while generating secure, privacy protected, continuous objective data in real-world (home-based) settings for months to years. The system includes passive infrared presence sensors placed throughout the home, door contact sensors installed on exterior doors, connected physiological monitoring devices (such as scales), medication boxes, and wearable actigraphs. Driving sensors are also installed in participants' cars and computer (PC, tablet or smartphone) use is tracked. Data is annotated via frequent online self-report options that provide vital information with regard to the data that is difficult to infer via sensors such as internal states (e.g., pain, mood, loneliness), as well as data referent to activity pattern interpretation (e.g., visitors, rearranged furniture). Algorithms have been developed using the data obtained to identify functional domains key to health or disease activity monitoring, including mobility (e.g., room transitions, steps, gait speed), physiologic function (e.g., weight, body mass index, pulse), sleep behaviors (e.g., sleep time, trips to the bathroom at night), medication adherence (e.g., missed doses), social engagement (e.g., time spent out of home, time couples spend together), and cognitive function (e.g., time on computer, mouse movements, characteristics of online form completion, driving ability). Change detection of these functions provides a sensitive marker for the application in health surveillance of acute illnesses (e.g., viral epidemic) to the early detection of prodromal dementia syndromes. The system is particularly suitable for monitoring the efficacy of clinical interventions in natural history studies of geriatric syndromes and in clinical trials.

Unobtrusive sensors and pervasive computing technology incorporated into the daily home life of older adults enables meaningful health and activity changes to be recorded continuously for months to years, providing ecologically valid, high frequency, multi-domain data for research or clinical use.

An end-to-end suite of technologies has been established for the unobtrusive and continuous monitoring of health and activity changes occurring in the ...

Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based assessments of IADL are easy to use but prone to subjective bias. Here, we describe a novel virtual reality (VR) test to assess two complex IADL tasks: handling financial transactions and using public transportation. While a participant performs the tasks in a VR setting, a motion capture system traces the position and orientation of the dominant hand and head in a three-dimensional Cartesian coordinate system. Kinematic raw data are collected and converted into &#39;kinematic performance measures,&#39; i.e., motion trajectory, moving distance, and time to completion. Motion trajectory is the path of a particular body part (e.g., dominant hand or head) in space. Moving distance refers to the total distance of the trajectory, and time to completion is how long it took to complete an IADL task. These kinematic measures could discriminate patients with cognitive impairment from healthy controls. The development of this kinematic measuring protocol allows detection of early IADL-related cognitive impairments.

We designed a virtual reality test to assess instrumental activities of daily living (IADL) with a motion capture system. We propose a detailed kinematic analysis to interpret the participant&#39;s various movements, including trajectory, moving distance, and time to completion to evaluate IADL capabilities.

The inability to complete instrumental activities of daily living (IADL) is a precursor to various neuropsychological diseases. Questionnaire-based ...

An Application for Pairing with Wearable Devices to Monitor Personal Health Status

The current protocol aims to showcase the technology integration, providing a detailed description of adopting the HealthCloud app, developed by the Healthy Landscape and Healthy People Lab, National Taiwan University (HLHP-NTU), on smartphones and smartwatches to collect data on users' real-time psychological and physiological responses and environmental information. A flexible and integrated research method was proposed because it can be difficult to measure multi-dimensional aspects of personal data in on-site studies in landscape and outdoor recreation research. An on-site study conducted in 2020 at the National Taiwan University campus was used as an application example. A dataset of 385 participants was used after excluding invalid samples. During the experiment, participants were asked to walk around campus for 30 min when their heart rate and psychological-scale items were measured, together with several environmental metrics. This work aimed to provide a possible solution to help on-site studies track real-time human responses that match ambient factors. Due to the app's flexibility, its use on wearable devices shows excellent potential for multidisciplinary research studies.

The present protocol introduces a non-commercial self-developed application for collecting real-time on-site data, including psychological scales, GPS location, heart rate, and blood-oxygen saturation level, as well as the application's operating procedures. An empirical study conducted in Taiwan in 2020 was used as an application example.

The current protocol aims to showcase the technology integration, providing a detailed description of adopting the HealthCloud app, developed by the ...

Environment

Enhancing Classroom Engagement: AI-Based Analysis of Student Attention Levels

Artificial Intelligence-Based System for Detecting Attention Levels in Students

The attention level of students in a classroom can be improved through the use of Artificial Intelligence (AI) techniques. By automatically identifying the attention level, teachers can employ strategies to regain students' focus. This can be achieved through various sources of information.
One source is to analyze the emotions reflected on students' faces. AI can detect emotions, such as neutral, disgust, surprise, sadness, fear, happiness, and anger. Additionally, the direction of the students' gaze can also potentially indicate their level of attention. Another source is to observe the students' body posture. By using cameras and deep learning techniques, posture can be analyzed to determine the level of attention. For example, students who are slouching or resting their heads on their desks may have a lower level of attention. Smartwatches distributed to the students can provide biometric and other data, including heart rate and inertial measurements, which can also be used as indicators of attention. By combining these sources of information, an AI system can be trained to identify the level of attention in the classroom. However, integrating the different types of data poses a challenge that requires creating a labeled dataset. Expert input and existing studies are consulted for accurate labeling. In this paper, we propose the integration of such measurements and the creation of a dataset and a potential attention classifier. To provide feedback to the teacher, we explore various methods, such as smartwatches or direct computers. Once the teacher becomes aware of attention issues, they can adjust their teaching approach to re-engage and motivate the students. In summary, AI techniques can automatically identify the students' attention level by analyzing their emotions, gaze direction, body posture, and biometric data. This information can assist teachers in optimizing the teaching-learning process.

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

This paper proposes an artificial intelligence-based system to automatically detect whether students are paying attention to the class or are distracted. This system is designed to help teachers maintain students' attention, optimize their lessons, and dynamically introduce modifications in order for them to be more engaging.

The attention level of students in a classroom can be improved through the use of Artificial Intelligence (AI) techniques. By automatically identifying ...

Engineering

Home-Based Monitor for Gait and Activity Analysis

Current outcomes in neuromuscular disorder clinical trials include motor function scales, timed tests, and strength measures performed by trained clinical evaluators. These measures are slightly subjective and are performed during a visit to a clinic or hospital and constitute therefore a point assessment. Point assessments can be influenced by daily patient condition or factors such as fatigue, motivation, and intercurrent illness. To enable home-based monitoring of gait and activity, a wearable magneto-inertial sensor (WMIS) has been developed. This device is a movement monitor composed of two very light watch-like sensors and a docking station. Each sensor contains a tri-axial accelerometer, gyroscope, magnetometer, and a barometer that record linear acceleration, angular velocity, the magnetic field of the movement in all directions, and barometric altitude, respectively. The sensors can be worn on the wrist, ankle, or wheelchair to record the subject&#8217;s movements during the day. The docking station enables data uploading and recharging of sensor batteries during the night. Data are analyzed using proprietary algorithms to compute parameters representative of the type and intensity of the performed movement. This WMIS can record a set of digital biomarkers, including cumulative variables, such as total number of meters walked, and descriptive gait variables, such as the percentage of the most rapid or longest stride that represents the top performance of patient over a predefined period of time.

This innovative device uses magneto-inertial sensors to permit gait and activity analysis in uncontrolled environments. Currently in the qualification process as an outcome measure in the European Medical Agency, one of the applications will be to serve as a clinical endpoint in clinical trials in neuromuscular diseases.

Current outcomes in neuromuscular disorder clinical trials include motor function scales, timed tests, and strength measures performed by trained clinical ...

Bioengineering

Video Movement Analysis Using Smartphones (ViMAS): A Pilot Study

The use of smartphones in clinical practice is steadily increasing with the availability of low cost/freely available &#34;apps&#34; that could be used to assess human gait. The primary aim of this manuscript is to test the concurrent validity of kinematic measures recorded by a smartphone application in comparison to a 3D motion capture system in the sagittal plane. The secondary aim was to develop a protocol for clinicians on the set up of the smartphone camera for video movement analysis.
The sagittal plane knee angle was measured during heel strike and toe off events using the smart phone app and a 3D motion-capture system in 32 healthy subjects. Three trials were performed at near (2-m) and far (4-m) smartphone camera distances. The order of the distances was randomized. Regression analysis was performed to estimate the height of the camera based on either the subject&#39;s height or leg length.
Absolute measurement errors were least during toe off (3.12 &#177; 5.44 degrees) compared to heel strike (5.81 &#177; 5.26 degrees). There were significant (p &#60; 0.05) but moderate agreements between the application and 3D motion capture measures of knee angles. There were also no significant (p &#62; 0.05) differences between the absolute measurement errors between the two camera positions. The measurement errors averaged between 3 - 5 degrees during toe off and heel strike events of the gait cycle.
The use of smartphone apps can be a useful tool in the clinic for performing gait or human movement analysis. Further studies are needed to establish the accuracy in measuring movements of the upper extremity and trunk.

Video Movement Analysis Using Smartphones ViMAS: A Pilot Study

This manuscript describes the method to test the concurrent validity of kinematic measures recorded by the smartphone application in comparison to a 3D motion capture system in the sagittal plane. This protocol will enable clinicians to set up smartphones for video capture of human movement.

The use of smartphones in clinical practice is steadily increasing with the availability of low cost/freely available &#34;apps&#34; that could be used to ...

Medicine

Design and Analysis for Fall Detection System Simplification

This paper presents a methodology based on multimodal sensors to configure a simple, comfortable and fast fall detection and human activity recognition system that can be easily implemented and adopted. The methodology is based on the configuration of specific types of sensors, machine-learning methods and procedures. The protocol is divided into four phases: (1) database creation (2) data analysis (3) system simplification and (4) evaluation. Using this methodology, we created a multimodal database for fall detection and human activity recognition, namely UP-Fall Detection. It comprises data samples from 17 subjects that perform 5 types of falls and 6 different simple activities, during 3 trials. All information was gathered using 5 wearable sensors (tri-axis accelerometer, gyroscope and light intensity), 1 electroencephalograph helmet, 6 infrared sensors as ambient sensors, and 2 cameras in lateral and front viewpoints. The proposed novel methodology adds some important stages to perform a deep analysis of the following design issues in order to simplify a fall detection system: a) select which sensors or combination of sensors are to be used in a simple fall detection system, b) determine the best placement of the sources of information, and c) select the most suitable machine learning classification method for fall and human activity detection and recognition. Even though some multimodal approaches reported in literature only focus on one or two of the above-mentioned issues, our methodology allows simultaneously solving these three design problems related to a human fall and activity detection and recognition system.

We present a methodology based on multimodal sensors to configure a simple, comfortable and fast fall detection and human activity recognition system. The goal is to build a system for accurate fall detection that can be easily implemented and adopted.

This paper presents a methodology based on multimodal sensors to configure a simple, comfortable and fast fall detection and human activity recognition ...

A Method for Quantifying Upper Limb Performance in Daily Life Using Accelerometers

A key reason for referral to rehabilitation services after stroke and other neurological conditions is to improve one's ability to function in daily life. It has become important to measure a person's activities in daily life, and not just measure their capacity for activity in the structured environment of a clinic or laboratory. A wearable sensor that is now enabling measurement of daily movement is the accelerometer. Accelerometers are commercially-available devices resembling large wrist watches that can be worn throughout the day. Data from accelerometers can quantify how the limbs are engaged to perform activities in peoples' homes and communities. This report describes a methodology to collect accelerometry data and turn it into clinically-relevant information. First, data are collected by having the participant wear two accelerometers (one on each wrist) for 24 h or longer. The accelerometry data are then downloaded and processed to produce four different variables that describe key aspects of upper limb activity in daily life: hours of use, use ratio, magnitude ratio, and the bilateral magnitude. Density plots can be constructed that visually represent the data from the 24 h wearing period. The variables and their resultant density plots are highly consistent in neurologically-intact, community-dwelling adults. This striking consistency makes them a useful tool for determining if upper limb daily performance is different from normal. This methodology is appropriate for research studies investigating upper limb dysfunction and interventions designed to improve upper limb performance in daily life in people with stroke and other patient populations. Because of its relative simplicity, it may not be long before it is also incorporated in clinical neurorehabilitation practice.

This protocol describes a method to quantify upper limb performance in daily life using wrist-worn accelerometers.

A key reason for referral to rehabilitation services after stroke and other neurological conditions is to improve one's ability to function in daily life. ...

Capturing Representative Hand Use at Home Using Egocentric Video in Individuals with Upper Limb Impairment

Impaired hand function after neurological injuries can have a major impact on independence and quality of life. Most existing upper limb assessments are carried out in person, which is not always indicative of hand use in the community. Novel approaches to capture hand function in daily life are required to measure the true impact of rehabilitation interventions. Egocentric video combined with computer vision for automated analysis has been proposed to evaluate hand use at home. However, there are limitations to the duration of continuous recordings. We present a protocol designed to ensure that the videos obtained are representative of daily routines while respecting participant privacy. 
A representative recording schedule is selected through a collaborative process between the researchers and participants, to ensure that the videos capture natural tasks and performance, while being useful for hand assessment. Use of the equipment and procedures is demonstrated to the participants. A total of 3 h of video recordings are scheduled over two weeks. To reduce privacy concerns, participants have full control to start and stop recordings, and the opportunity to edit the videos before returning them to the research team. Reminders are provided, as well as help calls and home visits if necessary. 
The protocol was tested with 9 stroke survivors and 14 individuals with cervical spinal cord injury. The videos obtained contained a variety of activities, such as meal preparation, dishwashing, and knitting. An average of 3.11 &plusmn; 0.98 h of video were obtained. The recording periods varied from 12-69 d, due to illness or unexpected events in some cases. Data was successfully obtained from twenty-two out of 23 participants, with 6 participants requiring assistance from the investigators during the home recording period. The protocol was effective for collecting videos that contained valuable information about hand function at home after neurological injuries.

A protocol is proposed to capture natural hand function of individuals with hand impairments during their daily routines using an egocentric camera. The goal of the protocol is to ensure that the recordings are representative of an individual&#39;s typical hand use during activities of daily living at home.

Impaired hand function after neurological injuries can have a major impact on independence and quality of life. Most existing upper limb assessments are ...

Setup of Consumer Wearable Devices for Exposure and Health Monitoring in Population Studies

Wearable sensors, which are often embedded in commercial smartwatches, allow for continuous and non-invasive health measurements and exposure assessments in clinical studies. Nevertheless, the real-life application of these technologies in studies involving a large number of participants for a significant observation period may be hindered by several practical challenges. 
In this study, we present a modified protocol from a previous intervention study for the mitigation of health effects from desert dust storms. The study involved two distinct population groups: asthmatic children aged 6-11 years and elderly patients with atrial fibrillation (AF). Both groups were equipped with a smartwatch for the assessment of physical activity (using a heart rate monitor, pedometer, and accelerometer) and location (using GPS signals to locate individuals in indoor "at home" or outdoor microenvironments). The participants were required to wear the smartwatch equipped with a data collection application on a daily basis, and data were transmitted via a wireless network to a centrally administered data collection platform for the near real-time assessment of compliance. 
Over a period of 26 months, more than 250 children and 50 patients with AF participated in the aforementioned study. The main technical challenges identified included restricting access to standard smartwatch features, such as gaming, internet browser, camera, and audio recording applications, technical issues, such as loss of GPS signal, especially in indoor environments, and the internal smartwatch settings interfering with the data collection application.
The aim of this protocol is to demonstrate how the use of publicly available application lockers and device automation applications made it possible to address most of these challenges in a simple and cost-effective way. In addition, the inclusion of a Wi-Fi received signal strength indicator significantly improved indoor localization and largely minimized GPS signal misclassification. The implementation of these protocols during the roll-out of this intervention study in the spring of 2020 led to significantly improved results in terms of data completeness and data quality.

Commercial smartwatches equipped with wearable sensors are increasingly being used in population studies. However, their utility is often constrained by their limited battery duration, memory capacity, and data quality. This report provides examples of cost-effective solutions to real-life technical challenges encountered during studies involving asthmatic children and elderly cardiac patients.

Wearable sensors, which are often embedded in commercial smartwatches, allow for continuous and non-invasive health measurements and exposure assessments ...