Off On the Wrong Foot?

Putting fitness trackers and health apps to the accuracy test

When it comes to exercise, some things never change. Running, biking or swimming a mile is, well, still a mile, as it’s always been. What’s changed, however, are the vast improvements in technology related to how you might track that mile. Wearable fitness trackers, and smartphone and tablet health and exercise apps are all the rage, and users track and share progress to goals and hold the data as gospel. But are these technologies really as accurate as we think they are?

"People buy these activity monitors assuming they work, but some of them are not that accurate or have never been tested before," said Greg Welk, a kinesiology professor at Iowa State University in Ames.1

In technical terms, fitness trackers—sometimes referred to as intelligent electronic devices (IED)—are accurate, sensitive and reproducible only if they can provide information on the variable being measured, record the output and help aid in controlling the process.2

Things don’t always function as expected, however.

Some fitness trackers, for instance, measure the distance you walk, the number of steps you climb and the calories you burn through a pair of green, light-emitting diodes that monitor the levels of reflected light from your skin and measure your heart rate.3

Ambient light and the movement of your muscles, tendons and capillaries can sometimes interfere with measurements, however. To get a more accurate result, heart rate monitors require chest straps that measure the heart’s electrical activity. Or, they can require users to place their fingers over a sensor.4 Sensors worn on wrists appear to be less accurate than chest-strap models, according to a recent test by Jon Zaroff, a cardiologist at Kaiser Permanente Medical Center in San Francisco.5

A less invasive way to measure calories burned is through skin resistance (sweat) and temperature sensors, which measure body heat and sweat quantity to accurately calculate energy expenditure.6

An accelerometer is another tracker that measures steps and other movements using algorithms to translate readings into measurements such as distance traveled or the number of "very active minutes." Accelerometers also include a built-in altimeter that allows them to gauge how hard you’re working by factoring in stair and hill climbs.7

One study suggested the accuracy of accelerometers’ measurements of calorie expenditure can vary by model. Other researchers say the devices work better in tracking strenuous workouts, but may underestimate less intense activity.8

"At higher intensities of exercise, it went off a little bit, but it was still reasonable," said Matthew Lee, an exercise physiologist and professor at San Francisco State University. "The accuracy of such devices will vary from person to person. For example, a device may not fit as well if someone has thinner arms, which may affect the device’s readings."9

Power meters calculate how much energy your body produces by measuring the force the foot exerts on a bicycle pedal and how long that force is applied. Runners can use a footpod attached to their shoelaces to track the precise number of steps and calibrate stride sensors on a track. Bicyclists can use a RPM sensor for the wheel and a cadence sensor for the pedals, which can be calibrated by using the wheel RPM sensor.10

But don’t assume the data doled out by the device are always dead-on.

"The individual needs to be smarter than just the device … to be able to see when those numbers may not be correct," said Clinton Brawner, a clinical exercise physiologist at Henry Ford Hospital in Detroit.

For a simple test of a device that monitors pulse rate, Brawner recommended checking your pulse on the underside of your wrist, counting the beats for about 15 seconds and multiplying the number by four. From there, you can see how this number compares to the reading the device gives.11

Medical apps

In addition to the fitness trackers being worn on wrists or hitched to waists, more people are downloading apps on smartphones and tablets to manage their health and promote healthy lifestyles. A recent study found one-third of physicians have recommended apps to patients to:

  • Improve the patient-physician relationship by promoting shared health management.
  • Connect patients with peer and support communities online to more actively engage them in their own care.
  • Assist caregivers in following physician instructions.
  • Provide useful data to patients who are self-monitoring (for example, a diabetic patient watching blood glucose levels) and physicians keeping tabs on patients with uncontrolled chronic conditions.12

Keith Wick, a 29-year-old information systems technician for the state of California, manages his Type 1 diabetes on his Android smartphone—tracking his blood sugar levels by entering data six times a day onto a virtual spreadsheet. "I’m on my phone all day, and I find it easier to keep track in a digital form," he said.13

Those in the medical field are using smartphones and tablets more to diagnose, predict and treat medical conditions by quickly accessing data and providing patient-specific recommendations.14 Surveys show 90% of physicians use smartphones or tablets in their day-to-day professional activities.15 Apps can provide reference to:

  • Staging systems, such as severity staging of chronic obstructive pulmonary disease.
  • Published decision-making algorithms that perform minimal calculations considered at low risk for causing errors.
  • Calculations that determine the severity of liver disease, the likelihood of having a pulmonary embolism and risk stratification in acute coronary syndrome.16

There’s a subset of these health-related apps that claim to function as actual medical devices, tracking blood pressure, treating acne and even testing urine samples. Physicians and federal regulators warn that these programs claiming to diagnose or treat medical conditions may be unreliable and even dangerous.17

For example, one app claims to give a smartphone the ability to measure blood pressure. An emergency room physician assessed the app and found it gives inaccurate readings.18

"There’s just no plausible medical way that some of these apps could work," said Nathan Cortez, an expert in medical technology law and regulation at Southern Methodist University’s law school in Dallas. "Besides wasting your money, these apps may actually do harm. If you’re diabetic and your app is misreading your blood glucose levels, you may give yourself more insulin than you need and go into diabetic shock. Patients rely on these apps when they should be seeking real medical advice," Cortez said.19

The U.S. Food and Drug Administration (FDA) has tried to eliminate the distribution of misleading healthcare-related apps being used in clinical settings because the scoring results can affect a clinician’s decision.20

In 2013, for instance, the FDA ruled it will regulate only apps that function as medical devices, transform a device into a medical device or perform patient-specific analysis and then provide a diagnosis or treatment on the basis of it.21 These included an app-device combination that transforms a smartphone into an ECG rhythm-strip recorder, an ultrasonography machine, a blood pressure cuff or a pulse oximeter.22

Of the estimated 26,000 healthcare apps—approximately 7,400 of which are aimed at physicians—a vast majority are not regulated by the FDA.23

A Journal of Medical Internet Research study found most medical calculator smartphone apps are accurate and can be used in clinical settings, but there is still a need to verify medical apps before using them in patient care because of findings that show clinically significant errors in some smartphone calculations.24

Given that using smartphones as medical calculators makes them a medical device, a system to verify smartphone calculation accuracy would be useful to reduce the chance of errors affecting patient care.25

The most reliable medical apps tend to result when developers, physicians and experts in health law collaborate. For example, HemMobile is a Pfizer app that helps hemophilia patients track their infusions.26 Every aspect of the HemMobile program, from display to medical terminology, required a Pfizer review committee’s approval.27

Iltifat Husain, founder and editor-in-chief of iMedicalApps, a review site, encourages people to read the entire app description before buying. The best course of action, however, is to speak with a doctor about apps that may aid treatment.28

—compiled by Chelsea M.S. Kolterjahn, contributing editor


  1. Darren Yates, "I Heart New-Gen Fitness Bands," Tech Life Australia, Feb. 18, 2015, www.techlife.net/2015/02/i-heart-new-gen-fitness-bands.html.
  2. Electrical4u.com, "Electrical Measuring Instruments: Types Accuracy Precision Resolution Speed," 2011-2015, http://electrical4u.com/electrical-measuring-instruments-types-accuracy-precision-resolution-speed.
  3. Rachael Rettner, "How Well Do Fitness Trackers Monitor Heart Rate?" Live Science, March 18, 2014, http://tinyurl.com/live-science-tracker.
  4. Shira Springer, "Fitness Tracking Devices Can Hurt Rather Than Help," Boston Globe, Feb. 6, 2015, http://tinyurl.com/globe-fitness.
  5. ScoobysWorkshop.com, "Review of Fitness Performance Monitors: Wearables, Fitness Bands and Activity Trackers," 2015, http://tinyurl.com/scoobys-tracker.
  6. Yates, "I Heart New-Gen Fitness Bands," see reference 1.
  7. Springer, "Fitness Tracking Devices Can Hurt Rather Than Help," see reference 4.
  8. Rettner, "How Well Do Fitness Trackers Monitor Heart Rate?" see reference 3.
  9. ScoobysWorkshop.com, "Review of Fitness Performance Monitors: Wearables, Fitness Bands and Activity Trackers," see reference 5.
  10. Rettner, "How Well Do Fitness Trackers Monitor Heart Rate?" see reference 3.
  11. ScoobysWorkshop.com, "Review of Fitness Performance Monitors: Wearables, Fitness Bands and Activity Trackers," see reference 5.
  12. David Lee Scher, M.D., "The Big Problem With Mobile Health Apps," Medscape Business of Medicine, March 4, 2015, www.medscape.com/viewarticle/840335.
  13. Joshua A. Krisch, "Questioning the Value of Health Apps," New York Times, March 16, 2015, http://tinyurl.com/nyt-health-apps.
  14. Rachel Bierbrier, Vivian Lo and Robert C. Wu, "Evaluation of the Accuracy of Smartphone Medical Calculation Apps," Journal of Medical Internet Research, March 2, 2014, www.jmir.org/2014/2/e32.
  15. Scher, "The Big Problem With Mobile Health Apps," see reference 12.
  16. Bierbrier, "Evaluation of the Accuracy of Smartphone Medical Calculation Apps", see reference 14.
  17. Ibid.
  18. Scher, "The Big Problem With Mobile Health Apps," see reference 12.
  19. Krisch, "Questioning the Value of Health Apps," see reference 13.
  20. Bierbrier, "Evaluation of the Accuracy of Smartphone Medical Calculation Apps", see reference 14.
  21. Scher, "The Big Problem With Mobile Health Apps", see reference 12.
  22. Ibid.
  23. Ibid.
  24. Bierbrier, "Evaluation of the Accuracy of Smartphone Medical Calculation Apps", see reference 14.
  25. Ibid.
  26. Krisch, "Questioning the Value of Health Apps," see reference 13.
  27. Ibid.
  28. Ibid.

Who’s Who in Q

NAME: Charles E. Holman.


EDUCATION: Master’s degree in project management from Embry-Riddle Aeronautical University in Daytona Beach, FL.

INTRODUCTION TO QUALITY: Holman’s formal quality training and education occurred 11 years into his U.S. Air Force career. While assigned to a precision measurement equipment laboratory (PMEL), Holman’s team was challenged to increase production and the quality of equipment while decreasing resource waste. After learning some quality fundamentals and what a quality organization is, his team was able to effectively realign dated management philosophies into a future-focused process quality culture.

CURRENT JOB: Holman is an active-duty Air Force Sergeant serving as an Air Force metrology laboratory lead evaluator in the Laboratory Certification Branch, Air Force Metrology and Calibration Division, Agile Combat Support Directorate, Air Force Life Cycle Management Center, located in Heath, OH.

PREVIOUS JOBS: Holman joined the Air Force in 1997 and has held several positions, including PMEL calibration technician, PMEL tactical air navigation calibration work area supervisor, PMEL microwave frequency calibration work area supervisor, compatible development systems analyst noncommissioned officer in charge (or work area supervisor), PMEL electronics section calibration work area supervisor and PMEL quality program manager.

ASQ ACTIVITIES: Holman is a member of ASQ’s Columbus, OH, section and is active in the ASQ Measurement Quality Division, serving as internet liaison; ASQ-certified calibration technician.

OTHER ACTIVITIES/ACHIEVEMENTS: Project Management Institute-certified associate in project management, Community College of the Air Force-certified professional manager and Kaplan University-certified Six Sigma Black Belt.

PUBLISHED WORKS: Articles for QP and the U.S. Air Force metrology and calibration newsletter.

RECENT HONORS: ASQ Inspection Division’s 2015 Chuck Carter International Inspector of the Year Award.

PERSONAL: Married to Oxana, with one son, Eddie.

FAVORITE WAYS TO RELAX: Spending time with the family and the occasional vacation.

QUALITY QUOTE: "You have to quantify to qualify." Holman often found himself constantly reinforcing to his teams that without data or supporting evidence, any and all statements, assumptions and conclusions can be challenged.

ASQ News

FUTURE REPORT ASQ’s latest Future of Quality study will be released this month. The report, titled "Quality Throughout," features prominent thought leaders examining the future of several trend areas impacting quality, including leadership adaptability, internet security and openness, and smart manufacturing. ASQ has published six "Future of Quality" reports since 1996. Authors of the seventh report include: Izabel Christina Cotta Matte, chief strategic planning and budget officer of Porto Alegre in Brazil; Jonathan Zittrain, author of The Future of the Internet—and How to Stop It; Greg Watson, author of Corporate Governance: Quality at the Top; and Noriaki Kano, author of Guide to TQM in Service Industries. Visit www.asq.org/futureofquality to access the report.

PDSA TRAINING ASQ has unveiled a new web-based course centered on the plan-do-study-act (PDSA) problem-solving model. "Plan-Do-Study-Act Plus QTools" guides users through the PDSA model and introduces seven quality improvement tools. Geared toward those new to quality, the interactive, self-paced, five-hour course is divided into 30-minute modules. The course costs $159 for ASQ members and $229 for nonmembers. For more information, visit http://tinyurl.com/pdsa-plus.

LEARNING LEAN The Lean Enterprise Division (LED) launched a lean education and training program. The LED Learning Series teaches lean practices and can help those who want to become lean certified. For more information and to access a free training module, "Introduction to Lean," visit http://asqled.qualitycampus.com/cloud/index.asp and click the "welcome" button.

NEW CASE STUDIES ASQ’s Knowledge Center has released two new case studies. The first one tells how a Polish dialysis center used visual management techniques to increase safety and improve how the medical facility was organized. Read the case study at http://tinyurl.com/polish-dialysis-center. The second study features the Chicago Public Schools and the steps it took to cut waste and save money in its contract procurement process. To read that case study, visit http://tinyurl.com/chicago-schools-case-study.

TEAMS WORK Forty teams from 13 countries participated as finalists in this year’s International Team Excellence Award Process at ASQ’s World Conference on Quality and Improvement this month. The teams were part of the quality impact sessions and live team case study presentations at the event held in Nashville. Watch for more coverage of the winners and a wrap-up of this year’s conference in an upcoming issue of QP.

MEDAL NOMINATIONS ASQ is seeking nominations for its Feigenbaum Medal, which recognizes quality professionals 35 years old or younger who display leadership, professionalism and potential in quality. The medal is named for Armand V. Feigenbaum, a quality pioneer who died late last year. Nominations are due Oct. 1. For more details and to download a nomination form, visit http://asq.org/about-asq/awards/feigenbaum.html.

ASQ Journal Spotlight

QP occasionally highlights an open-access article from one of ASQ’s seven other journals.

This month, read "Does Your Organizational Culture Support Your Business Strategy?" which appeared in the January edition of the Journal of Quality and Participation (JQP).

Authors Dave Eaton and Gabriella Kilby write about the importance of aligning culture to support strategy as an investment that organizations must make.

To access the five-page article in PDF format, click on the "Current Issue" link on JQP’s webpage: http://asq.org/pub/jqp. From there, you also can find a link to information about subscribing to the quarterly publication.

By the Numbers


The number of certifications ASQ issued last year as part of its partnership with the Manufacturing Institute, which is designed to help fill advanced manufacturing positions. The Manufacturing Institute and its partners set a goal of issuing 500,000 industry-based certifications to individuals by 2016. In 2014, the institute and its partners issued 129,084 credentials, for a total of 419,528 certifications since the program’s inception in 2011. For more information about the Manufacturing Skills Certification System, visit www.themanufacturinginstitute.org.

Average Rating


Out of 0 Ratings
Rate this article

Add Comments

View comments
Comments FAQ

Featured advertisers