Fitness trackers score low in measuring burned calories

Fitness trackers might not be the way to go if you wish to know how many calories you’ve burnt. Instead, the devices can come in handy when measuring heart rate, scientists found after testing.

A team from Stanford University School of Medicine evaluated the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2 on a group of 60 diverse individuals in order to see how accurately the devices could measure energy expenditure and heart rate.

The participants, 31 women and 29 men, wore the seven devices while walking or running on treadmills or using stationary bicycles. Each volunteer’s heart was measured with a medical-grade electrocardiograph.

Metabolic rate was estimated with an instrument for measuring the oxygen and carbon dioxide in the breath, a good proxy for metabolism and energy expenditure. Results from the wearable devices were then compared to the measurements from the two “gold standard” instruments.

The results were surprising as the scientists expected the opposite outcome.

“The heart rate measurements performed far better than we expected,” said Euan Ashley, professor of cardiovascular medicine, of genetics and of biomedical data science at Stanford. “But the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me.”

While most of the trackers scored high when it comes to measuring heart rate, having an error rate of less than 5 percent, most of them failed to accurately predict energy expenditure. Even the most accurate device was off by an average of 27 percent. And the least accurate was off by 93 percent.

“People are basing life decisions on the data provided by these devices,” said Ashley.”But consumer devices aren’t held to the same standards as medical-grade devices, and it’s hard for doctors to know what to make of heart-rate data and other data from a patient’s wearable device, he said.

The scientists published their findings in the Journal of Personalized Medicine saying that it is an independent evaluation of the performances of some of the most popular fitness trackers on the market.

Euan Ashley said that in part, their research was done in order to provide customers and doctors alike with more accurate information since companies do test the devices but it is hard for buyers to know exactly how well they perform and what kinds of test the devices were subjected to.

And while fitness trackers could prove reliable in measuring heart rates, scientists argue that the discrepancy between the device measured and the real energy expenditure could be due to mathematical algorithms that make assumptions that don’t fit individuals.

“All we can do is see how the devices perform against the gold-standard clinical measures,” said graduate student Anna Shcherbina, co-author of the study. “My take on this is that it’s very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone’s fitness level, height and weight, etc. Heart rate is measured directly, whereas energy expenditure must be measured indirectly through proxy calculations,” she explained.

Scientists are now getting ready for phase two of the study in which they want to track the devices’ performances in non-laboratory settings. Volunteers will have to wear the fitness trackers while going about their days, including exercising and walking in the open-air. And they will also be provided with portable EEG machines.

The conclusion of the study is that people should not base their food-eating decisions on the calculations made by fitness trackers but rather consult a medical professional. Also, doctors can take heart rate data provided by these devices into the calculation as they seem to be rather reliable.