Is Your Tech Working for You? Accuracy of Activity Trackers

I have never used an activity tracker. I’ve never seen the need for one because I incorporate physical activity into my daily life: I walk to work, have a standing desk, take lots of walk breaks, and workout four to five times a week. I avoid driving as much as possible – I have a 2004 Toyota Echo that I bought new and I have driven only 80 000 km over the past 11 years – and walk or ride my bike to run most errands. The two questions I ask myself when I have to go someplace are: can I walk there or can I ride there?

It seems very simple to me, but I know it’s not. I know there are a lot of different barriers to physical activity from socioeconomic barriers to how we build and design our cities. There have been and are a lot of different strategies employed to get people around or over those barriers. With the recent boom in wearable tech, monitoring physical activity levels has become the next gen of barrier hopping. My question is, “How accurate are those fancy bracelets?” Luckily, two recent studies answer my question.

Let’s start with the study that looked at a wide range of activity trackers. It was published in International Journal of Behavioral Nutrition and Physical Activity, by a research team from Australia. Ty Ferguson led the team and they looked at the effectiveness of seven consumer-level activity trackers in healthy adults: Fitbit One, Fitbit Zip, Jawbone UP, Misfit Shine, Nike Fuelband, Striiv Smart Pedometer, and Withings Pulse. As you can see, I’ve provided links for all the monitors so you can go over each one’s specs.

To determine how accurate the activity trackers measured physical activity and sleep, the research team compared them to research-grade accelerometers.

Ferguson and his team recruited 21 healthy adults with an average age of 33 years and 52 percent of them were female. Participants were recruited using a technique called “convenience sampling.” Convenience sampling can be problematic because the researchers recruit people who are, well, convenient. That is to say, people they have easy access to like university students or patients at a clinic. The problem with this type of sampling is that it might not represent an entire population; therefore, your results might be biased towards a small group of people who are close to the same age, from a similar ethnic group, or share the same gender. Put another way, the results might not apply to everyone. Furthermore, having a small sample size, like 21 participants, can also create a similar problem or compound the problem when combined with convenience sampling.

As for Ferguson’s research, the focus is on testing different devices and not observing how people respond to using the devices; therefore, convenience sampling and sample size might not have influenced their results, but it’s important to know how people have been selected for a study so you can judge how applicable the results are to larger populations or even yourself.

Before beginning the study, participants were instructed in the use of their trackers and told to wear them for 48 hours. After the 48-hour period, the trackers were collected and the data downloaded. The participants were then given a different tracker and repeated the same routine until they had used all the activity trackers. The study took place over a two-month period.

The results showed that all the consumer-level trackers performed very close to research-grade monitors when measuring steps; however, three monitors slightly over-counted steps (Srtiiv, Zip, and One) and four under counted (Fuelband, Shine, Up, and Pulse). The research team noted that the Fuelband had an average step under count of 2 529 steps per day (26 percent).

When it came to measuring moderate to vigorous physical activity (MVPA), their performance ranged from weak to strong when compared to the research-grade monitors. The Shine underestimated MVPA by 15 minutes a day (26 percent) and the Striiv overestimated by 190 minutes a day (325 percent). The research team attributed this large discrepancy between trackers to how each manufacturer defined and measured physical activity. Several of the devices measured a variety of physical activity variables, but none of them were explicitly defined as MVPA. Therefore, do your research and try to purchase a tracker that best matches your physical activity goals.

Five of the trackers measured total daily energy expenditure (TDEE): Shine, UP, Pulse, Zip, and One. The trackers performed moderate to strong in this category with all five underestimating TDEE anywhere from 477 kcal (One) to 898 kcal (UP). The researchers found that the two Fitbit devices (One and Zip) outperformed the Up when compared to the research-grade monitors.

Finally, sleep. Four of the trackers measured sleep: Shine, Up, Pulse, and One. All the trackers performed very well when compared to the research-grade monitor. However, they all overestimated sleep, but not as much as the Shine with an average of 44 minutes.

The researchers argue that although all the devices performed in the moderate to strong range when compared to research-grade devices, their inaccuracies could become frustrating for some consumers. Specifically, those relying on the trackers to accurately measure TDEE so they can balance their energy expenditure with energy intake. Keeping this in mind, Ferguson and his team found that the Pulse and Fibit One and Zip were the strongest performers. Furthermore, the researchers did not find that performance matched price with the Fuelband being the most expensive tracker and weakest performer, and the Zip the least expensive tracker and one of the best performers.

Study two, a very short write-up, looked at the step counting and energy expenditure accuracy of Fitbit One and Fitbit Flex. The study was published in the International Journal of Cardiology by a research team led by Keith Diaz.

The research team recruited 23 healthy adult participants between the ages of 20 and 54 years (57 percent were female).

The participants completed a four-stage treadmill test while wearing the two types of Fitbits. Their energy expenditure was measured using gas exchange calorimetry and the number of steps taken by manual counting of steps using a video recording of the treadmill test.

The research results showed that Fitbit One and Fitbit Flex reliably estimated energy expenditure and step counts during walking and running activities.

Just to be clear about both studies, neither of them had conflicts of interest. To be clearer, neither of the studies received money from the companies of the products they researched.

Now you have a bit of research to guide you in your tech buying endeavours. If you can’t afford any of this wearable tech, you can always use the old school methods of journaling, mapping out a route, or even counting your steps per minute. Google has made the mapping a lot easier than the old, old school method of string and a physical map, if you want to see how far you’ve run or walked. You can also drive a route to figure out the distance. Or you can just step outside and start walking and don’t come back for 20, 30, or 60 minutes. It’s simple.

Rodney Steadman 07 July 2015

Works Cited

Diaz K, Krupka D, Chang M, Peacock J, Ma Y, Goldsmith J, Schwartz J, & Davidson K (2015). Fitbit®: An accurate and reliable device for wireless physical activity tracking. International Journal of Cardiology, 185, 138-140 DOI: 10.1016/j.ijcard.2015.03.038

Ferguson T, Rowlands A, Olds T, & Maher C (2015). The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. International Journal of Behavioral Nutrition and Physical Activity, 12 (1) DOI: 10.1186/s12966-015-0201-9

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s