The ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing) program has become the standard for computerized concussion evaluation. Hundreds of thousands of athletes across the country will take the test to create a cognitive baseline, especially during the fall as school sports begin.
The program is used to diagnose concussions and provides a guideline for how quickly to allow athletes to return to play. However, an in-depth story in The Bulletin of Bend, Ore. questioned the validity of ImPACT testing.
But a number of independent studies have found the test gives inconsistent results, has high error rates and can be easily misled by neurological factors unrelated to concussions.
Officials with The Center Foundation, which provides the tests for free to Central Oregon athletes in high-risk sports, stress that ImPACT is just one of many tools used to diagnose and manage concussions, and not the sole determinant of whether an athlete can return from a concussion. Yet, it remains the only one of those tests that student athletes in the region are required to take in order to play.
"There's not a lot of research that has shown conclusively that ImPACT assessment, in and of itself, adds significant benefit to the other things that one could assess in a face-to-face setting with a physician," said Dr. Richard Wennberg, a neurologist at the Krembil Neuroscience Centre of Toronto Western Hospital in Canada.
The Bend, Ore., Center was one of the first facilities to adopt the ImPACT program in 2001 after it was developed two years earlier by University of Pittsburgh Medical Center researchers. The Center study said problems with ImPACT have arisen in recent years when testing was showing the opposite of what clinicians were observing.
Steven Broglio, director of the NeuroSport Research Laboratory at the University of Michigan, said most athletes with concussions experience a big dip in cognitive function immediately after the concussion, but then make steady progress toward recovery. Each individual may progress at a different rate, but generally patients continue to improve unless they return to play too early or engage in other types of discouraged activities.
While at the University of Illinois, Broglio and his colleagues were regularly seeing athletes who improved on ImPACT scores a week after their concussion, but when retested four or five days later, were down on some measures and up on others. Their scores seemed to rise and fall with no rhyme or reason.
"It just didn't make sense to us clinically and that was actually what prompted our investigation," Broglio said.
In 2007, they conducted baseline tests with more than 100 students, using a variety of computer-based neurocognitive tests, including ImPACT. They tested the same students 45 and 50 days later. None of the students had incurred a concussion during the test period, yet the retest results showed little consistency with the baseline tests. On a scale from 0 to 1, where 0 reflects no correlation and 1 perfect correlation, the five scores generated by ImPACT at baseline and retest had correlation ranging from 0.15 to 0.39. An acceptable cut-off for a useful test, according to the researchers, is about 0.75.
Too often the ImPACT incorrectly gave scores consistent with having a concussion.
"Keep in mind these are all healthy college students," Broglio said, "and we had somewhere in the neighborhood of a 40 percent false-positive rate."
Now some researchers are questioning the use of the ImPACT program. And baseline testing is being called into question. ImPACT developers counter that their tests are being held to a higher standard.
"No test is perfectly reliable," said Dr. Mark Lovell, who co-developed ImPACT and until recently served as CEO of ImPACT Applications, the company that markets the test. "Many of the tests â blood pressure, heart rate, EEG -- that neurologists use are remarkably unreliable, and we accept that because they're the best tools we have."
Lovell believes part of the criticism of ImPACT stems from the misuse of the test as a stand-alone product to diagnose concussions and make return-to-play decisions.
"I need to emphasize this, and I have always said this: I don't think that ImPACT or any other test should be used in a vacuum," Lovell said. "It shouldn't be looked at to say, 'He's good to go or he's not good to go.' We've never said that."
-- Bill Bradley, contributing editor