Thursday, April 05, 2007

Voice Risk Analysis

The on line edition of the Guardian newspaper, Guardian Unlimited, a British publication had a story claiming that insurance benefit claimants are "to face lie detector tests" using Voice Risk Analysis technology. John Hutton, who is Secretary of State for Work and Pensions in the UK, is the person responsible for this introduction of this so-called fraud detection device. He says
"Our investigators are successfully using sophisticated 21st century techniques to stop criminals. The introduction of this cutting-edge technology will be another weapon in the battle against benefit fraud."
Use of phrases like "sophisticated," "21st century technology," and "cutting edge," must make all British citizens jump with joy at this resourceful high tech bureaucrat.

The article notes that insurance companies have been using this technology "for years" to weed out "fraudsters." Of course, if it has been used for years, that could mean it is not a 21st century technology but merely a 20th century technology and that it surely by now not "cutting edge." Actually, a little digging into the research on Voice Stress Analysis devices shows them to have very dull edge.

The fact is, first, that pieces of voice risk analysis equipment are not "lie detectors." They have much lower reliability than lie detectors and, as we all know, even lie detectors are not accepted in American courts as proving the truth or falsity of claims by defendants and witnesses. I don't know whether the Brits accept them or not.

I have searched for the most recent research on the accuracy of voice risk analysis machines and the most recent I have found and can get access to is a paper called "Evaluation of Voice Stress Analysis Technology," written by Hopkins, C.S., et al, published in Proceedings of the 38th Annual Hawaii International Conference in System Sciences, 2005. In this study, voice samples containing claims known to be true or false, were submitted to five different Voice Stress Analysis devices and five different device operators, three of whom had 26 or more years of law enforcement experience.

Brits will, I think, be unimpressed with their results for the strongest claim they could make is that these devices were more accurate than flipping a coin (my choice of language). I give you here the raw results of the study.
What we see here is that 198 true statements were coreectly determined to be true but 118 true statements were incorrectly determined to be false. We also see that 127 false statements were correctly determined to be false but 73 false statements were incorrectly determined to be true. In short, 37.3 per cent of the true statements were adjudged to be false and 36.5 per cent of the false statements were adjudged to be true. These five machines were therefore more or less equally good/bad at sorting out true and false statements. I wonder if the consistency of the devices in dealing with true and false statements will be comforting to citizens of the UK. Of course, we Americans are surely in much the same boat at least in the case of private insurers.

The authors of the paper note that polygraph and voice stress analyzers, are not “lie detectors,” contrary to the Guradian headline quoted above, "Benefit claimants to face lie detector tests," but then editors of on line and print journals do tend to prefer lively leads into stories to attract attention to strictly true ones. I illustrated this preference for the the lively in my blog Present Tense in Journalism.

I am less concerned with the "little white lies" of journalists than with the use and abuse of language by law enforcement and businesses. There are numerous ways in which this occurs ranging from judges forcing the use of transcripts of audio tapes to a jury's listening to the tapes themselves to "admissible" instances of the use of hearsay evidence. The problems with the latter is that there is overwhelming proof that people cannot recall any more than the gist of what they hear and that any given person's take on the gist of what someone has said will be highly unreliable due to the fact what what we take someone to have said reflects our contextual expectations in the context in which we hear it and the fact that memory for what people say and do is notoriously unreliable. As I noted in a footnote to my blog The Evolution of Language -- III that my wife once said something like "The grass has sprouted and I need to take the netting off," but I heard something like, "The grass has grown and I need to mow it," my error being due to what I had seen on letting her dog out while she was out of town. This may be an extreme case of mishearing but mishearing is as comonplace as speaking itself.

Tweet This!


Blogger Mr K said...

Thats quite a disturbing development- hadn't read that article. Actually an accuracy of 80% would probably not be good enough- 65% is really quite awful. When I would expect the majority of people being interviewed would be innocent, you end up with the majority of people accused actually being innocent without overwhelmingly accurate tests....

12:17 PM

Blogger The Language Guy said...

Yes, indeed, mr k, this is quite disturbing. Like you I would think that the majority of people making claims would do so honestly, some of them will be "detected" to be lying and some fraudsters will be "detected" to be telling the truth. That is quite a pickle.

Fraud is commonplace, unfortunately, so while a majority of people making claims are likely to be honest many too many won't. Something does need to be done about it. Whether the ministry has ways of investigating more deeply the results of their voice risk analyses I do not know. However, it behooves applicants to come with lots of documents supporting their claims. Surely the government has a file on corrupt doctors and other relevant sources of such documents.

10:00 AM

Blogger Dreamcatcher said...

You hit the nail on the head. "Lie Detectors" are actually nothing more than "stress detectors". If a person even thinks they're being accused of something, they will become stressed and physiologically and vocally respond exactly the same way as a person who is stressed because they are lying.

In this particular case, somebody who depends on these benefits could become very stressed thinking, "oh man, if they think I'm lying, I'm in big trouble." Tada! They show up as a liar.

12:45 PM


Post a Comment

<< Home