Criminology, probation and stuff

Some musings on criminology with a focus on probation

Posts Tagged ‘methodology

Switching off street lamps ‘could triple road deaths’ | Mail Online

leave a comment »

The Daily Mail has reported that Switching off street lamps ‘could triple road deaths’. As ever, I was sceptical from the outset so I decided to actually download the report which is available here . Firstly, just to clarify, the report is a meta-analysis of studies done by other people. The Cochrane Collaboration is vigorous in its selection criteria for including studies in their meta-analyses- even so there are methodological issues within the original studies. The first thing that struck me was this quote (page 2):

Authors’ conclusions
The results from this systematic review suggests that street lighting may prevent road traffic crashes, injuries and fatalities. However, further well designed studies are needed to determine the effectiveness of street lighting in middle and low-income countries.

This doesn’t quite concur with the Mail’s statement that:

Researchers say there is overwhelming evidence that street lamps save the lives of ‘a significant number’ of pedestrians, cyclists and motorists each year.

The report does not argue this and even if it did, there are numerous other major methodogical issues with the whole project which could undermine the results.

  • ‘the methodological quality of the trials was generally poor’ (page 1).
  • The research did not compare the number of fatal crashes with an area control (page 1) so we do not actually know if turning the lights off will increase the number of fatalities.
  • The risk of bias in these studies was judged to be high.
  • 11 of the studies used day time figures as a control so we are not actually comparing ‘night time no lights’ with ‘night time with lights’.
  • The other 5 studies used a seperate area as a control unit so we are actually comparing this area, with street lights, with another (and crucially different area) that has no street lights. to 10.03). Only one study with an area control specified an intention to match the control roads by speed limit, road type and police district
  • There were no randomised control trials in the analysis
  • None of the included studies had a data collection period of less than 12 months for either the before (range 1-3 years) or the after (range 1-8 years) periods,

I’m not going to continue, I think I’ve made my point. I am not criticising the meta-analysis itself- the authors are explicit in the poor quality of the whole exercise. The issue, as if it’s not obvious, is the media reporting of it. You will notice that that most of the first quotes in this post come from pages 1 and 2 of the report (ironically, page 2 is the ‘Plain Language Sumamary’). The Daily Mail author obviously didn’t get that far.

Note the quotation marks in the quote from the Mail above- ‘a significant number’. I think they must have fabricated that quote. If one searches for it in the report, it comes up with 0 (zero) occurrences. Look:

A Significant Number?

Not only has the Daily Mail not read the report but they’re making things up as well! Amazing!


Written by criminologyandstuff

January 21, 2009 at 4:50 pm

Posted in media, Research

Tagged with , ,