Correlation Conundrum

Correlation and causation are tricky subjects. When two things vary with each other, we usually infer a connection. Suicides rates for example, are associated with unemployment rates. But correlations can also be quite spurious. A researcher named Tyler Vigen who tracks spurious patterns for fun showed, for instance, that the consumption of mozzarella cheese varies the award of engineering PhDs.

Sometimes, the correlation is genuine but researchers infer the wrong relationship. For example, malaria literally means "bad air". The disease is prevalent in swampy areas. Swamps are ideal breeding grounds for mosquitoes so there is indeed a relationship. But swamp air also smells of rotten vegetation and stagnant water. The disease was assumed to be caused by bad air.

Some of the finest epidemiological work has involved sorting through data and working out correlations. One of the pioneers of this sort of healthcare detective work was Dr John Snow (1813-1858) of London. Snow was a society physician. He administered chloroform to Queen Victoria to ease her through childbirth. He was considered a crank when he wrote a monograph claiming cholera was a water-borne disease. Other doctors thought it was an air-borne disease.

Cholera had been imported to the UK on the East India Company's ships. By the 1830s, there were regular outbreaks of cholera in London, especially near the port areas. Municipal improvements actually led to higher incidence.

Cholera is caused by bacteria (vibro cholerae), which lives in contaminated food and water. In the 1840s, London started putting together a sewage system. Houses were connected to municipal sewage lines, instead of using the old system of local septic tanks and cesspits. Untreated sewage flowed out into the River Thames which also supplied drinking water.

Two companies supplied drinking water to London. In many places, they supplied water to the same streets. In 1850, a biologist , Arthur Hassell, showed that both supplies were heavily contaminated by sewage. He published a study in Lancet which led to a clean water act in 1852. The water companies were given three years (until 1855) to start pumping cleaner water from upstream.

One company, The Lambeth Water Company, moved its pumping station upstream almost immediately. The other, The Southwark and Vauxhall Water Company, only shifted upstream in 1855. In 1854, there was another cholera outbreak at a time when S&V was supplying polluted water while Lambeth supplied cleaner water.

Snow crunched the records for cholera deaths. Lambeth had supplied water to 26,107 houses where there were 98 deaths, S&V had supplied water to 40,046 houses with 1,263 deaths. In other parts of London, where the water supply was from local pumps and other sources, there had been 1,422 deaths in 256, 423 houses.

Lambeth had the lowest death rate with about 37 deaths per 10,000 houses. Places with local supply had 59 deaths/10,000 houses, and S&V had 315 deaths/ 10,000. The difference between S&V and Lambeth was striking. S&V admittedly supplied more contaminated water and its customers had almost 10 times as many deaths. 

Snow lived in Soho, which was not connected to either the sewers, or water supply systems. There had been 616 deaths. He drew a map, where he filled in the locations of 13 local water pumps, and houses where the deaths had occurred.

There was a huge cluster around one particular pump, at the intersection of Broad Street and Cambridge Street. As many as 578 of the deaths had occurred within 250 metres of that pump. In his obsessive fashion, Snow visited the families of cholera victims who lived elsewhere in Soho and discovered that many had also worked in the Broad Street area and probably drank from that pump.

The clinching evidence for him was the presence of a local brewery on Broad St. The workers there drank only the beer they brewed (!), drawing water from their own private pump. None of them had fallen ill. (The brewing process kills cholera bacteria.) Snow managed to get the Broad Street pump shut down. The Soho epidemic stopped. Much later, it was learnt that the water there had been contaminated by leakage from a nearby cess pit, where a cholera victim's clothes had been washed.

Snow died in 1858 before his theory was fully accepted. London started building a modern sewage system soon after. As sewage contamination eased off, so did cholera epidemics. In 1883, Robert Koch had isolated and identified the bacteria.

Snow's meticulous maps of Broad Street and his transcription of municipal records of deaths, with the addresses cross referenced to the water supply, are works of art. He did it all by hand, in the teeth of ridicule, at a time when the concept of a data scientist did not even exist.