Unfortunately, the promise of Big Data was heralded without understanding its two inherent flaws:

1. Firstly, that it is big. Very big. So, while the data from a global bank might contain major and earth enriching insights, extracting them from the noise of data is very hard.
2. Second, it is data, and data without any context is often worse than useless as it misleads where it should enlighten. A false correlation can lead a CEO to make a completely wrong acquisition, a misunderstood causation can lead a political party into a voter meltdown.

Big data can be actively dangerous as it misleads the recipient to believe in a correlation that might be utterly false but because of the overwhelming size of the data it is judged to be unquestionably true.

In fact, scientists have outlined six different types of bad correlations:

• Fluke correlations: One of the bedrock truths of statistics is that, given enough trials, almost any possible occurrence can happen, though you might need to wait a long, long time. The more possible events that might take place, the more potential, albeit unlikely, "fluke" events there are.
• Ephemeral correlations: Some extreme correlations may jump right out at us and scream "Significant!" only to fade upon repeated observations. Though they may not be statistical flukes, such correlations may vanish under the influence of proper causation analysis.
• Uncorroborated correlations: Statistical correlations that are based purely on number crunching are weak. The arrival of storks and babies in Holland may have seen a correlation, but there is absolutely no causation despite subsequent myths told to small children.
• Artifactual correlations: Some correlations may be produced by "artifacts" that are entirely separate from "natural" factors being studied. Artifacts might dilute, distort or reverse any "natural” correlations being studied statistically.
• Wrongheaded correlations: Some correlations proceed from metrics and data that should never have been developed in the first place. This happens a lot in pseudo-science when someone attaches numbers to fundamentally qualitative phenomena, or attaches the wrong numbers to a quantitative phenomenon.
• Hyped correlations: Hype can infect the narrative that frames any correlation, no matter how well that correlation is grounded in the subject domain, the data and the statistical models.

Read Less

Causal Analytics is a part of Etiology; the science of why things occur. While this is an old science it is at the forefront of computer science as computers transition from decision-support to decision-making across many fields in our society.

Computer science is now at the beginning of a Causal Analytics revolution where the ability to record huge quantities of data while varying conditions through automated experimentation can provide unimaginable insights into the cause of events. Leading technology firms like Google, Amazon and Netflix make extensive use of Causal Analytics to understand human behaviour, and in the case of Google’s DeepMind, to reinvent the game of chess at such a profound level that it is almost beyond human understanding. Causal analytics has also been a powerful force in customer preference testing where it enables companies to accelerate decision making in product and interface design.

Headlamp applies its Casual Analytics technology for consumer preference testing, among other applications, in providing consulting services for the creation and revision of websites to optimize customer engagement and in related technical services, such as troubleshooting services and software enhancements for a variety of computer hardware and software platforms.

In economics, statistics are often subjected to regression testing to establish Causal relationships between decisions and outcomes. Without the ability to subject different population samples to different circumstances to prove causation, the science of economics has been troubled by false correlations that seemingly show causality, but are not in truth related.

But Causal Analytics is not new; in medicine it is also known as epidemiology; an approach that in the 1950s showed through a study of London bus drivers versus bus conductors that exercise is good for you (the drivers die young), and proved that smoking causes lung cancer (yes, this was once in doubt; plastics and cars were potentially thought to be the cause).

More recently, Causal Analytics in medicine has been used in molecular epidemiology, where the Causal inferences of molecular experiments are being used to create personalized medical treatments for everything from the cure of cancers to the regrowing of bone cartilage.

Read Less

The inference engine is a component of the Causal Analytics system that applies logical rules to the knowledge base to deduce new information. In practice, data is collected from a source (say the US electrical grid) and stored in a knowledge base. From there the inference engine is used to test different hypotheses against the data and deduce new knowledge. Going further, the inference engine can then determine new hypotheses to test against, based on the new knowledge. This recursive approach can lead to an automated analysis of data against a huge number of hypotheses and variables.

Inference engines work primarily in one of two modes; either forward chaining and backward chaining. Forward chaining starts with the known facts and asserts new facts. Backward chaining starts with goals, and works backward to determine what facts must be asserted so that the goals can be achieved.

In Causal Analytics, the inference engine is used not just to test different hypotheses against existing datasets, but through the new knowledge gained from the data to establish new lines of experimentation and testing that will create a fresh wave of data. This recursive experimentation is used to gain highly reliable insights into the causation of specific actions.

Read Less

1. The availability of sufficient data pools
2. The on-line environment where real-time experimentation can occur
3. The access to sufficient computing power to conduct the real-time Analytics and formulate new experimental waves.

Causal computing as a branch of probability mathematics has its own mathematical notation that was created by Judea Pearl, the godfather of Causal Analytics.

Download a white paper on the statistical model and mathematics behind Causal Analytics: Bayesian Causality

Read Less

An example would be a search engine optimisation firm whose fees are aligned to the number of web orders received by its client, or a company that rewards its research scientist for the success of products that they have invented.

Gainsharing’s goal is to improve performance across whatever factors are stipulated; increase in revenue, higher profitability, greater innovation, etc.

Gainshare in Causal Analytics works through the gains that organisations make in their key metrics through the adoption software such as that developed by Headlamp Software. For example, an e-commerce company may find that its sales rate for sports products increases 82% per annum through the use of Causal Analytics. A percentage of the resultant extra profit of $22 million is then paid as the fee for the software, with no other licensing fees required.

Read Less

In December 2017 the world changed profoundly when Deepmind’s AlphaZero beat Stockfish. This game between two chess computers could easily have been of no consequence, but these were two very different chess computers.

Stockfish was then the strongest open-source chess engine in the world and a traditionally programmed chess computer that relies on human defined algorithms to decide the next move – although admittedly thinking many moves ahead of what the average human chess player can do. AlphaZero is a Causal Analytics system that has never been programmed with a single chess strategy, just the rules of the game; the rest it has learned through iterative experimentation by playing 100,000 games a day against itself.

The result of the challenge was a clear win to Alpha Zero; in a 100-game match it won 28 games, drew 72 draws, and had zero losses. Perhaps more importantly, along the way to its victory it completely redefined the accepted strategy of chess – what openings to make, what strategies to play – and won by just simply having worked out what truly works best rather what we as humans think works best.

When you extrapolate that concept across society it utterly changes our understanding and reliance on computers. The role of computers will no longer be digital assistants to help humans in their deliberations, but the determinants of strategy through a Causal Analytics algorithm.

What strategy to follow in the finding of a stem cell treatment for Parkinson’s disease? What strategy to employ for an e-commerce site? What approach to the optimisation of the power grid? The answer no longer lies in the domain of human intelligence and experience but in the power of Causal Analytics.

Read Less

Ecommerce

When the world’s leading ecommerce site wants to know how to restyle a page or add additional offerings, they decide what to do through Causal Analytics. The positioning of a photo can change buying behavior by 0.25%. The color of a button can increase the chance of being clicked by 0.5%. The location of the checkout can advance sales by 0.75%. All fairly insignificant by themselves but taken together they spell out a company that incrementally increase sales by double digit growth.

The automated AI-driven Causal Analytics leads to millions of experiments per day on the e-commerce site, each producing data that after analysis will lead to the next informed wave of experiments. Wave upon wave, day after day, each learning from the previous wave and each wave determining the ever-greater success of the Ecommerce site.

Medical Research

Medical Research is at the cutting edge of data science for the discovery of new treatments. The use of Causal Analytics in stem cell therapies is enabling a new generation of personalized medicine for treatments such as cancer, Parkinson’s disease and cartilage growth.

The era of molecular epidemiology has created a data rich environment that enables substantial in-vitro experimentation. Through these insights new patient-specific treatments can be created using stem cell therapies and other approaches that will transform the lives of patients.

Over the next 20 years medical research is expected to be the biggest beneficiary of research gains enabled by the introduction of Causal Analytics.

Games

Although trivial compared to the applications in commerce and medicine, Causal Analytics has already made it arrival felt in the world of games. In 2017 DeepMind’s AlphaZero system used Causal Analytics to beat convincingly the world’s leading conventional chess computer. DeepMind also developed AlphaGo, a system that defeated the reigning Go world champion in a game that many described as he most complex on the planet – and in so doing used highly unconventional approaches as it does in chess.

The world of board games is a great proving ground for Causal Analytics as it moves from a test environment to full deployment.

 

Download a Headlamp white paper on the use of Casual Analytics in e- commerce: Moving the world from correlation to causality.

 

 

Read Less