Unsupervised Learning
Posts
The Difference Between Business Intelligence, Reporting, Metrics, and Analytics

The Difference Between Business Intelligence, Reporting, Metrics, and Analytics

After 20 years in technology I’ve taken away one thing when it comes to religious tech battles: When people are disagreeing, they’re probably using different definitions for the various terms.

Business Intelligence is no different. You can spend three hours on Google searching for definitions of Business Intelligence, Metrics, Analytics, and Reporting, and find 3-5 conflicting meanings. And not only different meanings between pieces, but conflicts within the same piece.

This article will provide a framework in which all the terms can be used in a consistent way.

Conflicting views

Most of the rancor comes from vendors who provide one of these offerings to their customers, and their product managers and marketing teams go out of their way to make sure nobody confuses it with a “lower” art form.

Mine is an opinion piece too, but I don’t have a BI product and I purposely started by ingesting all others to triangulate the truth.

Here are some of the differences you’ll see drawn from various blog posts and opinion pieces online. Note that some of these distinctions were legitimate in the past but are no longer the case.

Reporting is supposedly only related to the past, which makes sense given that you can’t “give a report” on something that hasn’t happened yet. Realtime reporting in this context just means a very small delta between the past and present, i.e., the report updates quickly.
Metrics is generally the umbrella term that people use for all these terms, which I think is correct. It’s simply a measurement of something.
There’s used to be a common belief that if you can “drill into” a piece of data, that means it’s Business Intelligence rather than Reporting. This used to be somewhat true back when every piece of software didn’t produce interactive charts, but now being interactive has nothing to do with the content’s value or depth of analysis. 
One strong distinction was between Operational and Strategic, with these even having separate technical infrastructures in many cases. Basically the Operational systems would be more frequently updated, and the BI system would pull certain data from that system at certain intervals, place it into a separate BI system, and BI reports would only be produced from that second system.
Whether information updated slowly or quickly was also a key distinction in the past, and this is because in the past you would run “reports”, which would often only be monthly, weekly, or daily. As a result, systems that could run reports more often started getting their own names. 
Business Analytics seems relatively synonymous with Business Intelligence, meaning it’s more advanced analysis.

My attempt at agnostic definitions

In short, I think these terms are chaotic right now because technology has dissolved many of the distinctions that made them separate.

So what I did was read dozens of these articles online and a couple of Business Intelligence books. Then I triangulated what most agreed upon for the definitions. Then I removed all the definition differences that came from legacy computing. And finally I went back to first principles for what these words mean in other industries and contexts, e.g., “intelligence”.

Data, information, and intelligence

Data are raw, unorganized facts. (“Login from X at time Y.”)
Information is a collection of data that help to answer a basic question (“How many people logged in within this period?”)
Intelligence is the combination of information into a form that tells a story and informs decisions. (“People who don’t go through the training are 74% less likely to make a final purchase.”)

Knowledge is often listed as part of this hierarchy, but I believe that to be incorrect. Knowledge is not something that’s provided by a metrics or BI system, but rather the effect that’s created when information and intelligence are consumed by a user. Knowledge, in other words, is the result within the user that enables better decisions.

Operational, tactical, and strategic

Operational relates to the day-to-day functioning of the organization
Tactical relates to the implementation of the strategy
Strategic relates to the state and trending of the strategy

Metrics, reporting, analytics, and business intelligence

Metrics are just measurements, and they can apply to pretty much everything from the most basic reporting to Business Intelligence.
Reporting is any view of the current state or the past that provides information or intelligence.
Business Intelligence provides intelligence about an organization.
Analytics is synonymous with metrics, but has the connotation of including both information and intelligence in a relatively real-time fashion.

Discussion

Using these definitions we can talk about real-world use cases without tripping all over ourselves. Here are some observations about these different types of metrics.

Unsupervised Learning — Security, Tech, and AI in 10 minutes…

Get a weekly breakdown of what's happening in security and tech—and why it matters.

You can have reports that happen monthly or every second. Update frequency is not part of the definition.
You can have reports that show information or intelligence. A report is just a presentation of something, and the content can vary. Think of it like a “screen”. What’s on a screen? That’s the same with a report.
There is a temporal aspect to Business Intelligence that’s interesting. A modern BI platform should be able to at the very least show you different slices of the past, get you as close to current as possible, and through its intelligence capabilities (combined sources, innovative visualizations that tell stories, narrative, trends, etc.) provide some indication of the future as well.
The use of Machine Learning is not some new tier in the hierarchy of data, information, and intelligence. ML simply helps inform the intelligence side, i.e., people who have X and Y are 93% more likely to do Z within 21 minutes of purchase. In this context think of ML-like Spark or Hadoop; it’s just underlying technology that helps you produce information and intelligence.
You can combine the various definitions above to create different outputs, e.g., Operational Information, Tactical Information, Strategic Intelligence, etc, with the keys being “answering a basic question”, “telling a story”, “relating to basic day-to-day business function”, “implementation of strategy”, or, “state or trends around strategy”. Combining those you end up with a matrix of different types of business value.
Providing operational metrics/reporting is often considered an operational position because issues with the reporting could cause production-related problems. This is another reason that such a bright line was drawn between operational and strategic metrics in the past, as the latter were always delayed and untethered from day-to-day activities.
Highly attractive and interactive interfaces should be separated from whether they’re showing information or intelligence. A good interface can show one, the other, or both. For example, a pivot table in Excel can slice widget sales a million different ways, and if you have the ability to display those views in an attractive, modern, and interactive way then you might be compelled to call it “intelligence”. But the distinction between information and intelligence is whether a story is being told from the content, not whether or not the interface is attractive. Keep those separate in your mind.
At the high end of intelligence lies the ability to make recommendations. The Holy Grail here is, “if you make the following change, we expect the A and B metrics to move by C and D amounts.” Like ML, this is not a new type of intelligence, it’s just better intelligence combined with built-in analysis. It’s moving into the territory of the user of the platform, i.e., decision-making based on what was presented.

Decision support maturity

I think it’s useful to imagine the motion from data, to information, and then intelligence, as moving along a spectrum of decision support maturity. But I think there are six levels, not three.

No Data: zero decision support, you’re operating blind
Data: virtually no decision support, because the data require interpretation
Information: minimal decision support, answers basic questions
Intelligence: significant decision support, creates a narrative
Recommendations: exemplary decision support, narrative + recommendation
Automated Changes: removing the need for a decision altogether

Basically, because the whole purpose of this space is to reduce the load on the decision-maker, the pinnacle of that is actually making a decision unnecessary.

You’ll still want to report on automated changes to make sure you understand what’s happening.

As you move from information through to automated changes, that’s the evolution that’s occurring. You’re making the job of the decision-maker increasingly easy by doing more of the work for them, until—finally—they don’t have a decision to make at all.

Summary

Much of the confusion in this space is caused by people conflating metrics technologies with the purpose of metrics itself.

The entire purpose of all of this—from a printed bar chart to a realtime, next-gen, predictive, AI, recommendation engine platform—is decision support. We’re helping people make better decisions. That’s it.

My recommendation is to think of things as data, information, and intelligence—processed to inform at the operational, tactical, and strategic levels.

Don’t let marketing and other forms of dogma take you away from these first principles.

Notes

A special thanks to Tracy T. for her knowledge in all things data.
The automated changes maturity level is still pretty far off in most fields, simply due to the complexity of most businesses. It’s one thing to automatically take a drone out of a fleet if it seems to be failing, but another thing altogether to hire, fire, or adjust budgeting for major projects using automation.