There are two visions for the future of data analysis and analytics. Tech and business magazines and IT evangelists see artificial intelligence (AI) and predictive analytics on the rise.

When I talk with business users and executives, especially ones already working with data warehouses, they see the future (and invest) in self-service business intelligence (SSBI).

One term – data analytics – having two meanings – AI and SSBI – this is the classic setup for misunderstandings, failing project pitches, and failed projects. But what exactly are the differences between AI and SSBI? And are they complementing or competing concepts?

The rise of self-service business intelligence

Business intelligence teams (and data warehouses) are established in many companies and organizations. They have been implemented years ago and are constantly in use. Data warehouses collect data from various (operational) databases, clean and transform it, and generate periodic reports and provide answers to ad-hoc queries. As a kind of grassroots movement, especially experienced users started to complain. They know the data in their data warehouse and its potential value for the business and their work. However, they cannot unlock all benefits. The limitations of their rigid business intelligence installation hinder them. Thus, the user feedback is often not flattering – too expensive, too inflexible, too slow.

This contrasts my personal observation that business intelligence and data warehouse engineers know the data models very well, often better than the business. Thus, if we want to understand why they are considered to be too expensive, too inflexible, and too slow, we should not look at the qualification of individual engineers and how they perform single tasks. The focus should be on the operational model. More precise criticism proves this:

Figure 1: Limitations of many business intelligence services from the users' perspective

Self-service business intelligence – the core concepts

The four points make it clear that ‘self-service business intelligence’ is more than enabling users to submit queries against an existing data warehouse with its current data set. Furthermore, the best indication of not matching the user expectation is when they ask the data warehouse team for reports containing all data. This is when they start querying the data themselves in Excel or Access and avoid the business intelligence service bottleneck as much as possible.

Many challenges are process-related, but the solution is software. SSBI software provides features that change how business users and business teams collaborate. I want to illustrate this by looking at some features of Microsoft Power BI. There are many more vendors, but a quick look at Google Trends proves that Power BI gets nearly as much attention as the Business Intelligence topic itself (Figure 2).

Figure 2: The rise of Power BI. A global Google Trends evaluation looking at this software solution compared with the general area of business intelligence

Users can create inspiring reports in Power BI with various diagrams including drill-down functionality or presenting numeric values on geographic maps. They can create complete cleansing and transformation pipelines using a GUI with features such as filtering or replacing values. This sounds like Excel on steroids, but it is a different software product class. Users define and execute reliable ETL processes on their local machine. This helps for periodic or recurring evaluations. In short, SSBI tools provide a comprehensive access layer for users for data access, for querying and transforming data, and for producing visually appealing reports (Figure 3, ①).

A trivial feature in Excel, but revolutionary in the world of databases, addresses the challenge of the long delays of getting new data into the data warehouse. Users can load data from their local laptops into Power BI and use them together with data stored in databases and data warehouses (Figure 3, ②). This offers users new opportunities for ad-hoc queries. They can run them without involving the business intelligence team. If they need a periodic report, they can implement a temporary solution to bridge the time till the business intelligence team implemented the final report – or they can hand over their reports to the business intelligence teams as a specification.

The ability to implement queries with a mixture of data warehouse and local data is also helpful from a governance perspective. It is always clear which data and reports are in the responsibility of a business intelligence team and which are in the user sphere.

Many companies benefit a fourth core concept of SSBI: enhanced collaboration features. They are rarely mentioned in articles, but Microsoft charges extra for them. This is a clear indication of the value of these features for larger companies. I also remember an event of another major player in the SSBI market, Qlik, some time ago in Zurich. One customer presentation was exactly about that: enabling controllers in an international organization to share and annotate reports instead of sending them around via Email and merging various feedback threads in yet other Excel sheet. SSBI can make copying, comparing, and consolidating Excel sheets obsolete (Figure 3, ③).

These three SSBI innovations do not relate to AI. This explains why pitching AI projects to the business does not work if they talk about analytics and have SSBI in mind. However, SSBI can serve as a kind of Trojan Horse. The tools usually allow integrating R or Python code (Figure 3, ④). As always, there are prerequisites and limitations – additional local installations might be needed, the modules work only on the presentation layer, etc. That is the “half-empty glass perspective”. The “half-full glass perspective” is that many users – all SSBI users – are closer using such tools in their company than ever before.

Figure 3: Self-service BI concepts

Comparing artificial intelligence use cases and self-service business intelligence use cases

AI derives insights automatically using algorithms, math, and statistics. In contrast, self-service business intelligence enables humans to analyze data more efficiently, to provide better reports, and share the reports. So, are AI and SSBI separate topics and not related to each other? Or, more provocatively, does AI make SSBI obsolete?

AI is great for finding patterns in large data sets. It all starts with a training set. The algorithms go through the set and derive and learn rules. These rules can be applied to new data the algorithm has not seen before. Looking at the behavior of today’s customers, companies can estimate how lucrative a new client is. Certainly, humans can do the same: they load data into Excel or Power BI and try to find correlations between gender, age, income, etc. to estimate how much a new customer can pay for the next holiday trip. However, humans – with or without SSBI tool – have no chance against AI. AI analyzes thousands of input variables and looks at relationships between dozens of variables. Humans cannot do the same. Thus, AI makes the use case “tool for users to manually analyze large data sets” for SSBI obsolete, because this stands this should not be done by humans anymore. However, it depends on the company culture how they move. Not everyone is ready to switch to AI-generated knowledge.

There is also some good news for the SSBI vendors. There is a second reason why employees have to analyze large data sets: understanding data singularities. In companies, large reports often contain some “surprising” data constellations. The management has to understand the reason. Is it a mistake in the data or is the reality different than the expectation? Why is the revenue dropping in the US whereas the rest of the world prospers? Why do we see a peak in profit in Antarctica where we do not have any branch? Controllers spend a lot of time with such tasks. The tasks are about cleansing data and understanding whether data discrepancies make sense or are caused by errors. There is nothing to learn and generalize using AI.

Figure 4: Comparing the use cases for AI and SSBI

So, the conclusion is: vendors sell both artificial intelligence and self-service business intelligence for “data analytics.” Customers have to understand what they exactly want to achieve, analyzing data singularities or learning rules. AI beats humans in finding rules and patterns, though some employees want to continue doing their Excel analysis (and demand SSBI to optimize doing their obsolete work). Here, the concept of artificial intelligence and self-service business intelligence compete. In contrast, AI is of no help when it comes to understanding data singularities. This is the true domain of self-service business intelligence. So, from a high-level perspective, self-service business intelligence complements artificial intelligence, because they have unique strengths best suited for different use cases.

Klaus Haller is a Senior IT Project Manager with in-depth business analysis, solution architecture, and consulting know-how. His experience covers Data Management, Analytics & AI, Information Security and Compliance, and Test Management. He enjoys applying his analytical skills and technical creativity to deliver solutions for complex projects with high levels of uncertainty. Typically, he manages projects consisting of 5-10 engineers.

Since 2005, Klaus works in IT consulting and for IT service providers, often (but not exclusively) in the financial industries in Switzerland.