Business intelligence is evolving at a blistering pace. In the last couple of years, some versatile business intelligence technologies have emerged that empower users to retrieve data from many different sources, build sophisticated models, and analyze data like never before.
They provide extensive reporting capabilities and powerful dashboards that can help management make decisions that have a positive impact on the company’s growth and profitability.
Several business intelligence tools have become so easy to use that business users, including financial managers, have learned to create many of the reports needed for their daily operations without intervention from IT. Modern BI tools, like Tableau, Power BI, and QlikView, are easy to implement. A few of them are very affordable, and some are even free! Yes, that is the case, for example, of Microsoft Power BI Desktop.
So why can’t we use these wonderful modern BI tools to pull information directly from your ERP, CRM, Email servers, spreadsheets, social media, Big Data platforms, and other data sources? Wouldn’t that give business users the freedom to do analytics on their own without depending on IT all the time? In other words, why would we still need a data warehouse?
There are many use cases for leveraging the data discovery capabilities of these tools to connect directly to different data sources to create reports and dashboards for analytical purposes. I used to spend countless hours manually pulling data from different sources into Excel, and using pivot tables to build business reports and charts, which I would later use in my PowerPoint presentations to executive management. Since I started using modern BI tools, I have been able to appreciate the great value they provide. For the majority of business users that have been relying on Excel, modern BI tools offer huge gains in productivity.
As the volume, diversity, and complexity of the data increases, however, the need for preparing the data for consumption beforehand becomes more evident.
Let’s take a look at some of the drawbacks of solely relying on BI tools for business analytics:
- If you are a business user, every time you want to generate a new report, you will have to spend time planning and setting up the connections and queries to retrieve the right data from multiple sources. This is very inefficient.
- If many users are frequently running queries against core operational systems like an ERP, this could cause performance issues for the ERP and BI users, impacting the productivity and performance of the organization, a highly undesirable outcome that should be avoided.
- What if you get an error while building the report or dashboard? Even if you can figure out the reason for the error, it will again require you to spend a lot of time just creating a new report or dashboard or updating your existing ones.
- What happens if there is a change or upgrade in one of the source systems or even your BI application? Again, the report may not run as expected or not run at all, and you will have to escalate the issue to IT, wasting precious time and resources.
- When users can directly access the critical systems of your organization there is an increased risk that they could compromise the integrity of the data either accidentally or on purpose. There is a huge security risk with this approach.
These are just a few reasons why it is not a best practice to exclusively rely on the BI tool to directly connect to mission-critical data sources for analytical purposes. There may be cases when this is justified, but they tend to be the exceptions.
Let’s now review some of the benefits of building a modern data warehouse using best practices that have been accumulated after many years of experience, and adapted to the new potential that the latest BI tools offer:
- One of the key benefits is performance improvement. Unlike operational systems, data warehouses are designed to collect and organize large amounts of historical data for analytical purposes. A well designed and correctly implemented data warehouse is the foundation for optimal business intelligence and analytics. The ETL (extract, transform, and load) process is performed periodically using special tools that are optimized to extract the data from multiple sources, transform it by applying specific rules or operations so it conforms to your organization’s needs. The ETL can be performed daily as a batch process or in near real time, depending on the needs of the business. The result is data that is available in the right format to be consumed by business users using business intelligence tools. They will not have to waste time integrating the data, since it is already consolidated in the data warehouse.
- Users can create more valuable reports. They can slice and dice the data by the key descriptive attributes for their business. For example, a report could show total sales for the year, or total sales monthly, weekly, or daily, by region, product, sales representative, business unit, etc. They can create dynamic dashboards. Many of these different reports and dashboards can be generated within minutes, instead of hours or days.
- More reliable and consistent data. The data in the data warehouse has been assembled from the right data sources, cleansed, standardized with common labels and definitions so that all business units can speak the same language, and prepared for user consumption.
- Improved security. The data warehouse provides secure access to information for different types of users.
The data warehouse is so important that when most analysts and consultants refer to the term “business intelligence” or “BI”, it is implicit that the data warehouse is part of the concept. Modern BI tools used in combination with modern data integration and data warehousing strategies offer the best of both worlds: Agility, ease of use, self-service, combined with higher performance, more reliable and consistent data that can be trusted, and integration with new Cloud and Big Data platforms.
The business case for many companies that still do not have a data warehouse or those organizations that have not modernized their legacy data warehouse is very compelling. If you reach out to an experienced business intelligence consulting firm, they will help you build the business case, provide the plan, data analysis, and architecture required for your company to take this vital digital transformation step to stay competitive, reduce costs, and increase revenue and profits.