Data, Data Everywhere…  

Throughout my travels, I hear a common complaint.  Please feel free to substitute names, departments and activities that feel appropriate to you.  “How could that VP of Sales make a decision like that when she doesn’t even know what she is talking about?”, or “The guy in the back decided to make the ‘Process Better’ (bunny ear quotes) by making his job easier and our work harder”, or, “if they had just asked me. I could set them straight.” We all realize that the better decision would be made if only the decision maker had more and better information. That is the crux of the problem.  It is always easier to point out the flaw, in hindsight, rather than determine the solution before the fact.  

Data analysis is becoming a bigger key to the solution.  For thousands of years manual data collection was the only choice and it was limited at best.  As noted in Wikipedia, “Businesses then lacked the computing resources necessary to properly analyze the data, and as a result, companies often made business decisions primarily on the basis of intuition.” [i].  Now in the 21^st century we have huge caches of data at our fingertips. Everywhere we look we can access data our ancestors never dreamed of.  We can generate data easily.  We can search the internet and download tons of unqualified data.  Go to your favorite search engine and type “business intelligence”.  I found 53,600,000 entries, including my Wikipedia reference above.  In the first ten entries companies such as Microsoft, Oracle, Cognos, IBM, SAS & the CIA were noted.  This is an illustrious and awe inspiring group. I chose to quote Wikipedia.  Why did I choose this source to quote?  Three reasons: 1-It was simple.  2-It was quick.  3-It was easy to understand.  I didn’t need to sift and sort, analyze and correlate.  This is the primary goal of every Business Intelligence solution.  Churn all of the data, qualify it, validate it and make it simple, quick and easy to understand. Dashboards – Executive Information – Alerts –  I don’t care which you choose, just keep it simple and keep it coming.

BusinessIntelligence.com provides a great source of articles, white papers, BI news, research and solution searches. However, too much data is just as bad and too little data. When a search for “business intelligence” starts with over fifty three million choices, it is easy to become overwhelmed. Looking at business Intelligence we have many different approaches with different requirements for skill sets and resources.  We cannot be sure which approach will succeed and which will fade away.  Our companies cannot afford to commit to more than a few, and probably just one or two.  How can we sift through the choices? Crystal Reports, Crystal Analysis, Microsoft Reporting Services, Microsoft Analysis Services,  OLAP, Excel Pivot tables, LogiXML, Data Warehousing, Alerts, Dashboards, etc, etc, etc…

Let me try to break these into smaller pieces.  Reporting is straightforward data gathering.  This is not to say reports cannot be complicated or sophisticated.  Reports connect directly to a source (database tables, views, or worksheet) and select records based on a query or filter.  Crystal Reports, Microsoft Reporting Services, Excel Pivot Tables (and external data queries) and LogiXML are in this group.  Each tool has various features and interface differences.  Some of these tools require greater technical skills to optimize but they all are straightforward in gathering data.

Data Warehouses use a two step process to bring us data.  There are many approaches to build the data warehouses. Most of them make my head hurt, so I will try and keep it simple. First data is gathered (like reports) into a repository.  A query is made to a database and the data is loaded or transformed into the new system. This can be a database, an OLAP (Online Analytical Processing) “cube” of data, or other filing system.  Data can be collected from disparate systems and normalized into the repository.  For example, a large global operation has many divisions with different ERP solutions.  It would be extremely difficult, if not impossible, to create a single report to connect to each ERP solution and gather customer data together. To facilitate the data gathering, each system runs a query and extracts customer information.  The customer identification is called “Customer Number” in one, “Account ID” in another, and “Customer ID” in a third.  The data is read loaded into a field called “Cust_ID” in the warehouse.  Data is qualified before loading. Field sizes and characteristics are also normalized in this transformation.   Data is gathered and warehoused in the new system.  A single set of reporting & analytical tools can access the data in a single secured location.  There is no need to open security to all systems around the globe for centralized reporting. Data can then be displayed as sliced and diced in a cube drilldowns, Dashboards, emailed as an Alert, or in a basic report format.  Crystal Analysis, Microsoft Analysis Services and LogiXML are some of the systems that can create and report on data Warehouses.

In spite of the well intentioned “Business Intelligence” initiatives, I suspect many businesses are still making decisions based on intuition…wc