Published on the 29/09/2015 | Written by Donovan Jackson
The biggest problems facing business intelligence are still the simplest– getting ‘clean’ data and avoiding silos…
However, a North Shore BI consulting company said rather than futile efforts to eliminate silos, they should be embraced as a natural way of working, with systems designed to cope and even enable them.
That’s the position taken by Darryl Wolfaardt, MD of Mettle Consulting. “Extract, transform and load is still the major bugbear with BI; probably 80 percent of the time spent on BI projects goes into getting the data right,” he said.
It’s not the first time iStart encountered just this issue in the short space of one week; Wellington startup Eight Wire recently made the news for its solution to BI data compatibility.
Wolfaardt, on the other hand, said his quest for a better way led him to BIML. That’s Business Intelligence Markup Language, and he said this is a factor in the ‘democratisation’ of BI, together with improved metadata and better education of the users of BI tools.
BIML was created by software vendor Varigence, for which Mettle has recently secured the New Zealand rights; it, said Wolfaardt, provides for the establishment of a metadata ‘umbrella’ over BI initiatives which makes each individual effort more obvious to other people within any given organisation.
An ‘umbrella’ layer is necessary to allow the natural way of working with information to continue, while delivering improved company-wide insight into the various initiatives underway.
That’s because, said Wolfaardt, the notion of ‘breaking down siloes of information’, common to many BI initiatives, flies in in the face of the practical reality of working with data. “The emergence of siloes is such a common theme of BI discussions precisely because of the tendency to grab information and then package it into a useable format – and typically, that format means the emergence of yet another silo,” he explained.
Wolfaardt described BIML as an accelerator which allows BI initiatives to be delivered in fractions of the time taken by traditional approaches, by addressing the ETL problem. “The BIML toolset is metadata driven; define rules, frameworks and patterns and apply them to the data sources you’re introducing to your data warehouse or data mart.”
Rather than seeking to combat the silo problem, he said if anything, BIML may make it easier to create them.
Wolfaardt said BIML and its metadata methodology is a first step towards a metadata approach to data. “For organisations to start using patterns and frameworks to leverage their information needs, they need to gradually grow into a program of work that combines Technical Metadata – ETL – and Business Metadata – Knowledge and Context. This enables a more complete view of the world and reduces duplication of effort and associated costs.”
