Vincent Kwan, Head of Data, CLSA
Big Data and Data Science took the lead when various disruptive innovations shook the business world. The business feared lagging, resulting in a rush to invest in Big Data technology. Imagine a dashboard that provides real-time aggregate KPI views across all business lines with the ability to drill into every detail you can ask, e.g., what is the percentage fulfillment of the yearly sales KPI? Why is it over or under? Which customers contribute the most revenue from which transactions? How are business lines impacted if the business environment changes, and how should they respond? To receive answers to all these inquiries, a vast amount of data must be collected, processed, and structured so it can be easily retrieved, analyzed, projected, and referenced. Though in reality, the failure rates of Big Data or Data Science projects are between 60-85%, according to Gartner and other research sources.
Failure to achieve the above is usually due to problems with the overarching grand plan. A grand plan is proposed to fulfill traditional business expectations but becomes a sugar-coated poison because it gives a false impression of determinism without taking into account the agility of the implementation and adoption of new technology.
Our approach is to deal with the gap between the business expectation and the agility of the technology involved. This is divided into a Data Strategy with four quadrants. Each quadrant will evolve and grow with the plan and link up with each other to construct the whole data ecosystem with the incorporation of governance and intelligence. Here we drill into each quadrant.
The first quadrant is Data Governance. This is an overhyped buzzword that has evolved to cover everything in the data field. We take a more realistic approach and confine it to the Policy and Standard with systems designed to monitor governance, implementation, and data quality management. The establishment of a Data Governance Committee is an important step in assuring the awareness of data within business lines and IT.
• Completeness
• Accuracy
• Uniqueness
• Consistency
• Validity
• Timeliness
• Authenticity
• Security
• Privacy
The second quadrant is the Data Platform which provides the architecture from which big data technology builds a centralized platform for business use cases. Use cases are commonly divided into batch processing and real-time processing. There are many mature technologies available, from the traditional Hadoop/MapReduce to the Cassandra/Spark/Kudu/Impala with Apache Flink for real-time processing, to name a few. Before choosing a specific technology, it is important to run the scenario through an identified use case within your corporation. The objective is to build a Data Platform that will enable technology to resolve pain points within the business.
The third quadrant is the Data Application. This is the area where our buildout differs from traditional data platform projects, which are mainly technology focused. Data in our data platform are all linked or backed by business use cases. We do not blindly pull or ingest all the data into data lakes. Data in the platform has to have a use case attached. This strategy enables our data platform to ingest more useful and quality data. There is also a side advantage to this strategy. By building data applications, team members gain insights into the linkage between data and the business, which in turn helps to improve the data quality and standards. Thus, we take a 'Build to Capture' strategy. Not only do we aim to build an application that is intended to solve business problems, but we also work to capture quality data in the Data Platform.
The objective is to build a Data Platform that will enable technology to resolve pain points within the business
Our fourth quadrant is Data Intelligence. After capturing quality data in the Data Platform, we have the complete data set ready to apply to data science such as machine learning and AI. From here, we may discover relevant correlated patterns to enable business operations and decision-making, e.g., cross-selling and risk impact analysis. The data platform can also serve traditional MIS reporting for business intelligence, given the quality data set.
To successfully build a Big Data platform with intelligence incorporated, we require technology and business to work hand in hand. Without collaboration from either side, the implementation will easily fail. Using the above four quadrants of Data Strategy, we set firm cornerstones to drive the development of the Data Platform with quality data based on Data Governance and Data Application. Only by setting a strong basis for Data Intelligence can the business truly unlock the power of data to enhance profitability and charter a new business arena.
Weekly Brief
Read Also
