This Forum is hosted by Dr. Yan Zhao, Owner of the ArchiTech Consulting LLC, an expert enterprise and business architecture firm providing consulting, R & D, and training services to digital enabled enterprise and business. With highly qualified subject matter experts, it aims at achieving optimal business performance by creating innovative and suitable solutions to support efficient decision making and to reduce system complexity and cost.
Wednesday, May 28, 2014
Big Data, All Data, or Just Data
Big data is a hot topic these days, and it is extended to “All Data” recently. Actually, we are dealing with data from multi-sources, multi-channels, and multi-media; and the data can be from real-time business operation, data warehouse, back-office business data (e.g. financial, CRM, HR), social media, etc. The accumulated data amounts are enormous, although each individual data source can be big or small. The data correlations from different sources are as important as the analysis of “big data”. Therefore, we cannot ignore the importance of data management and processing.
Information and data management is getting more and more challenged due to the velocity and volume of data generation, as well as the tendency of increased data production. An important step for information and data management could be how and what data should be collected. The speedy growing of data could make people wonder even we can store them (e.g. using the new technologies like Hadoop), how we can catch up in processing them to make them really meaningful during analysis, e.g. to transform the data into meaningful information, and transform information into useful knowledge. Therefore, we need to improve the data collection mechanisms, e.g. to have cleaner data and better data organization mechanisms, with data/information correlation indicators. More meaningless data involved more burdens to the process. If data processing speeds continue lag behind data collection speeds, the data collected will not be as useful as expected, but add burden to the slow process. Yes, this is the new challenge in dealing with data, regardless big data or all data.