Aspects of Data Cataloguing for Enterprise Data Platforms

摘要

As the adoption of enterprise Big Data platforms mature, the necessity to maintain systematic catalogue of data being processed and managed by these platforms becomes imperative. Enterprise data catalogues serve as centralized repositories of storing such metadata about data being handled by such platforms, enabling better data governance, security and control. Standardized approaches, methodologies and tools for data catalogues are being discussed and evolved. This paper introduces some key variables, attributes and indexes that need to be handled in such cataloguing solutions -- such as data contexts, data-system relationships, data quality, reliability, sensitivity and accessibility. The paper also discusses specific approaches on how each of these aspects can be adopted and applied to different enterprise contexts effectively.