SYSTEMS AND METHODS FOR MANAGEMENT OF DATA ANALYTICS PLATFORMS USING METADATA

摘要

A data analytics system is disclosed that can include a data repository configured to store data for multiple clients, a metadata repository separate from the data store, an access control system, and a policy store. The data analytics system can automatically generate metadata for data in the data repository using a metadata engine, the metadata including technical metadata and usage metadata, and store the metadata in the metadata repository. The data analytics system can obtain a client policy governing access to the data. The data analytics system can receive a request to provide the data, the request including instructions to create a pipeline to provide the data. The data analytics system can authorize, by the access control system, the request using the policy and usage metadata; create the pipeline using the technical metadata; and provide the data using the pipeline.