Posts

BigData Analytics Platform

Image
BigData Analytics Platform   System Components :  -  Data Sources : Externally available data sources from which data which will be collected such as DMP/DSP, CRM, Email marketing, First data, Google analytics, Legacy Systems, etc -   Data Model  : The d ata model and schema that will be used to store raw, in-process, analysed and aggregated data in data stores such as Hadoop, Event Stores and Data Warehouse.  - Data Ingestion : The data ingestion system will consist of data collectors to import data from data sources and inject into the data staging servers. There will be set of micro services developed to run the data collectors for different data sources. Data collectors can be batch, or continuous feeds processors, or they can even be push-pull systems based on the data sources and the interaction protocols such as http, etc.  - Data Staging : Data staging are intermediary servers to temporarily host the incoming ingested data ( from data collecto