Operationalization & Optimization role is required for BigData Platform: to ensure effective, balanced, secure and auditable use of Tools & Workflows
Methodology:
- Templates: Operational Configurations by function
- Monitoring: Multiple Job Execution Pipelines
-
| Set 1: |
Cluster / YARN Optimizations
- YARN - Applications deployments
- Performance &Throughput Mgmt.
|
-
| Set 2: |
Data Engineering:
- Data Security, Audit Trails
- Operational Job queues
- Data Archival & Lifecycle Mgmt.
|
-
| Set 3: |
Data Lake Governance
- Data Quality, Metadata, Lineage
- Data Domain Specific Taxonomies
- End-User Search via SOLR Indexing
- Data Lifecycle Management
|
