Technical Lead - Data Engineering8-12 yearsNoidaGraduation/Post Graduation
- Define client needs and oversee project milestones to ensure expectations, timelines, and budgets are met
- Define data platform architecture, design, and review code and make required changes
- Will be responsible for the overall quality of project deliverables and the successful implementation of defined solution for the customer
- Building long-term and superior client relationships, proactively managing client expectations and ensuring that change control is used when scope boundaries are exceeded
- Communicate a compelling and inspired vision to all customer levels, from CTOs/CIOs to engineering managers and programmers.
- Maintain a strong network and promote the organization at various meetings, forums, panels, publications, and conferences to establish thought leadership in the industry
- Responsible for POCs on any tech stacks introduced, or possible version upgrades.
- Leverage extensive knowledge of distributed systems, process flows, and procedures to aid analyses and recommendations for solution offerings
- Hands-on and technical experience in 4 or more areas:
- Hadoop, HDFS, MR
- Spark Streaming, Spark SQL, Spark ML
- Apache NiFi, Apache Airflow
- Hive / Pig / Sqoop
- NoSQL Databases HBase/Cassandra/Neo4j/MongoDB
- Performance Optimization
- Good to have knowledge of Elastic Search
- Good to have an understanding of Google Analytics data streaming.
- Data security (Kerberos/Open LDAP/Knox/Ranger)
- Should have a good overview of the current landscape and the ability to visualize technology and industry trends
- Working knowledge of Big Data Integration with Third-party / in-house built Metadata Management, Data Quality, Master Data Management Solutions, Structured/Unstructured data
- Have been active in the community in terms of articles/blogs/speaking engagements at conferences