The Big Data Lead is an accomplished leader in Analytics, Data Engineering, Architecture, governance and Framework Design and someone who guides teams to success.
Define client needs and oversee project milestones to ensure expectations, timelines, and budgets are met
Define data platform architecture and design, have hands on capability to review code and make required changes
Leverage extensive knowledge of distributed systems, process flows and procedures to aid analyses and recommendations for solution offerings
Responsible for the overall quality of project deliverables and the successful implementation of defined solution for the customer
Build long-term, superior client relationships and proactively manage client expectations, and ensure that change control is used when scope boundaries are exceeded
Communicate a compelling and inspired vision to all customer levels, from CTOs/CIOs, to engineering managers and programmers.
Maintain a strong network and promote the organization at various meetings, forums, panels, publications, and conferences to establish thought leadership in the industry
Responsible for POCs on any tech stacks introduced, or possible version upgrades.
At least 8+ years of experience in design and development using various database technologies with recent 4+ years associated Hadoop technologies stack and programming languages
Hands-on and technical lead experience in 2 or more areas:
Hadoop, HDFS, MR
High Availability architecture and DR setup
Spark Streaming, Spark SQL, Spark ML
Worked with Hortonworks Data Platform as Architect CDH (Cloudera Distribution for Hadoop) as developer/administrator
Hive / Pig / Sqoop
NoSQL Databases HBase/Cassandra/Neo4j/MongoDB
Visualisation & Reporting frameworks like D3.js, Zeppellin, Grafana, Kibana Tableau, Pentaho
Scrapy for crawling websites
Good to have knowledge of Elastic Search
Google Analytics data streaming.
Data security (Kerberos/Open LDAP/Knox/Ranger)
Should have a very good overview of the current landscape and ability to visualise technology and industry trends
Ability to successfully lead a team of big data engineers and big data developers through all phases of the development life cycles, including requirements definition, architecture, design, development, implementation, and testing
Working knowledge of Big Data Integration with Third party / in house built Metadata Management, Data Quality, Master Data Management Solutions,Structured/Unstructured data
Have been active in the community in terms of articles / blogs / speaking engagements at conferences