Review and understand business requirements ensuring that development tasks are completed within the timeline provided and that issues are fully tested with minimal defects
Partner with software development team to implement best practices and optimize performance of Data applications.
Collaborate across the company and interact with our customers to define, design and showcase new concepts and solutions
Research on new Big Data technologies, assessing maturity and alignment of the technology to business and technology strategy.
Collaborate with other developers to ensure that client needs are met at all times
Work in a rapid and agile development process to enable increased speed to market against a backdrop of appropriate controls.
Implement good development and testing standards to ensure quality of deliverables
Rapidly understand and translate clients’ business challenges and concerns into a solution oriented discussion.
At least 4+ years of experience in design and development using Hadoop technologies stack and programming languages
Hands-on and technical lead experience in 2 or more areas:
Hadoop, HDFS, MR
High Availability architecture and DR setup
Spark Streaming, Spark SQL, Spark ML
Worked with Hortonworks Data Platform as Architect CDH (Cloudera Distribution for Hadoop) as developer/administrator
Hive / Pig / Sqoop
NoSQL Databases HBase/Cassandra/Neo4j/MongoDB
Visualisation & Reporting frameworks like D3.js, Zeppellin, Grafana, Kibana Tableau, Pentaho
Scrapy for crawling websites
Good to have knowledge of Elastic Search
Google Analytics data streaming.
Data security (Kerberos/Open LDAP/Knox/Ranger)
Should have a very good overview of the current landscape and ability to visualise technology and industry trends
Working knowledge of Big Data Integration with Third party / in house built Metadata Management, Data Quality, Master Data Management Solutions,Structured/Unstructured data
Have been active in the community in terms of articles / blogs / speaking engagements at conferences