Title:  Software Engineer (Data Processing)

Job Summary

We are seeking a skilled and innovative Data Processing Engineer to join our team. As a Data Processing Engineer, you will be responsible for the design, development and validation of software for Data Processing Engineering across both cloud and on-premises environments.. You will be responsible for contributing to the design, and development and testing of code. The software applications you build will be used by our internal product teams partners and customers.

We are looking for a hands-on lead engineer who is familiar with Python, Spark, Scala and Java. Any cloud experience is an advantage. You should be passionate about learning, be creative and have the ability to work with and mentor junior engineers.

Job Requirements

• Experience in design and build the Data Processing Platform, and understand scale, performance and fault-tolerance
• Expertise in identifying the right tools to deliver product features by performing research, POCs and interacting with various open-source forums
• Work experience on technologies related to NoSQL, SQL and in-memory databases
• Good understanding of best-in-class monitoring processes to enable data applications meet SLAs
• Work experience with Python, and Java to write data pipelines and data processing layers.
• Strong in CS fundamentals, Unix shell scripting and Database Concepts
• Working expertise with Data processing pipeline implementation, Kafka, Spark, NOSQL DB's especially MongoDB (Cassandra, TSDB preferred) and SQL
• Awareness of Data Governance (Data Quality, Metadata Management, Security, etc.)
• Knowledge and experience with Kafka, Cassandra or Presto is an added advantage
• Familiarity with GenAI, Agile concepts, Continuous Integration and Continuous Delivery
• Experience in Linux Environment with containers (Docker & Kubernetes) is an advantage

Education

•    A minimum of 5 years of experience is required and must be hands-on with coding.
•    A Bachelor of Science Degree in Electrical Engineering or Computer Science, a Master Degree, or a PhD; or equivalent experience is required