Design, architect, and build stream processing systems which deal with unbounded data
Design, build, and deploy software using technologies such as Kafka, Mqtt, Spark, Serverless, Kubernetes.
Work with the data engineering team to develop & maintain data pipelines for batch & stream processing.
Work closely with Development Architects on designing/implementing code optimizations.
Identify, design, and implement internal process improvements and tools to automate data processing and ensure data integrity while meeting data security standards
Work with data scientist and business analytics team to assist in data ingestion and data-related technical issues
Perform in-depth analysis of performance issues to identify hotspot and opportunities for optimization
Provide technical consulting and mentoring to team members on optimization techniques, data processing design, implementation and conformance to architecture standards.
Minimum bachelor’s degree in Computer Science, Applied Mathematics, Engineering, or any other technology related field. An equivalent of the same in working experience is also accepted for the position.
Minimum 8 years of software development and system integration experience including 4 years of experience in data warehousing / distributed system (e.g. Hadoop)
Good Java / J2EE, Python, Scripting experience
Proficient in Big data tools and ecosystem (e.g. Hadoop, HIVE, HBase, Kafka, Spark etc)
Expert in building and optimizing big data pipelines, architectures, and data sets
Experience in data ingestion, cleaning and processing tools.
Excellent experience in data processing using Scala/Python/Java
Good in user requirements gathering, maintenance, and support
Highly organized, self-motivated, pro-active, and able to plan
Experience managing users and vendors is a plus
Past hands on experience with Advanced Analytics is desirable
Good Architectural concepts around Cloud Computing and base software Development
Interested candidate, please submit your updated resume in MS WORD format.
We regret that only shortlisted candidates will be notified.