Big Data Engineer
- Abu Dhabi, United Arab Emirates
- 2 years ago
- Confidential
- Permanent
- Full time
Job Description
-
Date Posted:
2 years ago -
Location:
Abu Dhabi, United Arab Emirates -
Offered Salary:
Confidential -
Gender:
Any -
Headcount:
1 -
Industry:
Engineering - Others -
Visa Preference:
Emp Visa - Holders -
Joining Timeline:
15 - 30 Days, 10 - 15 Days, Immediate
Job Description
Job Details
Division: Technology
Reporting Manager: Head of Data Engineering
Job Location: Abu Dhabi, United Arab Emirates
Job Purpose
As an ALEF Big Data Engineer you will have a strong understanding of big data technologies with an exceptional ability to code. You will provide technical leadership, working closely with the wider team to ensure high quality code is delivered in line with the project goals and delivery cycles. You will work closely with other teams to deliver rapid prototypes as well as production code for which you will ensure high accessibility standards are upheld. We expect familiarity with modern frameworks and languages, as well as working practices such as Clean Code, TDD, BDD, continuous integration, continuous delivery, and DevOps.
Key Responsibilities
Job Specific Responsibilities:
Defining and developing services and solutions
Define, design, and develop services and solutions around large data ingestion, storage, and management such as with RDBMS, No SQL DBs, Log Files, Events.
Define, design, and run robust data pipelines/batch jobs in a production environment.
Architecting highly scalable, highly concurrent, and low latency systems
Maintain, support, and enhance current systems.
Contribute to paying down technical debt and use development approaches that minimize the growth of new technical debt.
Contribute feedback to improve the quality, readability, and testability of the code base within your team.
Mentor and train other developers in a non-line management capacity.
Build tools (One of SBT, Gradle, Maven).
Ensure all software built is robust and scalable.
Collaborating with Internal and external stakeholders
Participating in sprint planning to work with developers and project teams to ensure projects are deployable and monitorable from the outside.
Work with third-party and other internal providers to support a variety of integrations.
As part of the team, you may be expected to participate in some of the 2nd line in-house support and Out-of-Hours support rotas.
Proactively advise on best practices.
Education, Experience and Required Skills
Educational Qualification
Degree in Computer Science, Software Engineering or related preferred
Processes & Practice
Agile
Scrum/Kanban/Lean
TDD/BDD
CI/CD
XP
Experience: Minimum 5 years
Essential Skills:
- Follow Clean Code/Solid principles
- Adhere and use TDD/BDD.
- Outstanding ability to develop efficient, readable, highly optimized/maintainable and clear code.
- Highly Proficient in either Functional Java or Scala
- Knowledge of AWS Big Data/Analytics services - S3, EMR, Glue, Redshift, QuickSight, Kinesis.
- Experience of big data environments (also advising best practices/new technologies to Analytics team)
- Experience of handling large data sets and scaling their handling and storage.
- Experience of Storing Data in systems such as Hadoop HFDS, S3, Kafka.
- Experience of designing, setting up and running big data tech stacks such as Hadoop, Spark and distributed datastores such as Cassandra, DocumentDBs, MongoDB, Kafka.
- In depth knowledge of Hadoop technology ecosystem - HDFS, Spark, Impala, Hbase, Kafka, Flume, Sqoop, Oozie, SPARK, Avro, Parquet
- Experience debugging a complex multi-server service
- In depth knowledge and experience in IaaS/PaaS solutions (eg AWS Infrastructure hosting and managed services)
- Familiarity with network protocols - TCP/IP, HTTP, SSL, etc.
- Knowledge of relational and non-relational database systems
- Understanding continuous integration and delivery.
Desirable Skills:
- Experience of designing Batch Processing with Spark and Stream Processing with either Spark Streaming or Samza.
- Understanding and experience of Search Data applications/platforms such as ElasticSearch, Splunk and others.
- Familiar with Microservices Architecture
- Mentor and train other developers in a non-line management capacity.
- Experience mentoring or helping colleagues optimise their code..
- System administration and configuration management skills.
- Experience presenting work at user groups, business and to peers.
- Experience of designing and maintaining public HTTP APIs.
- Other languages (Python, Javascript, Clojure, Kotlin etc).
- Other NoSQL databases such Neo4J, Cassandra, Redis etc.
Related Jobs
- Permanent
Jumeirah Hotels & Resorts
Engineer - Jumeirah Group & Corporate
Dubai, United Arab Emirates
- Permanent
NOV
Lead Mechanical Engineer (O&G, Mechanical Equipment - Static & Rotary)
Dubai, United Arab Emirates
- Permanent
Johnson Controls
Project Development Engineer, HVAC - UAE Nationals Only
Dubai, United Arab Emirates
- Permanent
KBR, Inc.
Senior HSE (Pipeline) Engineer (Project) (Offshore Site)
Abu Dhabi, United Arab Emirates
- Permanent
Emirates Airlines and Aviation
Senior Technical Engineer - Integration (UAE Nationals Only)
Dubai, United Arab Emirates
.jpeg?Expires=1721611891&Signature=Ap-zY~19c2qT~K-o1zHQilJeaC-tNeG9v8fCOdV9nYG8fNVxZ7fYOwFqU6XGCOouDEPtrsyFLDKZ5WWl~cOgWaSEMjiw68ss8jK9q-qYdSP3bT2EbzormYTsLIESTsSt0tHRewNghg6K5CfYdOIw9sfgtgxpVZ2l-H89IRyPfA75Sv5ukdv5UuWgo6kudi8e7fX0iiEAJJ0NmC6XMOjLrUcilo6gwzYB0PZZB2A4rYH0mn6h4Pc~f9T5gxEPZtbcprPvRhI16P1DDbl6N4GRBIC234Qglwoi6QL7~~sYmo4paGXd6YclfnxM0up6i-bmrHSpKoMwLXlhp82Lgnkn9A__&Key-Pair-Id=K2NS64SN1PHYK3){kind=logo}
- Permanent
Al-Futtaim
Sr. Mechanical Engineer I Real Estate I Al Futtaim Engineering
Dubai, United Arab Emirates
- Permanent
GE Renewable Energy
TareeGE Graduate Program - Customer Support Engineer
Dubai, United Arab Emirates
- Permanent
Parsons Corporation
Health, Safety and Environmental Engineer
Abu Dhabi, United Arab Emirates
- Permanent
Emirates Airlines and Aviation
Vice President - Engineering Projects
Dubai, United Arab Emirates
