Lead Data Engineer
- Abu Dhabi, United Arab Emirates
- 2 years ago
- Confidential
- Permanent
- Full time
Job Description
-
Date Posted:
2 years ago -
Location:
Abu Dhabi, United Arab Emirates -
Offered Salary:
Confidential -
Gender:
Any -
Headcount:
1 -
Industry:
Engineering - Others -
Visa Preference:
Emp Visa - Holders -
Joining Timeline:
15 - 30 Days, 10 - 15 Days, Immediate
Job Description
Overview
The opportunity
Core42 is the UAEs national-scale enabler for cloud and generative AI, combining G42 Groups expertise across multiple technology disciplines into a single platform for public sector and large enterprise transformations. Building on our capabilities as sovereign cloud and HPC specialist, we bring generative AI, cybersecurity, professional and managed services expertise to enable national-scale program deployments across industries.
Responsibilities
Your key responsibilities
Technical Leadership and Strategy
- Develop and implement comprehensive data strategies, guiding the technical direction of projects while aligning with the organization's long-term objectives.
Mentorship and Team Development
- Mentor junior data engineers, fostering a culture of continuous learning and development within the team.
Data Pipeline Architecture and Development
- Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics.
Data Integration
- Work closely with data scientists, ML engineers, and stakeholders to ensure that data is accessible, consistent, and reliable for ongoing projects.
API and Data Services
- Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed.
Data Storage
- Manage and optimize data storage solutions, including relational databases, Search Engines like Elasticsearch and NoSQL databases, to support the requirements of machine learning models.
- Understand data engines and structure to effectively design solutions for transactional, analytics, and search purposes.
Data Quality and Governance
- Implement processes to monitor data quality and ensure production data is always accurate and available for key stakeholders.
Performance Optimization and Scalability
- Ensure the high performance and scalability of data systems to accommodate increasing data demands.
Collaboration and Support
- Collaborate with ML engineers to assist in data-related technical issues and provide architectural guidance and solutions.
Security and Compliance
- Ensure compliance with data security and privacy policies.
Documentation
- Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams.
Research and Innovation
- Stay updated with emerging technologies and trends in data engineering and integrate innovative solutions to enhance data processing and analysis capabilities.
Project Management and Delivery
- Lead data engineering projects from inception to completion, ensuring timely delivery, budget adherence, and quality standards.
Stakeholder Management and Reporting
- Engage with key stakeholders to understand their data requirements, delivering insightful reports and analyses to support decision-making.
Qualifications
Skills and attributes for success
- Bachelors degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience.
- 3+ years of experience in a Data Engineering role.
- Proficiency in SQL and programming languages like Python, Java, and Scala.
- Hands-on experience with big data technologies like Hadoop, Spark and Flink.
- Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar.
- Strong understanding of data warehousing concepts, ETL processes, and data modeling.
- Experience with API development and integration with data services.
- Experience with cloud platforms like AWS, GCP.
- Knowledge in DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes.
- Experience with real-time data processing.
Technical stack
- Programming Languages: Python, Java, Scala, SQL, Bash
- Big Data Technologies: Hadoop, Spark, Flink
- Databases: MySQL, PostgreSQL, MongoDB, Cassandra, HBase, Redis
- Cloud Platforms: Azure, AWS
- API Development: RESTful APIs, GraphQL, OpenAPI
- Data Services: Kafka, RabbitMQ
- Containers: Docker, Kubernetes
- Monitoring: Prometheus, Grafana ELK, Cloud Native monitoring tools.
What we look for
If you are a performance-driven, inquisitive mind with the agility to adapt to ambiguity, you will fit right in. You should be eager to explore opportunities to build meaningful collaborations with stakeholders and aspire to create unique customer-centric solutions. Bias for action and a passion to conquer new frontiers in the AI space is at the heart of the Core42 community.
What working at Core42 offers
Culture: An open, diverse and inclusive environment with a global vision that encourages personal growth and focuses on ground-breaking, industry-first innovations.
Career: Outstanding learning, development & growth opportunities via structured training programs and innovative, high-tech projects.
Work-Life: A hybrid work policy to strike the perfect balance between office and home.
Rewards: A competitive remuneration package with a host of perks including healthcare, education support, leave benefits and more.
Related Jobs
- Permanent
Jumeirah Hotels & Resorts
Engineer - Jumeirah Group & Corporate
Dubai, United Arab Emirates
- Permanent
NOV
Lead Mechanical Engineer (O&G, Mechanical Equipment - Static & Rotary)
Dubai, United Arab Emirates
- Permanent
Johnson Controls
Project Development Engineer, HVAC - UAE Nationals Only
Dubai, United Arab Emirates
- Permanent
KBR, Inc.
Senior HSE (Pipeline) Engineer (Project) (Offshore Site)
Abu Dhabi, United Arab Emirates
- Permanent
Emirates Airlines and Aviation
Senior Technical Engineer - Integration (UAE Nationals Only)
Dubai, United Arab Emirates
.jpeg?Expires=1721611891&Signature=Ap-zY~19c2qT~K-o1zHQilJeaC-tNeG9v8fCOdV9nYG8fNVxZ7fYOwFqU6XGCOouDEPtrsyFLDKZ5WWl~cOgWaSEMjiw68ss8jK9q-qYdSP3bT2EbzormYTsLIESTsSt0tHRewNghg6K5CfYdOIw9sfgtgxpVZ2l-H89IRyPfA75Sv5ukdv5UuWgo6kudi8e7fX0iiEAJJ0NmC6XMOjLrUcilo6gwzYB0PZZB2A4rYH0mn6h4Pc~f9T5gxEPZtbcprPvRhI16P1DDbl6N4GRBIC234Qglwoi6QL7~~sYmo4paGXd6YclfnxM0up6i-bmrHSpKoMwLXlhp82Lgnkn9A__&Key-Pair-Id=K2NS64SN1PHYK3){kind=logo}
- Permanent
Al-Futtaim
Sr. Mechanical Engineer I Real Estate I Al Futtaim Engineering
Dubai, United Arab Emirates
- Permanent
GE Renewable Energy
TareeGE Graduate Program - Customer Support Engineer
Dubai, United Arab Emirates
- Permanent
Parsons Corporation
Health, Safety and Environmental Engineer
Abu Dhabi, United Arab Emirates
- Permanent
Emirates Airlines and Aviation
Vice President - Engineering Projects
Dubai, United Arab Emirates
