JOB SUMMARY :
The Data Engineer will design, develop, and maintain data pipelines and infrastructure to support business intelligence, analytics, and machine learning initiatives. This role requires a deep understanding of data integration, data modeling, and performance optimization to enable efficient data processing and analysis. The ideal candidate will ensure data availability, reliability, and security while collaborating with cross-functional teams to drive data-driven decision-making across the organization.
RESPONSIBILITIES :
- Develop, test, and maintain scalable ETL (Extract, Transform, Load) processes to integrate data from various sources.
- Build and optimize data pipelines and architectures for both structured and unstructured data.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
- Implement and maintain data security protocols to ensure compliance with industry standards and regulations.
- Perform data quality checks and ensure consistency, accuracy, and reliability of datasets.
- Monitor and troubleshoot data pipelines, resolving any issues to maintain data flow.
- Design and implement database schemas and data storage solutions for performance optimization.
- Stay updated on emerging technologies and trends in data engineering to recommend improvements.
QUALIFICATIONS :
-
Education Required :
- Bachelor’s degree in computer science, Information Systems, Engineering, or a related field.
- Master’s degree or certifications in Data Engineering, Big Data, or Cloud Platforms is a plus.
-
Skill Required :
- Proficiency in programming languages such as Python, Java, or Scala.
- Strong expertise in SQL for data manipulation and querying.
- Experience with data pipeline and workflow management tools (e.g., Apache Airflow).
- Knowledge of big data frameworks (e.g., Hadoop, Spark) and cloud platforms (e.g., AWS, Azure, GCP).
- Familiarity with data modeling, data warehousing, and ETL/ELT processes.
- Understanding of database systems (e.g., Oracle, Microsoft SQL Server, PostgreSQL, NoSQL databases like Redis or MongoDB).
- Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
- Excellent problem-solving and analytical skills.
- Strong communication skills to collaborate effectively with team members and stakeholders.
-
Experience :
- 0-3 years of experience in data engineering, software development, or a similar field.