Job Details
Peak Partner s client, a global software development company, is looking to hire a Lead Data Engineer to join their team to work on innovative technologies in the Data Engineering space.
This position is mainly remote, but candidates must be local to the Baltimore County area and
report into the office as needed.
Responsibilities:
- Design, develop, and support scalable data processing pipelines using Apache Spark and Java/Scala.
- Lead a talented team and make a significant impact on the client s data engineering capabilities.
- Implement and manage workflow orchestration with Airflow for efficient data processing.
- Proficiently use SQL for querying and data manipulation tasks.
- Collaborate with cross-functional teams to gather requirements and ensure alignment with data engineering solutions.
Requirements:
- Bachelor s degree in computer science or another relevant discipline, and a minimum of five years relevant experience in Data Engineering.
- Solid experience with Apache Spark for large-scale data processing.
- Proficiency in Java or Scala programming languages.
- Strong knowledge of Airflow for workflow orchestration.
- Proficient in SQL for data querying and manipulation.
- Familiarity with Avro and Schema Registry for efficient data serialization.
- Experience with containerization technologies such as Docker and orchestration tools like Kubernetes (K8S).
- Hands-on experience with cloud-based data engineering services, preferably Azure Data Factory or AWS Glue.
- Understanding of release management processes.
- Familiarity with version control systems such as Git and experience with release pipelines using Azure DevOps or similar tools.
If you think you have the qualifications stated above and are interested in exploring a new opportunity, we will welcome the chance to review your resume and talk further.