Data Engineering Lead - 100% Remote
Data Engineering Lead 100% Remote Duration 12 months + Salary DOE US Citizens and those authorized to work in the U.S are encouraged to apply. We are unable to sponsor at this time.
Key Responsibilities
- Lead the definition and breakdown of technical deliverables, assessing complexities and levels of effort.
- Collaborate with vendor and internal teams to address technical issues.
- Drive planning and prioritization in conjunction with the SCRUM master.
- Coordinate with onsite and offshore teams for timely project delivery.
- Collaborate with product managers to prioritize features, communicate with stakeholders, and clarify requirements.
- Collect and manage user-reported issues, ensuring prompt resolution and communication.
- Define comprehensive test plans, oversee their execution, and document results.
- Prepare documentation and deliver user training to facilitate the adoption of new tools and features.
- Design and implement data architecture and strategy, leveraging technologies such as Databricks, Unity Catalog, Privacera, and Collibra.
- Develop, optimize, and manage data pipelines for ETL processes using Databricks, with a focus on data integrity and quality.
- Design and maintain data models and schemas, incorporating Unity Catalog and Collibra data governance practices.
- Ensure data security and compliance with regulations using Databricks and Privacera's features.
- Establish a robust data governance strategy, defining standards, metadata management, lineage, and quality practices.
- Operationalize Machine Learning models in Batch and Real Time Data Pipelines, leveraging relevant governance setups.
- Collaborate with cross-functional teams including data scientists, engineers, and analysts to translate business requirements into scalable solutions.
- Lead and guide a team of data engineers, providing technical direction and mentorship.
- Your expertise will be instrumental in shaping data-driven solutions, driving innovation, and ensuring the integrity and security of our data ecosystem. Your role will involve both technical leadership and hands-on contributions to deliver impactful results.