Explore exciting opportunities at Xtedder and join our dynamic team!

Job Description:

Estamos à procura de um Senior Spark/PySpark Engineer para integrar a equipa de Data Engineering Analytics de uma empresa líder no setor da Energia. O foco será na modernização de pipelines ETL, curadoria de dados e desenvolvimento de estruturas de qualidade de dados usando tecnologias Spark e Databricks.

Responsibilities:

Refactor and modernize existing Python and PySpark ETL pipelines and Delta table structures.

Design and implement curated datasets for business analytics use cases.

Develop and enhance a Data Quality framework using the Great Expectations library with Spark.

Collaborate closely with cross-functional teams in a hybrid environment, applying software engineering best practices to data workflows.

Requirements:

•To focus on reworking and modernizing our python and pyspark ETL library and Delta tables. 

- Possibly leaning more toward software engineer than strict data engineer. 

- If they have Databricks experience that would be great. 

•To focus on data engineering to bring in new tables and curate them for end users. 

- Candidate needs to have strong python/pyspark and SQL knowledge. 

•To focus on building out our Data Quality framework with the Great Expectation spark library. 

Shape your future with us @Xtedder

Av. Engenheiro Duarte Pacheco Amoreiras
Torre 1, Piso 3 S9
1070-101
Lisbon, Portugal