Data Engineer

About:

The Data Engineer extracts, combines, transforms, loads, manipulates and cleanses data all whilst effectively handling interfaces to large amounts of data and creates/integrates data structures suitable for analysis.

The Data Engineer must understand data architecture and data management. Develop and deploys algorithms in performance-efficient code in analytical and data platforms. Liaise with enterprise architects, data governors, and subject matter experts to fill any gaps in the requirements or design.

Core Responsibilities:

• Works and experiments with state-of-the-art data storage and processing technologies.

• Builds large-scale batch and real-time data pipelines with data processing frameworks in cloud platforms or on-premise

• Leverages best practices in continuous integration and delivery

• Pre-process, clean and structures data to facilitate data exploration and advanced analytics/Machine Learning activities

• Learns and uses modern data preparation, integration and AI-enabled metadata management tools and techniques.

• Tracks data consumption patterns, working with data governance and information stewards within these teams, and participates in vetting and promoting content created in the business and by data scientists to the curated data catalogue for governed reuse

• Performs intelligent sampling and caching.

• Recommends and implements automation in existing and future integration flows.

• Helps improve data quality by driving advancements in data acquisition, testing and tooling

• Works in cross-functional agile teams to continuously experiment, iterate and deliver on new product objectives

• Analyzes requested changes to fill in the details fitting in the complete picture and while doing that aligns with all parties involved (Architects, Data Governance, …), either as preparation for yourself or for data engineer colleagues, before starting the implementation.

Education & Experience:

• Bachelor’s or Master’s Degree in Computer Science preferred or equivalent combination of education and experience

• Minimum 4 years of experience as a data engineer

• Experience in (captive) financial services and understands the processes and execution models in financial services is an asset

• Experience in Azure SQL, Azure Data Factory, Azure DevOps, Databricks, Delta Lake is a must, certification is recommended

• Experience in working with high-volume heterogeneous data sets 2

• Experience in agile software processes and data-driven development

• Experience with data lakes and dimensional models

Required Competencies and Skills:

• Azure Data Engineer Associate certification is recommended

• Databricks Data Engineer Professional certification is recommended

• Strive for clean, reusable and performant code

• Skilled ETL/ELT developer

• Strong understanding of data and information

• Business savvy, understand information from a business perspective and use that to more effectively develop enabling technologies

• Strong problem-solving and analytical skills

• Goal-oriented and positive attitude to deliver against targets

• Able to work independently and proactively

• Global, open, innovative, holistic and goal-oriented mindset

• Ability to set and meet deadlines

• Ability to work in high-pressure situations

• Excellent attention to detail

• Advanced level of English, oral and written

If you are interested in this position or know someone who would be perfect for this role please contact us to find out all details of the position!