Data Pipeline Developer

For our business, for clients, and for you

General description of the job

As a Senior Data Engineer, you will be a driving force towards data engineering excellence. Working with other data engineers, analysts, and the architecture function, you’ll be involved in building a modern data platform using a number of cutting-edge technologies in a multi-cloud environment. You’ll have the opportunity to expand your knowledge and skills across multiple areas, including data pipelines, data integrations, data preparation, data models, and analytical and reporting marts.

As the business grows, we want our staff to grow with us, so there will be plenty of opportunities to learn and upskill. While much of the work follows business requirements and design concepts, you will also play a key role in the continuous improvement of design patterns and automation processes.

Responsibilities

• Help break down large problems into smaller iterative steps.
• Contribute to defining the prioritisation of your squad’s backlog.
• Build the modern data platform (data pipelines, data integrations, data preparation, data models, analytical and reporting marts) based on business requirements using agreed design patterns.
• Determine the most appropriate tools, methods, and design patterns to meet business requirements.
• Proactively identify and suggest improvements.
• Prepare data for exposure through APIs.
• Document, support, manage, and maintain the modern data platform built within your squad.
• Provide guidance and training to downstream data consumers on best practices.
• Support and build new data APIs.
• Promote the effective use and value of data across the organisation.
• Comply with all company policies.
• Perform any other duties assigned by management.

Requirements & Skills

• at least 5 years of experience.
• Experience with Data Warehousing and Data Modelling.
• Experience with Data Lakes (AWS Lake Formation, Azure Data Lake).
• Experience with Cloud Data Warehouses (AWS Redshift, Azure Synapse, Snowflake).
• Experience with ETL/ELT/Pipeline tools (AWS Glue, Azure Data Factory, Fivetran, Stitch).
• Knowledge of messaging systems (AWS SNS & SQS, Azure ASQ, Kafka, RabbitMQ).
• Programming skills in SQL, Python, Scala, or Java.
• Experience with cloud workflow services (AWS Step Functions, Azure Logic Apps, Camunda).
• Experience with event and schedule management (AWS Lambda, Azure Functions).
• Knowledge of the Microsoft BI Stack (SQL Server, SSIS, SSAS, SSRS).
• Experience with reporting and visualisation tools such as Power BI, QuickSight, or Mode.
• Knowledge of NoSQL and Graph Databases (AWS Neptune, Azure Cosmos DB, Neo4j) – desirable.
• API Management experience – desirable.
• Excellent communication and interpersonal skills.
• Strong critical thinking, analytical, and problem-solving skills.
• Ability to prioritise and manage workload independently.
• Ability to work both independently and collaboratively within a team.
• Working knowledge of Agile Software Development Lifecycles.

Disclaimer: Unsolicited CVs sent to Apex (Talent Acquisition Team or Hiring Managers) by recruitment agencies will not be accepted for this position. Apex operates a direct sourcing model and where agency assistance is required, the Talent Acquisition team will engage directly with our exclusive recruitment partners.