Data Engineer

This role can be based out of our London or Haywards Heath office - with flexibility for remote working arrangements.

What you’ll be doing

  • As a member of the Data Team within Architecture, you will be responsible for the design, implementation, and ongoing development of a new Data Lake-House, defining and implementing processes that support the Lake- House function including data shipment from production sources, ingestion/transformation, cataloguing, reporting and delivery of valuable data artefacts to various lines of business.
  • You will work with the Architecture, Engineering, Business and Product teams to build out the Lake-House capability to support priority business needs; building with the data sourced from the production/transactional platform.
  •  As the Data Engineer you will work closely with the Engineering Team and the Data Analysts, to ensure that the launch product has correct and complete data available to produce reports and statements and the successful operation of the analytical toolset.  Subsequent releases will extend the Data Lake-House into supporting Data Scientists, Machine Learning and Predictive Modelling. This is an opportunity to build the new foundation for a financial technology data platform
  • This role will be working for the Head of Data Architecture and directly alongside the Data Analysts and Data Scientists. A successful first 3 months will see a production-ready Data Lake-House, extending from the S3 T1 (at the Engineering team's responsibility perimeter) into an analytic platform capable of supporting the Data Analysts producing predefined structured data outputs. This design is expected to include AWS Glue and QuickSight. Data will be Ingested from the production platform sources including the Event Bus (SNS), Postgres Databases, CloudWatch and various flat files in S3 buckets.

 What are we looking for in you:

  • Be the expert on what data is where at any time.
  • Have excellent interpersonal skills.
  • Strong verbal and written communication skills.
  • Be a strong self-organiser with high attention to detail.
  • Ability to adapt and prioritise, work under pressure, and meet deadlines.
  • Ability to think ahead and anticipate problems, issues, and solutions.

Necessary technical skills and experience required:

  • At least 4 years’ experience in working with or implementing Data Lake and/or Data Warehouse solutions using AWS capabilities and tools.
  • Degree in Computer Science or related discipline, or demonstrable Industry experience.
  • Demonstrable experience of designing, building or operating Data Warehouse, Data Lake or Lake-House solutions including data ingestion and replication, transformation from T1 through to T3 and implementing solutions that use the resulting data for reporting and analytics.
  • Work management through Agile/Scrum methodologies.
  • Understanding of Serverless and Event Driven concepts.

Advantage to have experience/certifications in the following areas:

  • Data modelling, profiling, and quality in production systems.
  • Financial services industry experience.
  • The AWS Ecosystem including S3, AWS GLUE, Postgres, DynamoDB, Athena, Lake Formation, Lambda, Python, Redshift, Kinesis, CloudWatch, DMS/CDC, QuickSight, SNS and SQS.
  • Best practices for data ingestion, storage (including tiering), file formats and structures such as Parquet, Avro, ORC.
  • Software development using CI/CD.
  • Certifications such as AWS Big Data Speciality, AWS Data Analytics Speciality.

Shieldpay is an equal opportunities employer. For Shieldpay building a fair and transparent workforce begins with the recruitment process that does not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, colour, nationality, ethnic or national origin, religion or belief, disability or age 

Additional information

  • Remote status

    Flexible remote

We usually respond within two weeks

Or, know someone who would be a perfect fit? Let them know!



We’re proud that we are able to offer a package of benefits that is above the UK fintech average, designed to look after our people at every stage of their journey with SPG. But don’t think we are resting on our laurels, we are constantly reviewing our offering, and listening to feedback, to make sure we are able to help and support our team…

  • Share options

    We are pleased to issue each new employee a package of share options which ensures that they really can be part of the Group's success.

  • Private medical insurance

    We offer cover for our employees with no financial excess, and coverage of pre-existing conditions, from what we believe is the industry leader in physical AND mental health cover.

  • Life assurance

    Our employees can rest assured that their loved ones will be supported should the worst happen.

  • Pension

    We have a matched workplace pension scheme via Nest

  • Interest-free season ticket loan

    It may not have had a big take-up in 2020 – but it’s there if our people need it.

  • Unlimited annual leave

    We did away with the old-school cap on annual leave allowances to ensure our employees are able to step away from the office and keep a healthy work/life balance. (and we underpinned this with a minimum amount to make sure that it is taken!)

  • Mental well-being days

    Our employees can take a day a month away from work without booking or need to seek approval – for those days when your head is just not in the right space.

  • Enhanced allowances

    From maternity leave to eyecare, we don’t just offer the basic – we make sure that we are able to go the extra mile and offer well above the average for our employees.

  • Mental well-being support

    We have a calendar of events throughout the year for our employees – as well as the offer of membership to download the Calm app for each employee.

  • Flexible working

    We were always flexible in our approach even before the pandemic – and we are pleased to treat our employees like real adults and judge them on the work they do – and not the hours that they have sat at their desk.

Workplace & culture

Our team is filled with ambitious, curious and entrepreneurial people. We hire and nurture inquisitive, creative minds looking to flourish in a supportive and collaborative environment, recognised by being placed within the top 50 startups to work for in 2021 by Tempo. 

We don’t like old fashioned corporate hierarchy. Instead we like to empower our people to make a change and be autonomous in their role with all the support you need with the other teams around you.

Already working at SPG?

Let’s recruit together and find your next colleague.


Applicant tracking system by Teamtailor