Job Description
The Company
Our client is the largest independent provider of payment solutions for the Australian financial services sector. For more than 50 years they have provided their clients with innovative and reliable payment solutions.
The Opportunity
The Data Engineer role is responsible for the ongoing enhancement & development of solutions within the client's DataLake & Analytics systems. This role also has responsibility for input into project initiatives to enhance delivered technology services.
Key Accountabilities of data engineer:
Develop & enhance the extensible corporate framework at the heart of the Databricks DataLake. Mostly Python and AWS components.
Produce the infrastructure for extraction, transformation, and loading (ETL) of data from a wide variety of data sources using 'Big Data' technologies
Create and maintain Data pipelines in an efficient manner - Maintain, troubleshoot, and improve the workflow and job scheduling of ETL processes
Assist in the design & development of analytic solutions. Mostly in Databricks SQL Analytics &/or PowerBI.
Ensure all designs & software are optimised to minimise cloud costs.
Manage and process complex, large data for analysis (e.g. hierarchies of datasets, inter-relationships, partitions, views, etc.)
Maintaining a high level of system availability, system monitoring, fault diagnosis, and change implementation and review
Analyse and resolve issues arising from the Client's datalake solutions and/or dataset
Track, analyse and review trends in commonly occurring support issues, and identify risk management strategies.
The ideal candidate for the data engineer role will have:
5+ years in Data Engineering - Strong knowledge in Databricks, DataLake, and Data Warehousing techniques.
Experience in one or more programming languages, Python preferred.
Experience with Oracle and/or SQL databases and writing SQL.
Knowledge and understanding of Databricks tuning and functions.
Experience in Analytics; Databricks SQL Analytics &/or PowerBI preferred.
Experience in scripting & Infrastructure as Code; eg: PowerShell, Terraform is desirable.
Experience in the financial services or payments sector is highly regarded.
Experience working within PCI-DSS compliant environment is desired.
Why Apply?
Professional development budget, training plan, and career guidance.
Inclusive Work Culture.
Flexible work arrangement.
Your next steps
If you would like to work for a company that places enormous value on their employees, then please apply now. Alternatively, please call Hima Arafath on 0452089967 for a confidential conversation.
Whether you are a sports fanatic, shopaholic, or part-time antique collector, we celebrate your uniqueness.
Parity Consulting works with clients who embrace diverse and inclusive environments and empower their teams to bring their authentic whole self to work. We encourage people with different beliefs, abilities, backgrounds, and life experiences to contact us
