PySpark Developer Job at VSG Business Solutions LLC, Canada

aEJaQWxpY2x4NzRhUk1kU3l5RG5xSldFSmc9PQ==
  • VSG Business Solutions LLC
  • Canada

Job Description

Responsibilities:

  • Develop, maintain, and optimize data processing pipelines using PySpark on Azure Cloud.
  • Collaborate with cross-functional teams to understand data requirements and translate business needs into technical specifications.
  • Design and implement scalable data models and data architecture to support analytical and reporting needs.
  • Perform data cleansing, transformation, and enrichment processes to enhance data quality and usability.
  • Monitor and troubleshoot data processing jobs, ensuring high availability and performance.
  • Implement best practices for data governance and security within Azure Cloud.
  • Stay up-to-date with emerging technologies in big data, cloud computing, and the financial services industry.

Requirements:

  • Minimum of 9 years of experience in PySpark development.
  • Proven experience working with Azure Cloud services (Azure Data Factory, Azure Databricks, etc.).
  • Prior experience in the banking, investment banking, or financial services sector is mandatory.
  • Proficient in PySpark for data processing and ETL tasks.
  • Experience with SQL and NoSQL databases.
  • Familiarity with data warehousing concepts and tools.
  • Knowledge of Azure services related to big data and analytics.

Preferred, but not required:

  • Experience with containerization technologies (e.g., Docker, Kubernetes).
  • Familiarity with machine learning frameworks (e.g., MLlib, TensorFlow).

Job Tags

Similar Jobs

St Jude Children's Research Hospital

Postdoctoral Research Associate - Hematology Job at St Jude Children's Research Hospital

 ...regulation of the bone marrow niche. The major research goal of our group is to gain mechanistic insights into cellular cross communication between bone marrow microenvironment and hematopoietic compartment, in steady state and pathophysiological conditions. You... 

University Tire & Auto Service

Body Shop Manager/Estimator Job at University Tire & Auto Service

 ...repair and maintenance to its customers throughout the Los Angeles area for over 33 years. The company is seeking a motivated Body Shop Manager / Estimator to join our team! The position is to be responsible for working closely with customers and staff members in order... 

Matthews Memorialization

Customer Service Representative ( Email Team) Job at Matthews Memorialization

Customer Service Representative ( Email Team) Location Aurora, IN : POSITION SUMMARY The Customer Service Representative is the communication link between customer/sales/production and provides high level, professional customer service by responding to customer inquiries... 

Guardian Angel Senior Services

Senior citizen caregiver HHA. Ipswich, Ma. Flexible schedule. Job at Guardian Angel Senior Services

 ...; Exciting Opportunity for Experienced Caregivers in Ipswich, MA! &##127775; Are you a compassionate caregiver with a heart for seniors? Were seeking dedicated individuals for a fulfilling role in the Ipswich area. While HHA and/or CNA certification is strongly preferred... 

MultiTech Systems

VIRTUAL DATA ENTRY ASSISTANT - FULL REMOTE Job at MultiTech Systems

We are seeking an entry level Data Entry Specialist. We are looking for a candidate with strong Excel knowledge and ability to think outside the box. The ideal candidate would have at least one year of data entry experience and can commit to a long-term contract. ...