We are sharing a specialised full-time consulting opportunity for data engineering professionals experienced in PySpark, SQL, distributed data processing, financial data infrastructure, pipeline development, and end-to-end data quality management.

This role supports an onsite engagement focused on building and maintaining the data infrastructure used by a fast-moving finance organization. Selected professionals will develop reliable data pipelines, optimize large-scale financial data workflows, resolve production issues, and collaborate with finance and engineering stakeholders to translate evolving business requirements into dependable technical solutions.

Key Responsibilities

Financial Data Pipeline Development

• Build and maintain PySpark-based pipelines supporting critical finance data workflows
• Develop scalable processes that deliver financial data accurately and on schedule
• Integrate information from multiple systems while maintaining consistent schemas and processing standards
• Improve pipeline reliability, observability, and operational performance across production environments

SQL Development & Data Validation

• Write and optimize SQL queries for large-scale financial datasets
• Join, transform, reconcile, and validate data across multiple sources
• Investigate discrepancies and ensure outputs meet accuracy and completeness requirements
• Develop repeatable validation checks supporting trusted financial analysis and operational use

Data Quality & Production Support

• Diagnose and resolve pipeline failures, processing delays, and data-quality issues
• Take ownership of incidents from initial investigation through final resolution
• Identify recurring failure patterns and implement durable technical improvements
• Maintain clear documentation covering pipeline behavior, dependencies, and operational procedures

Finance & Engineering Collaboration

• Partner with finance and technical stakeholders to clarify ambiguous data requirements
• Translate business priorities into practical data models, pipelines, and infrastructure
• Communicate technical tradeoffs, delivery risks, and implementation progress clearly
• Support a broad finance roadmap within a small, fast-moving engineering environment

Ideal Profile

Strong candidates may have:

• Approximately 2–4 years of professional data engineering experience
• Hands-on experience building and maintaining PySpark-based data pipelines
• Strong SQL skills across complex joins, transformations, aggregations, and validation workflows
• Practical knowledge of distributed or large-scale data-processing environments
• Experience troubleshooting production pipelines and owning data quality end to end
• Ability to work independently, prioritize effectively, and respond quickly to changing requirements
• Strong written communication and cross-functional collaboration skills
• Availability for a full-time onsite engagement in an approved location

Educational Background

• A bachelor's degree in computer science, software engineering, data engineering, information systems, or a related technical field is helpful
• Professional experience building production-grade data systems is highly relevant
• Equivalent hands-on experience in distributed processing, backend engineering, or data infrastructure may also be considered
• Additional training in cloud platforms, data architecture, or financial systems may be valuable

Nice to Have

• Experience supporting finance, accounting, forecasting, planning, or corporate reporting teams
• Familiarity with cloud-based data platforms and modern orchestration tools
• Experience designing monitoring, alerting, reconciliation, or automated data-quality frameworks
• Knowledge of dimensional modeling, warehouse architecture, and financial data structures
• Familiarity with performance tuning for Spark and large SQL workloads
• Experience working in a fast-growth technology environment with changing priorities
• Exposure to technical interviews or assessments focused on advanced SQL problem-solving

Why This Opportunity

• Build critical data infrastructure supporting high-priority finance operations
• Work directly with financial and engineering stakeholders on practical business needs
• Apply PySpark and SQL expertise to large-scale, production-oriented data workflows
• Take meaningful ownership of pipeline reliability, data accuracy, and technical delivery
• Join a focused onsite engagement with competitive hourly compensation and potential extension

Contract Details

• Full-time W-2 contingent employment arrangement
• Onsite work required in San Francisco, California; New York, New York; or Bellevue, Washington
• Initial engagement duration of approximately six months
• Potential extension depending on performance and business requirements
• Competitive rates between $55–$80 per hour depending on experience and technical depth
• Candidates must be based in the United States and authorized to work in the applicable location
• The selection process may include two technical assessments focused strongly on SQL
• This is a hands-on data engineering position centered on pipelines and infrastructure rather than business intelligence, dashboarding, or machine-learning research
• Work will not involve access to confidential or proprietary information from any employer, client, or institution

About the Platform

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.

By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy.