CrawlJobs Logo

Big Data / PySpark Engineering Lead - Vice President

https://www.citi.com/ Logo

Citi

Location Icon

Location:
India , Pune

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

The Applications Development Technology Lead Analyst is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications systems analysis and programming activities.

Job Responsibility:

  • Design and implement scalable, fault-tolerant batch and real-time data processing pipelines
  • Develop robust data models and schema designs optimized for both performance and storage efficiency
  • Evaluate and integrate emerging tools and frameworks (e.g., Spark, Flink, Kafka) into the existing stack
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Legacy Systems Decommissioning: Lead the strategic migration of data and logic from legacy platforms (e.g. on-premises SQL Servers) to a modern Data Lakehouse environment
  • ETL/ELT Transformation: Re-engineer existing stored procedures and complex legacy ETL jobs into scalable, distributed processing frameworks using Spark (Python) and Starburst/Trino
  • Validation & Parity Testing: Design and implement automated frameworks for Data Parity Testing to ensure 100% accuracy and consistency between legacy outputs and new big data results
  • Schema Evolution: Map and transform rigid, legacy relational schemas into flexible, high-performance formats optimized for the cloud (e.g., Parquet, Avro, or Iceberg)
  • Phased Cutover Management: Orchestrate a phased migration strategy (Parallel Run, Shadow Execution) to ensure zero downtime for downstream business applications and reporting tools
  • Performance Benchmarking: Establish performance baselines on legacy systems and ensure the new Big Data architecture meets or exceeds those benchmarks at scale
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Write clean, high-performance code in Python
  • Optimize complex SQL queries and fine-tune distributed computing clusters to reduce latency and costs
  • Ensure data integrity and security by implementing rigorous validation and encryption standards
  • Build and maintain CI/CD pipelines for automated testing and deployment of data jobs
  • Monitor system health and troubleshoot performance bottlenecks across the data lifecycle
  • Provide technical mentorship and conduct code reviews for junior and mid-level engineers
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Translate complex business requirements into technical specifications
  • Collaborate with Product Managers to ensure data availability for downstream analytics, business models and users
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements

Requirements:

  • Highly experienced and skilled technical lead with 12+years of experience with software building and platform engineering
  • Experience in Data Engineering, focused on Big Data ecosystems
  • Knowledge in Hadoop, YARN, Hive, Impala, Spark, and Spark SQL with extensive high volume of data processing pipeline development
  • Programming Expert level and hand on experience in Python
  • Familiarity with data formats like Avro, Parquet, CSV, JSON
  • Hands-on experience in writing SQL queries
  • Highly experienced with Unix based operating systems and shell scripting
  • Experience with source code management tools such as Bitbucket, Git etc
  • Big Data Tech Proficiency and hands-on in Hadoop, Spark, Hive, Kafka, and NoSQL databases (MongoDB, HBase)
  • Experience working with query engines like Trino, Presto, Starburst
  • Strong computer science fundamentals in data structures, algorithms, databases, and operating systems
  • Reverse Engineering, ability to read "spaghetti" SQL or old scripts and document the business logic before moving it
  • Data Lineage, Experience using tools (like Collibra or Informatica) to track where data comes from and where it’s going
  • Change Management, Experience managing the technical "shock" to the business when switching from legacy BI tools to modern query engines like Starburst

Nice to have:

  • Problem Solver: You don't just fix bugs
  • you identify the root cause to prevent recurrence
  • Communicator: You can explain the "why" behind a technical decision to non-technical stakeholders
  • Automation and AI Mindset: You believe that if a task has to be done twice, it should be automated. Familiarity with AI tools to expedite deliveries

Additional Information:

Job Posted:
March 01, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Big Data / PySpark Engineering Lead - Vice President

Data Analytics Senior Analyst - Assistant Vice President

The Data Analytics Senior Analyst is a seasoned professional role. Applies in-de...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years experience using tools for statistical modeling of large data sets
  • Database Development & Architecture:Design, develop, and maintain complex database solutions across MongoDB, Oracle, and other SQL databases. Create optimal data models, schemas, and stored procedures to support high-throughput applications
  • Data Pipeline Construction:Build and manage resilient, scalable ETL/ELT pipelines using Python to process and integrate large volumes of data from diverse source systems into our core data platforms
  • Big Data Engineering:Engineer and implement solutions within our Big Data ecosystem (e.g., Hadoop, Spark, Hive, Kafka) to handle large-scale data processing, batch analytics, and real-time data streams
  • Python Development:Write high-quality, production-ready Python code for data manipulation, API development, and automation. Utilize a range of libraries and frameworks relevant to data engineering (e.g., Pandas, PySpark, SQLAlchemy, PyMongo)
  • Performance Optimization:Proactively monitor, troubleshoot, and optimize the performance of our databases and data pipelines. Focus on query tuning, indexing strategies, and resource management to ensure low-latency data access
  • Data Quality and Integrity:Implement data quality checks, validation rules, and monitoring frameworks within the data pipelines to ensure the accuracy, consistency, and reliability of our KYC data
  • Collaboration:Work closely with application developers, data scientists, and data analysts to understand their data requirements and provide robust, well-documented data solutions and services
  • Technical Leadership:Provide subject matter expertise on database and data engineering best practices. Mentor junior engineers and contribute to a culture of technical excellence
Job Responsibility
Job Responsibility
  • Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flows
  • Coordinates and contribute to the objectives of data science initiatives and overall business through leveraging in-depth understanding of how areas collectively integrate within the sub-function
  • Assumes informal/formal leadership role through coaching and training of new recruits
  • Significantly influences decisions, work, and performance of all teams through advice, counsel and/or facilitating services to others in the business
  • Conducts strategic data analysis, identifies insights and implications and make strategic recommendations, develops data displays that clearly communicate complex analysis
  • Mines and analyzes data from various banking platforms to drive optimization and improve data quality
  • Delivers analytics initiatives to address business problems with the ability to identify data required, assess time & effort required and establish a project plan
  • Consults with business clients to identify system functional specifications. Applies comprehensive understanding of how multiple areas collectively integrate to contribute towards achieving business goals
  • Consults with users and clients to solve complex system issues/problems through in-depth evaluation of business processes, systems and industry standards
  • recommends solutions
  • Fulltime
Read More
Arrow Right

Data Analytics Senior Analyst - Assistant Vice President

The Data Analytics Senior Analyst is a seasoned professional role. Applies in-de...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years experience using tools for statistical modeling of large data sets
  • Database Development & Architecture:Design, develop, and maintain complex database solutions across MongoDB, Oracle, and other SQL databases. Create optimal data models, schemas, and stored procedures to support high-throughput applications
  • Data Pipeline Construction:Build and manage resilient, scalable ETL/ELT pipelines using Python to process and integrate large volumes of data from diverse source systems into our core data platforms
  • Big Data Engineering:Engineer and implement solutions within our Big Data ecosystem (e.g., Hadoop, Spark, Hive, Kafka) to handle large-scale data processing, batch analytics, and real-time data streams
  • Python Development:Write high-quality, production-ready Python code for data manipulation, API development, and automation. Utilize a range of libraries and frameworks relevant to data engineering (e.g., Pandas, PySpark, SQLAlchemy, PyMongo)
  • Performance Optimization:Proactively monitor, troubleshoot, and optimize the performance of our databases and data pipelines. Focus on query tuning, indexing strategies, and resource management to ensure low-latency data access
  • Data Quality and Integrity:Implement data quality checks, validation rules, and monitoring frameworks within the data pipelines to ensure the accuracy, consistency, and reliability of our KYC data
  • Collaboration:Work closely with application developers, data scientists, and data analysts to understand their data requirements and provide robust, well-documented data solutions and services
  • Technical Leadership:Provide subject matter expertise on database and data engineering best practices. Mentor junior engineers and contribute to a culture of technical excellence
  • Bachelor’s/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flows
  • Coordinates and contribute to the objectives of data science initiatives and overall business through leveraging in-depth understanding of how areas collectively integrate within the sub-function
  • Assumes informal/formal leadership role through coaching and training of new recruits
  • Significantly influences decisions, work, and performance of all teams through advice, counsel and/or facilitating services to others in the business
  • Conducts strategic data analysis, identifies insights and implications and make strategic recommendations, develops data displays that clearly communicate complex analysis
  • Mines and analyzes data from various banking platforms to drive optimization and improve data quality
  • Delivers analytics initiatives to address business problems with the ability to identify data required, assess time & effort required and establish a project plan
  • Consults with business clients to identify system functional specifications. Applies comprehensive understanding of how multiple areas collectively integrate to contribute towards achieving business goals
  • Consults with users and clients to solve complex system issues/problems through in-depth evaluation of business processes, systems and industry standards
  • recommends solutions
  • Fulltime
Read More
Arrow Right

Fullstack Big Data Developer Application Development Technical Lead Analyst Vice President

Discover your future at Citi. Working at Citi is far more than just a job. A car...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of Application development experience
  • 6+ years of experience in full stack development, with a focus on Bigdata and Python/Scala
  • 6+ years experience with big data technologies such as Python, Pyspark, Hadoop, Kafka, etc.
  • Experience with Core Java/J2EE Application with complete command over OOPs and Design Patterns
  • Commendable in Data Structures and Algorithms
  • Worked on Core Application Development of complex size encompassing all areas of Java/J2EE
  • Thorough knowledge and hands on experience in following technologies Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig , Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
  • Should have implemented or part complex project execution in Big Data Spark eco system, where processing volumes of data thorough understanding of distributed processing and integrated applications
  • Exposure to ETL and BI tools
  • Work in an agile environment following through the best practices of agile Scrum
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Design, develop, and maintain scalable and robust architecture for the project using Java/Python/Scala and other full stack technologies
  • Manage big data technologies such as python, pyspark to ensure seamless data integration, storage, and analysis
  • Fulltime
Read More
Arrow Right
New

Data Analytics Lead - Data Scientist - Vice President

The Data Analytics Lead / Data Scientist is a strategic professional who stays a...
Location
Location
India , Chennai; Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10-15 years of relevant experience in Data Analytics, Data Science, or Advanced Analytics roles
  • Advanced proficiency in SQL and relational database concepts
  • Strong programming experience in Python (required)
  • PySpark preferred
  • Hands-on experience building and deploying machine learning models (supervised and unsupervised)
  • Experience with ML libraries such as scikit-learn, XGBoost, TensorFlow, or PyTorch
  • Strong knowledge of statistical modeling, feature engineering, and model validation techniques
  • Experience with BI tools such as Tableau or Power BI
  • Familiarity with MLOps practices (model deployment, monitoring, versioning) is strongly preferred
  • Experience working with large-scale enterprise or financial datasets
Job Responsibility
Job Responsibility
  • Integrates subject matter and industry expertise within a defined area
  • Contributes to data analytics standards around which others will operate
  • Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinate and contribute to the objectives of the entire function
  • Employs developed communication and diplomacy skills are required in order to guide, influence and convince others, in particular colleagues in other areas and occasional external customers
  • Resolves occasionally complex and highly variable issues
  • Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/ taken
  • Responsible for volume, quality, timeliness and delivery of data science projects along with short-term planning resource planning
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Lead the design and execution of complex data analysis and AI/ML initiatives across large, structured, and unstructured datasets
  • Develop and deploy predictive, classification, clustering, and forecasting models to support business strategy and risk management
  • Fulltime
Read More
Arrow Right
New

Sales Advisor

We’re looking for a Sales Advisor to join our team at Vertu Nissan Stockton. Thi...
Location
Location
United Kingdom , Stockton-on-Tees
Salary
Salary:
28000.00 - 34000.00 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full driving licence (manual and automatic)
  • Ability to demonstrate great customer service
  • Strong communication skills
  • Confidence using technology
Job Responsibility
Job Responsibility
  • Support customers through their entire car-buying journey
  • Arrange test drives
  • Help customers select the right finance
  • Guide customers through accessories
  • Ensure every customer has a fantastic experience
  • Engage with customers online, in-store, on the phone or by video, nationally as well as locally
What we offer
What we offer
  • 25 days holiday rising with length of service - plus bank holidays
  • Access to online rewards platform giving cash back and discounts for multiple retailers
  • Preferential Service Rates
  • Colleague Purchase Scheme
  • Share Incentive Scheme
  • Pension
  • Enhanced Maternity and Paternity
  • Full training and comprehensive onboarding program
  • Opportunity to transfer to Sales Executive after 6-months with on-target earnings to £45,000+ plus a Company Car
  • True work-life balance
  • Fulltime
Read More
Arrow Right
New

Senior HR Business Partner

We are looking for an experienced Senior HR Business Partner to join our team in...
Location
Location
United States , Greenwood Village
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 5 years of experience in human resources roles, preferably within hyper-growth companies
  • Proven expertise in employee relations and handling workplace challenges
  • Strong knowledge of HR policies, procedures, and compliance standards
  • Experience in performance management, including setting goals and conducting reviews
  • Demonstrated ability to lead change management initiatives within organizations
  • Proficiency in onboarding and organizational development practices
  • Strong analytical skills for workforce analysis and strategic planning
  • Excellent communication and interpersonal skills to foster collaboration and engagement
Job Responsibility
Job Responsibility
  • Lead employee relations efforts by addressing concerns and resolving workplace issues effectively
  • Develop, implement, and refine HR policies to ensure compliance and alignment with organizational goals
  • Oversee performance management processes, including setting objectives, monitoring progress, and conducting reviews
  • Drive change management initiatives to support organizational transformation and growth
  • Collaborate on HR strategy development, ensuring alignment with business objectives
  • Facilitate onboarding processes to ensure new hires integrate successfully into the organization
  • Support organizational development by identifying and addressing workforce needs
  • Conduct workforce analysis to optimize resource allocation and improve efficiency
  • Manage internal communications to promote transparency and employee engagement
  • Assist in restructuring efforts and guide teams through transitions as needed
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right
New

Software Engineer

OneDrive and SharePoint are rapidly growing services at the center of Microsoft'...
Location
Location
United States , Redmond
Salary
Salary:
84200.00 - 165200.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, or related technical discipline with proven experience coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Experience collaborating with peers and partner teams to meet joint engineering goals
  • Solid coding, debugging, algorithm design, and problem-solving skills
  • Experience with cloud-scale services and server/service management features
  • Able to learn new systems quickly and adapt new methodologies to our services
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • You will work in large-scale distributed systems that are critical for customers around the world
  • You will design and deliver services that manage internet-scale data stores, enable best-in-the-world file and site browsing and editing performance, or add innovative features to how people manage and interact with their data and telemetry
  • Given the scope of OneDrive and SharePoint, we have positions that range from the very front end of how a customer interacts with our services, through core system logic, to mega-scale storage, to background infrastructure to support our services and the engineers that build and maintain them
  • Everything you do will be supported by a world-class engineering team
  • As an engineer in our team, you will contribute to and help shape the vibrant, inclusive engineering culture of OneDrive & SharePoint
  • You will be expected to do deep, data-driven, collaborative design for additions and changes to our products
  • You will write clean, efficient code and comprehensive tests
  • You will build excellent coverage with metrics and telemetry to ensure we understand exactly what is happening with our services at all times
  • You will deliver features that will support and serve hundreds of millions of customers around the world
  • Fulltime
Read More
Arrow Right
New

Service Provider Manager

As a Service Provider Manager, you'll be a key player in our team, acting as the...
Location
Location
Poland , Bydgoszcz
Salary
Salary:
Not provided
zalando.de Logo
Zalando
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Skilled at delivering results with minimal supervision in a dynamic and often ambiguous environment
  • Strong business and logistics acumen with critical thinking and exceptional problem-solving skills
  • Strong understanding of customer care, e-commerce supply chains, warehousing and outsourcing mechanisms
  • Experience in data analysis using PostgreSQL
  • Proficient in Google Sheets for building reports and tools
  • Ability to translate on-the-floor observations into hard data and clear visualizations
  • Excellent communication skills, capable of having challenging and constructive conversations
  • Ability to leverage relationships to achieve desired outcomes
  • Hands-on mentality, enjoy working in a fast-paced environment dealing with many topics simultaneously
  • Excellent communication skills in both Polish and English
Job Responsibility
Job Responsibility
  • Proactively manage and anticipate issues, preventing or mitigating deviations in budget and quality
  • Act as a trusted partner for both internal and external stakeholders, communicating clearly and effectively
  • Work comfortably in a matrix organization, leading cross-functional teams
  • Use a data-driven approach to tackle complex challenges
  • Apply learnings from past experiences and identify patterns to solve new problems
  • Challenge existing processes to raise standards and improve performance
  • Manage partners based on defined SLAs and KPIs
  • Ensure all regulations and company guidelines are followed
  • Proactively identify and escalate challenges and risks
  • Create and deliver new metrics using available data to gain deeper insights and drive continuous improvement
What we offer
What we offer
  • Peer-to-peer performance reviews twice a year
  • Extensive Zalando training platform
  • Regular all-hands, team meetings, Q&A-sessions, and quarterly anonymous employee surveys
  • Support from an international team of experts
  • mentoring and professional development opportunities
  • 2 days paid leave per year for volunteering
  • Diverse sports and health offerings - like Multisport, private medical care or prepaid sport/lunch card
  • Mental wellbeing support by Employee Assistance Program and professional consultants
  • Access to the employee shares program
  • Shopping discount on products shipped and sold by Zalando, and discount on Zalando Lounge
  • Fulltime
Read More
Arrow Right