CrawlJobs Logo

PySpark Module Lead

https://www.soprasteria.com Logo

Sopra Steria

Location Icon

Location:
India , Bengaluru

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

PySpark Module Lead position at Sopra Steria, a major Tech player in Europe. The role is within the Engineering, Development, Applications department.

Requirements:

  • Good experience in applying best coding practices
  • Good understanding and ability to develop simple to medium applications
  • Strong learnability and analytical skills
  • Strong team player with excellent communication skills
  • Mandatory skills: AWS, PySpark, PL SQL, Java
  • AWS Skills are on S3, Lambda, Cloudwatch, Dynamo DB, SQS, SNS, CloudFormation, Stack, Serverless
  • Total Experience Expected: 04-06 years
  • B.Tech, MCA or equivalent degree

Nice to have:

Desirable skills: Pytest, Junit, Maven, Basic understanding of DevOps concepts

What we offer:
  • Committed to fighting against all forms of discrimination
  • Fosters a work environment that is inclusive and respectful of all differences
  • All positions are open to people with disabilities

Additional Information:

Job Posted:
January 31, 2026

Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for PySpark Module Lead

Pyspark Module Lead

Sopra Steria is seeking a PySpark Module Lead based in Bengaluru, India. The pos...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Python
  • PySpark
  • SQL
  • JavaScript
  • Proficient with Skywise platform and tools e.g. Contour, Code-Workbook, Code, Slate and Ontology
Job Responsibility
Job Responsibility
  • Designing and building solutions for Airbus on foundry
  • Data engineering and integration
  • Data analysis and visualization
  • Application development within Skywise Framework
  • Working closely with Product Owners and core Airbus business functions to understand processes, capture requirements, and solve business problems
  • Full delivery lifecycle for designing, implementing, testing, documenting, and supporting applications
  • CI/CD pipeline development and automated deployments
  • Troubleshooting application issues
  • Providing support on developed tools
What we offer
What we offer
  • Inclusive and respectful work environment
  • Open to people with disabilities
  • Fulltime
Read More
Arrow Right

Pyspark Module Lead

We are seeking a highly skilled and motivated Data Engineer to join our dynamic ...
Location
Location
India , Noida
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop
  • Proven expertise in designing and deploying data pipelines
  • Strong problem-solving skills and ability to work effectively in a collaborative team environment
  • Excellent communication skills and ability to translate technical concepts to non-technical stakeholders
Job Responsibility
Job Responsibility
  • Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines
  • Utilize PySpark for data processing, transformation, and preparation for model training
  • Leverage AWS EMR and S3 for scalable and efficient data storage and processing
  • Implement and manage ETL workflows using Streamsets for data ingestion and transformation
  • Design and construct pipelines to deliver high-quality training and inference datasets
  • Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities
  • Optimize and fine-tune pipelines for performance, scalability, and reliability
  • Ensure IAM policies and permissions are appropriately configured for secure data access and management
  • Implement Spark architecture and optimize Spark jobs for scalable data processing
What we offer
What we offer
  • All positions are open to people with disabilities
  • Commitment to fighting against all forms of discrimination
  • Inclusive and respectful work environment
Read More
Arrow Right

Senior Software Engineer

The Senior Software Engineer will lead a team of data engineers to design, build...
Location
Location
United States , San Diego
Salary
Salary:
Not provided
gomillenniumsoft.com Logo
MillenniumSoft Inc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in developing data solutions
  • Proven experience in designing and implementing large-scale data pipelines
  • Hands-on expertise with AWS cloud technologies and distributed systems
  • Strong programming skills in PySpark, Python, and SQL
  • Data pipeline development using AWS tools (Step Functions, Airflow, etc.)
  • Distributed systems (S3, ADLS), and modern file/table formats (Parquet, Hudi)
  • ETL tools, orchestration technologies, and database design
  • Knowledge of regulatory standards like SOX, PCI, and HIPAA
  • Bachelor’s or Master’s in Computer Science, Information Systems, or related fields
  • AWS certification preferred
Job Responsibility
Job Responsibility
  • Design and implement scalable data pipelines for structured and unstructured data
  • Lead module architecture design for optimal performance, reliability, and maintainability
  • Mentor team members and ensure alignment with business goals
  • Collaborate with cross-functional teams to enable real-time and batch data processing
  • Ensure adherence to data security, privacy, and governance standards
  • Create and maintain technical documentation for architectures and decisions
  • Conduct code reviews and establish best practices
  • Develop monitoring systems for data pipeline reliability
  • Drive adoption of new technologies and workflows
  • Fulltime
Read More
Arrow Right
New

AI Lead, Client Technology - SVP

We are seeking a highly skilled and pragmatic AI Lead to design, develop, deploy...
Location
Location
United States , New York
Salary
Salary:
176720.00 - 265080.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
February 19, 2026
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Robotics, or a related quantitative field
  • 10+ years software engineering experience with recent hands-on coding, with a track record of rapid delivery and launching multiple AI features in production
  • Minimum 3+ years of professional experience in software development with a focus on AI, prompt engineering, machine learning and/or agentic AI systems
  • Strong proficiency in Python (FastAPI, Django, Flask, PySpark) or Java (Spring Boot, Spring Cloud, Spring Security), and SQL
  • JavaScript (React, Next.js, Node.js, TypeScript)
  • Full-stack development with a focus on rapid prototyping
  • Solid understanding of core AI concepts, including knowledge representation, automated planning, decision-making under uncertainty, and multi-agent systems
  • Experience with machine learning frameworks (e.g., TensorFlow, PyTorch) and relevant libraries (e.g., Scikit-Learn, NumPy, Pandas)
  • Familiarity with large language models (LLMs) like ChatGPT, LaMDA/Gemini, Llama, etc., and their application in agentic systems
  • Familiarity with specific agent frameworks (e.g., LangChain, AutoGen, CrewAI, RAG) or research in multi-agent reinforcement learning
Job Responsibility
Job Responsibility
  • Design, implement, and deploy intelligent agents, including perception, reasoning, planning, and action execution modules
  • Contribute to the development and implementation of generative AI solutions, ensuring they meet technical requirements and business objectives
  • Develop scalable and robust architectures for agentic systems and generative AI applications, ensuring high performance, reliability, and security
  • Integrate various machine learning models (e.g., LLMs, reinforcement learning, predictive models) to enhance agent capabilities and decision-making
  • Implement LLM integration using platforms like OpenAI, Anthropic, and Bedrock APIs
  • Develop agents that can automate complex tasks, optimize workflows, and solve real-world problems across various domains
  • MVP first approach, iterative improvement approach with a focus on 'time to value' (quick iterations, hypothesis testing, A/B experiments)
  • Utilize and contribute to agentic AI frameworks and development tools
  • Build full-stack applications that integrate existing ML/LLM tools and services
  • Design and implement metrics and evaluation strategies for agent performance, continuously optimizing and improving agent behavior
What we offer
What we offer
  • medical, dental & vision coverage
  • 401(k)
  • life, accident, and disability insurance
  • wellness programs
  • paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays
  • discretionary and formulaic incentive and retention awards
  • Fulltime
!
Read More
Arrow Right
New

Business Operations Program Manager

We are looking for a passionate and results-oriented Business Operations Program...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
netapp.com Logo
NetApp
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years experience as a Program Manager in managing the organizational business cadence of large technology teams
  • BS/BA or equivalent job-related experience
  • Demonstrated ability to work effectively with highly technical engineering teams, with a history of leading the team in delivering against product requirements
  • Solid ability to provide structure and vital process without compromising execution or the needs of the team, customer or product
  • Proven record to lead multi-functional teams through influence versus direct management
  • Demonstrates a strong bias to action
  • not afraid to tackle unknowns
  • Strong interpersonal, analytical, problem-solving and conflict resolution skills
  • Experience managing projects with remote/geographically dispersed team members
  • Must be able to keep information received fully confidential
Job Responsibility
Job Responsibility
  • Strategic Planning: Annual Fiscal Year (FY) Planning & Management
  • Own the annual AOFP process bottoms up and tops down target planning
  • Build monthly OPEX forecast for each team, review with Finance for submission each month
  • participate in monthly OPEX reviews with Finance
  • provide variance analysis as needed
  • Drive quarterly CAPEX planning, CAPEX requests and monitor execution for supported teams
  • act as focal point for changes and/or ad hoc requests throughout the quarter
  • Manage Cloud Spend forecast data by leader, project, and product to rationalize latest spend trends. Represent changes in product execution strategy back to cloud spend budget changes
  • Ensure monthly, quarterly and annual budget and headcount targets are met with little variance
  • Ensure per geo headcount targets are met (including site strategies for hiring), oversee requisition staffing process & reporting
What we offer
What we offer
  • Volunteer time off program is best in class, offering employees 40 hours of paid time off each year to volunteer with their favourite organizations
  • We provide comprehensive benefits, including health care, life and accident plans, emotional support resources for you and your family, legal services, and financial savings programs to help you plan for your future
  • We support professional and personal growth through educational assistance and provide access to various discounts and perks to enhance your overall quality of life
Read More
Arrow Right
New

Customer Contact Director – Patient Operations

We’re seeking an Customer Service Director to build and lead our growing support...
Location
Location
Philippines , Manila
Salary
Salary:
Not provided
eucalyptus.health Logo
Eucalyptus
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of experience in operations, ideally within healthcare, health-tech, or a patient-facing environment (10+ preferred)
  • Proven experience building or scaling a support function, not just managing one
  • Strong background in knowledge management, training, or learning and development
  • Deep familiarity with CRM/ticketing systems (Zendesk, Salesforce, Freshdesk, etc.)
  • Experience mentoring leaders and instilling structured management practices
  • Strong communication and reporting skills, with experience presenting to senior leadership
  • Analytical mindset
  • comfortable working with data and metrics (SQL proficiency a plus)
  • Experience leading hybrid teams working across non-standard hours and a willingness to be onsite a minimum of 2 days/week
Job Responsibility
Job Responsibility
  • Optimize daily operations for Patient and Medical Support teams across phone, chat, and email channels
  • Build and formalize the operational backbone — workflows, queue management, escalations, and reporting cadence
  • Track, analyze, and improve key performance metrics such as throughput, SLA adherence, and quality
  • Serve as the first point of contact for cross-functional project launches, ensuring operational readiness and clear communication
  • Establish the vision, structure, and roadmap for the support function as it scales
  • Design operating models, escalation frameworks, and performance metrics that promote clarity and accountability
  • Lead change management initiatives as systems, tools, and processes evolve
  • Partner with senior leadership to forecast resources and set strategic priorities
  • Build a learning and development framework for all support roles, from onboarding to leadership upskilling
  • Oversee knowledge systems (e.g. Zendesk, Confluence, Notion) to ensure accurate and accessible SOPs and training materials
What we offer
What we offer
  • Comprehensive health coverage (including up to two dependents)
  • Generous vacation, sick, and parental leave
  • 13th-month pay
  • Statutory benefits (SSS, PhilHealth, HDMF)
  • Wellness budget
  • Transport allowance
  • The best tools to help you do your best work
  • Access to learning budgets and mentorship from leaders across Australia, the UK, Germany, and Japan
  • Collaborate with top-tier teams locally and from South Africa
  • Gain global experience with opportunities to move across teams and explore new markets
Read More
Arrow Right
New

Senior Technical Delivery Manager

We're seeking a Senior Technical Delivery Manager to spearhead enterprise deploy...
Location
Location
Spain , Madrid
Salary
Salary:
Not provided
maisa.ai Logo
Maisa
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong, demonstrable experience in technical delivery, solutions architecture, or infrastructure engineering, with experience supporting enterprise software deployments in highly regulated industries (financial services, healthcare, manufacturing)
  • Deep understanding of cloud infrastructure (AWS preferred), containerization technologies, enterprise networking, as well as security & compliance frameworks relevant to enterprise AI (SOC 2, ISO 27001, GDPR)
  • Proven track record managing multi-stakeholder technical projects across geographies and organizational boundaries
  • Technical Competencies: Cloud & Infrastructure: AWS (VPC, IAM, ECS/EKS, CloudWatch), Docker, Kubernetes, CI/CD pipelines
  • Enterprise Integration: RESTful APIs, authentication protocols (OAuth, SAML), enterprise databases, ETL/data pipeline concepts
  • AI/ML Fundamentals: Understanding of LLM architectures, RAG systems, agent frameworks, and enterprise AI governance
  • Security & Compliance: Experience with enterprise security reviews, data residency requirements, encryption standards, and regulatory compliance
  • Key Attributes: Exceptional communication skills with ability to translate technical concepts for executive stakeholders and build trusted advisor relationships
  • Calm and composed under pressure, with proven ability to navigate high-stakes situations and de-escalate tension with diplomacy and professionalism
  • Structured problem-solver who thrives in ambiguous, fast-moving environments with composure under pressure
Job Responsibility
Job Responsibility
  • Lead end-to-end technical delivery of Maisa platform deployments across geos and regulatory environments, designing solution architectures for on-premise, private cloud, and hybrid environments that meet stringent enterprise security and compliance requirements while integrating with existing enterprise systems (e.g. legacy systems, ERP/CRM, identity providers, SIEM tools), as well as troubleshooting complex technical issues spanning containerization (Docker/Kubernetes), networking, API integrations, and enterprise authentication (SSO, MSK, RBAC)
  • Serve as the primary technical point of contact for strategic enterprise clients, translating complex AI process automation capabilities into business value through solution design workshops, architecture reviews, and executive stakeholder engagement
  • Partner with Support teams to ensure smooth operational hand off by developing technical documentation, SLA’s, escalation paths and playbooks, while serving as the escalation point for complex technical issues and critical incidents bridging between client team and Maisa internal
  • Bridge between Client Services, Product Engineering, Applied AI, Security, Support Operations, and customer teams to ensure alignment on requirements and timelines, develop deployment playbooks, identify product enhancement opportunities, and support pre-sales technical validation
  • Fulltime
Read More
Arrow Right
New

Nvh Test Engineer

We are seeking an experienced NVH Test Engineer who can hit the ground running f...
Location
Location
United States , Newark
Salary
Salary:
105000.00 - 125000.00 USD / Year
altenusa.com Logo
ALTEN Technology USA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Mechanical Engineering, Automotive Engineering, or related field
  • 5+ years of experience in NVH testing and analysis at the vehicle or powertrain level
  • Hands-on experience with accelerometers, microphones, and NVH instrumentation
  • Proficiency in LMS Test Lab or Artemis software
  • Strong skills in data processing and interpretation
  • Ability to work independently and deliver results under tight timelines
Job Responsibility
Job Responsibility
  • Plan, execute, and analyze NVH tests at the vehicle level, focusing on powertrain systems
  • Perform data acquisition and processing using accelerometers, microphones, and other instrumentation
  • Utilize LMS Test Lab or Artemis software for NVH measurements and analysis
  • Lead troubleshooting and root cause analysis for NVH-related issues
  • Prepare detailed test reports and communicate findings to cross-functional teams
  • Collaborate with design and development teams to recommend improvements based on NVH results
Read More
Arrow Right