CrawlJobs Logo

Pyspark Module Lead

https://www.soprasteria.com Logo

Sopra Steria

Location Icon

Location:
India , Bengaluru

Category Icon

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

Not provided

Job Description:

Sopra Steria is seeking a PySpark Module Lead based in Bengaluru, India. The position involves designing and building data solutions for Airbus, leveraging the Skywise platform and tools, and working along Product Owners and business functions. The role requires a strong understanding of Python, PySpark, SQL, and JavaScript, and proficiency with Skywise tools such as Contour, Code-Workbook, and Ontology.

Job Responsibility:

  • Designing and building solutions for Airbus on foundry
  • Data engineering and integration
  • Data analysis and visualization
  • Application development within Skywise Framework
  • Working closely with Product Owners and core Airbus business functions to understand processes, capture requirements, and solve business problems
  • Full delivery lifecycle for designing, implementing, testing, documenting, and supporting applications
  • CI/CD pipeline development and automated deployments
  • Troubleshooting application issues
  • Providing support on developed tools

Requirements:

  • Python
  • PySpark
  • SQL
  • JavaScript
  • Proficient with Skywise platform and tools e.g. Contour, Code-Workbook, Code, Slate and Ontology

Nice to have:

  • Knowledge of airline and MRO operations
  • AWS Cloud Services like EC2, RDS and Redshift
  • Agile/Scrum
  • SAFe
  • Palantir tools like Slate, Workshop
What we offer:
  • Inclusive and respectful work environment
  • Open to people with disabilities

Additional Information:

Job Posted:
May 05, 2025

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Pyspark Module Lead

Pyspark Module Lead

We are seeking a highly skilled and motivated Data Engineer to join our dynamic ...
Location
Location
India , Noida
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop
  • Proven expertise in designing and deploying data pipelines
  • Strong problem-solving skills and ability to work effectively in a collaborative team environment
  • Excellent communication skills and ability to translate technical concepts to non-technical stakeholders
Job Responsibility
Job Responsibility
  • Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines
  • Utilize PySpark for data processing, transformation, and preparation for model training
  • Leverage AWS EMR and S3 for scalable and efficient data storage and processing
  • Implement and manage ETL workflows using Streamsets for data ingestion and transformation
  • Design and construct pipelines to deliver high-quality training and inference datasets
  • Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities
  • Optimize and fine-tune pipelines for performance, scalability, and reliability
  • Ensure IAM policies and permissions are appropriately configured for secure data access and management
  • Implement Spark architecture and optimize Spark jobs for scalable data processing
What we offer
What we offer
  • All positions are open to people with disabilities
  • Commitment to fighting against all forms of discrimination
  • Inclusive and respectful work environment
Read More
Arrow Right

PySpark Module Lead

PySpark Module Lead position at Sopra Steria, a major Tech player in Europe. The...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Good experience in applying best coding practices
  • Good understanding and ability to develop simple to medium applications
  • Strong learnability and analytical skills
  • Strong team player with excellent communication skills
  • Mandatory skills: AWS, PySpark, PL SQL, Java
  • AWS Skills are on S3, Lambda, Cloudwatch, Dynamo DB, SQS, SNS, CloudFormation, Stack, Serverless
  • Total Experience Expected: 04-06 years
  • B.Tech, MCA or equivalent degree
What we offer
What we offer
  • Committed to fighting against all forms of discrimination
  • Fosters a work environment that is inclusive and respectful of all differences
  • All positions are open to people with disabilities
Read More
Arrow Right

Senior Software Engineer

The Senior Software Engineer will lead a team of data engineers to design, build...
Location
Location
United States , San Diego
Salary
Salary:
Not provided
gomillenniumsoft.com Logo
MillenniumSoft Inc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in developing data solutions
  • Proven experience in designing and implementing large-scale data pipelines
  • Hands-on expertise with AWS cloud technologies and distributed systems
  • Strong programming skills in PySpark, Python, and SQL
  • Data pipeline development using AWS tools (Step Functions, Airflow, etc.)
  • Distributed systems (S3, ADLS), and modern file/table formats (Parquet, Hudi)
  • ETL tools, orchestration technologies, and database design
  • Knowledge of regulatory standards like SOX, PCI, and HIPAA
  • Bachelor’s or Master’s in Computer Science, Information Systems, or related fields
  • AWS certification preferred
Job Responsibility
Job Responsibility
  • Design and implement scalable data pipelines for structured and unstructured data
  • Lead module architecture design for optimal performance, reliability, and maintainability
  • Mentor team members and ensure alignment with business goals
  • Collaborate with cross-functional teams to enable real-time and batch data processing
  • Ensure adherence to data security, privacy, and governance standards
  • Create and maintain technical documentation for architectures and decisions
  • Conduct code reviews and establish best practices
  • Develop monitoring systems for data pipeline reliability
  • Drive adoption of new technologies and workflows
  • Fulltime
Read More
Arrow Right
New

Senior Backend Engineer - Accounting Integrations

We're looking for a Senior Backend Engineer (with Tech Lead responsibilities) to...
Location
Location
Copenhagen; London; Madrid
Salary
Salary:
Not provided
pleo.io Logo
Pleo
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Expertise in building integrations or working on accounting systems (is an advantage, but not necessary)
  • Senior-level systems thinking, able to operate “outside the team” and influence stakeholders
  • Expertise in server-side languages, especially Kotlin, and experience with distributed systems, microservices, and cloud environments (e.g., AWS, Kubernetes)
  • Proficiency with relational databases like PostgreSQL, testing frameworks (e.g., JUnit, Testcontainers), and observability tools like Grafana
  • Strong problem-solving skills and a collaborative mindset to mentor and upskill your teammates
  • A proven ability to lead large projects, manage ambiguity, and maintain high standards for code quality and reliability
  • A passion for innovation and driving impactful solutions within a fast-paced scale-up environment.
Job Responsibility
Job Responsibility
  • Drive technical direction across integration platform + delivery work, with clear trade-offs and pragmatic sequencing
  • Define and raise engineering quality standards for integrations (testing expectations, reliability/monitoring, security, logging), including for work done by external developers/partners
  • Lead initiatives that enable scale, like reusable patterns/tooling (e.g., IDK) and frameworks that reduce one-off implementation work
  • Improve ways of working: planning, prioritisation, and protecting the team from constant mid-week churn
  • Collaborate with cross-functional teams to develop innovative solutions across Platform, Services, SMB, and other domains
  • Participate in code reviews, post-mortems, and provide mentorship to other engineers
  • Proactively address technical debt and guide the team through technical challenges and migrations.
What we offer
What we offer
  • Your own Pleo card (no more out-of-pocket spending!)
  • Lunch is on us for your work days - enjoy catered meals or receive a lunch allowance based on your local office
  • Comprehensive private healthcare - depending on your location, coverage options include Vitality, Alan or Médis
  • We offer 25 days of holiday + your public holidays
  • Option to purchase 5 additional days of holiday through a salary sacrifice
  • We use MyndUp to give our employees access to free mental health and well-being support
  • Paid parental leave.
Read More
Arrow Right
New

Bilingual customer service representative

Are you a dynamic and customer-focused individual looking for a challenging new ...
Location
Location
Canada , Calgary
Salary
Salary:
36000.00 - 39000.00 CAD / Year
https://www.randstad.com Logo
Randstad
Expiration Date
April 06, 2026
Flip Icon
Requirements
Requirements
  • Fluency in French and English
  • A high school diploma or equivalent
  • A minimum of 2 years of proven customer service experience, preferably in the telecommunications industry
  • Excellent written and verbal English communication skills
  • Strong problem-solving abilities and a proactive approach to challenges
  • A highly organized individual with the capacity to manage multiple tasks simultaneously
  • Solid computer skills
Job Responsibility
Job Responsibility
  • Provide superior customer service by identifying and assessing customer needs to achieve satisfaction
  • Build, manage, and maintain positive business relationships through open and interactive communication
  • Embrace and adapt to change in a fast-paced, performance-driven team environment
  • Offer technical support to customers, troubleshooting common issues and providing clear, effective solutions
  • Coordinate with appropriate internal teams to ensure the timely resolution of all customer issues
  • Accurately process customer orders based on specifications and terms (e.g., Enterprise or Government accounts)
  • Provide comprehensive post-sales support to maintain a high level of client satisfaction
  • Deal with conflict resolution with an emphasis on customer retention
  • As required, assist with on-site device deployment activities
  • Perform back-office administrative tasks, including scheduling, record keeping, and auditing program offerings and transactions for accuracy
What we offer
What we offer
  • A 1-year contract with the potential for a permanent position
  • Hybrid work schedule offering a healthy work-life balance (2-3 days work from home)
  • All necessary work equipment provided
  • Two weeks of paid vacation time
  • A positive and supportive company culture with a strong emphasis on core values
  • A comprehensive onboarding and mentorship program designed to set you up for success
  • Opportunity to work with a leader in the telecommunications industry and grow your sales & marketing career
  • Fulltime
Read More
Arrow Right
New

Student services representative

Are you passionate about customer service and enjoy working as part of a team? D...
Location
Location
Canada , Laval
Salary
Salary:
23.00 - 24.00 CAD / Hour
https://www.randstad.com Logo
Randstad
Expiration Date
April 13, 2026
Flip Icon
Requirements
Requirements
  • Minimum of 1 year of experience in customer service (ideally in a call center setting)
  • Strong communication skills in both French and English, spoken and written
  • Proficient with Microsoft Office and Windows environment
  • Excellent written and verbal communication skills
  • Ability to work independently with sound judgment and initiative
Job Responsibility
Job Responsibility
  • Respond to student inquiries in person, by phone, or by email, and refer them to the appropriate resources as needed
  • Manage academic platforms (class schedules, exam setup, activity tracking, access management, etc.)
  • Process training path requests and coordinate student registrations
  • Provide academic follow-up and support to ensure student success across various learning formats (on-site, online, or distance learning)
  • Serve as a liaison between students, academic advisors, the administration, and external partners
What we offer
What we offer
  • Competitive salary based on experience
  • Stable 35-hour workweek, Monday to Friday
  • Vacation policy in addition to office closure during the holiday period
  • Work-from-home every Friday
  • Workplace easily accessible by public transit
  • Free parking for those with a vehicle
  • Access to Randstad benefits: Discounts on group insurance
  • Unlimited online training platform
  • Personalized support throughout your assignment
  • A positive environment with guaranteed challenges and fun
  • Fulltime
Read More
Arrow Right
New

Teaching assistant

This is your opportunity to join a small, independent specialist school, designe...
Location
Location
United Kingdom , Copthorne
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
March 10, 2026
Flip Icon
Requirements
Requirements
  • Relevant SEN experience (personal or work)
  • Experience (voluntary or paid) working with children and young adults
  • GCSE Maths and English or equivalent
  • Eligibility to work in the UK
Job Responsibility
Job Responsibility
  • Supporting in the delivery of engaging and interactive lessons and educational activities
  • Supporting learners with SEN on a 1:1, small group and whole class basis
What we offer
What we offer
  • Free onsite parking
  • School networking events/ social events for teachers
  • Ready of supply of drinks, chocolate, biscuits and cakes in their welcoming and inviting staff room equally equipped with games, TV's and shared library
  • Brilliant resources and spacious classrooms/grounds
  • Career progression opportunities
  • Fully funded training and qualifications
Read More
Arrow Right
New

Social Worker

Are you an empathetic and dedicated Social Worker looking to make a meaningful i...
Location
Location
United Kingdom , Dalkeith
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
April 07, 2026
Flip Icon
Requirements
Requirements
  • A degree in Social Work or an equivalent qualification recognised by the Scottish Social Services Council (SSSC)
  • Current SSSC registration as a social worker
  • Significant post-qualification experience in a statutory Children and Families setting, with a proven track record of managing complex child protection caseloads
  • Expert-level knowledge of relevant Scottish legislation, including the Children's Hearings (Scotland) Act 2011 and the Social Work (Scotland) Act 1968
  • Exceptional ability to gather, analyse, and synthesise complex information under pressure to inform professional judgments
  • Demonstrated emotional resilience and ability to maintain professional effectiveness when dealing with highly distressing and challenging situations
Job Responsibility
Job Responsibility
  • Conducting holistic assessments of needs for older people within the community
  • Ensuring that care packages remain fit for purpose and promote independence
  • Visiting residents in residential and nursing care to monitor wellbeing and quality of care
  • Working closely with health professionals, families, and external providers to coordinate integrated care
  • Working within the framework of the Social Care (Self-directed Support) (Scotland) Act and the Adults with Incapacity (Scotland) Act
What we offer
What we offer
  • Flexible Hours: Control your work hours and choose assignments that fit your lifestyle
  • Competitive Pay Rates: We offer highly competitive remuneration for each placement
  • Professional Development: Opportunities to work across various departments, enhancing your skills and broadening your expertise
  • Free training: Keep up to date with your Continuous Professional Development
  • Pension Scheme: Contributory pension
  • Referral Bonus: Receive £300 in vouchers (subject to Ts and Cs)
  • Weekly payroll: Randstad runs a twice weekly payroll, enabling you to control your finances by receiving weekly pay
  • Dedicated Support: Our team is here to assist you at every step, from onboarding through the entirety of your placement. We can even help with sourcing accommodation and travel, if required
Read More
Arrow Right