Data Engineer (Python ETL) Job at VARTEQ

Big Data / Scala / Python Engineering Lead

The Applications Development Technology Lead Analyst is a senior level position ...

Location

India , Chennai

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

At least two years (Over all 10+ hands on Data Engineering experience) of experience building and leading highly complex, technical data engineering teams
Lead data engineering team, from sourcing to closing
Drive strategic vision for the team and product
Experience managing an data focused product, ML platform
Hands on experience relevant experience in design, develop, and optimize scalable distributed data processing pipelines using Apache Spark and Scala
Experience managing, hiring and coaching software engineering teams
Experience with large-scale distributed web services and the processes around testing, monitoring, and SLAs to ensure high product quality
7 to 10+ years of hands-on experience in big data development, focusing on Apache Spark, Scala, and distributed systems
Proficiency in Functional Programming: High proficiency in Scala-based functional programming for developing robust and efficient data processing pipelines
Proficiency in Big Data Technologies: Strong experience with Apache Spark, Hadoop ecosystem tools such as Hive, HDFS, and YARN

Job Responsibility

Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary

Fulltime

Data Engineer

We are seeking our first Data Engineer, someone who can refine our data infrastr...

Location

United States , New York City; San Francisco

Salary:

190000.00 - 250000.00 USD / Year

Hebbia

Expiration Date

Until further notice

Requirements

Bachelor's or Master's degree in Computer Science, Data Science, Statistics, or a related field
5+ years software development experience at a venture-backed startup or top technology firm, with a focus on data engineering
Significant hands-on experience in data engineering (ETL development, data warehousing, data lake management, etc.)
Adept at identifying and owning data projects end to end, with the ability to work independently and exercise sound judgment
Proficient in Python and SQL
comfortable working with cloud-based data stack tools
Familiar with big data processing frameworks (e.g., Spark, Hadoop) and data integration technologies (e.g., Airflow, DBT, or similar)
Experience implementing data governance, security, and compliance measures
Strong collaboration and communication skills, with the ability to translate business requirements into technical solutions
You are comfortable working in-person 5 days a week

Job Responsibility

Architect, build, and maintain ETL pipelines and workflows that ensure high data quality and reliability
Design and manage a central data lake to consolidate data from various sources, enabling advanced analytics and reporting
Collaborate with cross-functional stakeholders (product, engineering, and business) to identify data gaps and develop effective solutions
Implement best practices in data security and governance to ensure compliance and trustworthiness
Evaluate and integrate new technologies, tools, and approaches to optimize data processes and architectures
Continuously monitor, troubleshoot, and improve data pipelines and infrastructure for performance, scalability, and cost-efficiency

What we offer

PTO: Unlimited
Insurance: Medical + Dental + Vision + 401K + Wellness Benefits
Eats: Catered lunch daily + doordash dinner credit if you ever need to stay late
Parental leave policy: 3 months non-birthing parent, 4 months for birthing parent
Fertility benefits: $15k lifetime benefit
New hire equity grant: competitive equity package with unmatched upside potential

Fulltime

Data Engineer – Snowflake & ETL

We are seeking a Data Engineer in Hyderabad (WFO) with expertise in data enginee...

Location

India , Hyderabad

Salary:

Not provided

Right Angle Solutions

Expiration Date

Until further notice

Requirements

Minimum 5+ years of experience in data engineering, ETL, and Snowflake development
Proven expertise in Snowflake including SQL scripting, performance tuning, and data warehousing concepts
Hands-on experience with Matillion ETL for building and maintaining ETL jobs
Strong understanding of cloud platforms (AWS, Azure, or GCP) and cloud-based data architectures
Proficiency in SQL, Python, or other scripting languages for automation and data transformation
Experience with API integrations and data ingestion frameworks
Knowledge of data governance, security policies, and access control within Snowflake environments
Excellent communication skills with the ability to engage both business and technical stakeholders
Self-motivated professional capable of working independently and delivering projects on time
Qualification: BE/BS/MTech/MS or equivalent work experience

Fulltime

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...

Location

United Arab Emirates , Dubai

Salary:

Not provided

NorthBay

Expiration Date

Until further notice

Requirements

8+ years of experience in data engineering and data platform development
Strong hands-on experience with: AWS Glue
Amazon EMR (Spark)
AWS Lambda
Apache Airflow (MWAA)
Amazon EC2
Amazon CloudWatch
Amazon Redshift
Amazon DynamoDB
AWS DataZone

Job Responsibility

Design, develop, and optimize scalable data pipelines using AWS native services
Lead the implementation of batch and near-real-time data processing solutions
Architect and manage data ingestion, transformation, and storage layers
Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
Orchestrate complex data workflows using Apache Airflow (MWAA)
Develop and manage serverless data processing using AWS Lambda
Design and optimize data warehouses using Amazon Redshift
Implement and manage NoSQL data models using Amazon DynamoDB
Utilize AWS DataZone for data governance, cataloging, and access management
Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch

Fulltime

Software Engineer (Data Engineering)

We are seeking a Software Engineer (Data Engineering) who can seamlessly integra...

Location

India , Hyderabad

Salary:

Not provided

NStarX

Expiration Date

Until further notice

Requirements

4+ years in Data Engineering and AI/ML roles
Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field
Python, SQL, Bash, PySpark, Spark SQL, boto3, pandas
Apache Spark on EMR (driver/executor model, sizing, dynamic allocation)
Amazon S3 (Parquet) with lifecycle management to Glacier
AWS Glue Catalog and Crawlers
AWS Step Functions, AWS Lambda, Amazon EventBridge
CloudWatch Logs and Metrics, Kinesis Data Firehose (or Kafka/MSK)
Amazon Redshift and Redshift Spectrum
IAM (least privilege), Secrets Manager, SSM

Job Responsibility

Design, build, and maintain scalable ETL and ELT pipelines for large-scale data processing
Develop and optimize data architectures supporting analytics and ML workflows
Ensure data integrity, security, and compliance with organizational and industry standards
Collaborate with DevOps teams to deploy and monitor data pipelines in production environments
Build predictive and prescriptive models leveraging AI and ML techniques
Develop and deploy machine learning and deep learning models using TensorFlow, PyTorch, or Scikit-learn
Perform feature engineering, statistical analysis, and data preprocessing
Continuously monitor and optimize models for accuracy and scalability
Integrate AI-driven insights into business processes and strategies
Serve as the technical liaison between NStarX and client teams

What we offer

Competitive salary and performance-based incentives
Opportunity to work on cutting-edge AI and ML projects
Exposure to global clients and international project delivery
Continuous learning and professional development opportunities
Competitive base + commission
Fast growth into leadership roles

Fulltime

Senior Data Engineer

At Ingka Investments (Part of Ingka Group – the largest owner and operator of IK...

Location

Netherlands , Leiden

Salary:

Not provided

IKEA

Expiration Date

Until further notice

Requirements

Formal qualifications (BSc, MSc, PhD) in computer science, software engineering, informatics or equivalent
Minimum 3 years of professional experience as a (Junior) Data Engineer
Strong knowledge in designing efficient, robust and automated data pipelines, ETL workflows, data warehousing and Big Data processing
Hands-on experience with Azure data services like Azure Databricks, Unity Catalog, Azure Data Lake Storage, Azure Data Factory, DBT and Power BI
Hands-on experience with data modeling for BI & ML for performance and efficiency
The ability to apply such methods to solve business problems using one or more Azure Data and Analytics services in combination with building data pipelines, data streams, and system integration
Experience in driving new data engineering developments (e.g. apply new cutting edge data engineering methods to improve performance of data integration, use new tools to improve data quality and etc.)
Knowledge of DevOps practices and tools including CI/CD pipelines and version control systems (e.g., Git)
Proficiency in programming languages such as Python, SQL, PySpark and others relevant to data engineering
Hands-on experience to deploy code artifacts into production

Job Responsibility

Contribute to the development of D&A platform and analytical tools, ensuring easy and standardized access and sharing of data
Subject matter expert for Azure Databrick, Azure Data factory and ADLS
Help design, build and maintain data pipelines (accelerators)
Document and make the relevant know-how & standard available
Ensure pipelines and consistency with relevant digital frameworks, principles, guidelines and standards
Support in understand needs of Data Product Teams and other stakeholders
Explore ways create better visibility on data quality and Data assets on the D&A platform
Identify opportunities for data assets and D&A platform toolchain
Work closely together with partners, peers and other relevant roles like data engineers, analysts or architects across IKEA as well as in your team

What we offer

Opportunity to develop on a cutting-edge Data & Analytics platform
Opportunities to have a global impact on your work
A team of great colleagues to learn together with
An environment focused on driving business and personal growth together, with focus on continuous learning

Fulltime

Senior Data Engineer

Senior Data Engineer role driving Circle K's cloud-first strategy to unlock the ...

Location

India , Gurugram

Salary:

Not provided

Circle K

Expiration Date

Until further notice

Requirements

Bachelor's Degree in Computer Engineering, Computer Science or related discipline
Master's Degree preferred
5+ years of ETL design, development, and performance tuning using ETL tools such as SSIS/ADF in a multi-dimensional Data Warehousing environment
5+ years of experience with setting up and operating data pipelines using Python or SQL
5+ years of advanced SQL Programming: PL/SQL, T-SQL
5+ years of experience working with Snowflake, including Snowflake SQL, data modeling, and performance optimization
Strong hands-on experience with cloud data platforms such as Azure Synapse and Snowflake for building data pipelines and analytics workloads
5+ years of strong and extensive hands-on experience in Azure, preferably data heavy / analytics applications leveraging relational and NoSQL databases, Data Warehouse and Big Data
5+ years of experience with Azure Data Factory, Azure Synapse Analytics, Azure Analysis Services, Azure Databricks, Blob Storage, Databricks/Spark, Azure SQL DW/Synapse, and Azure functions
5+ years of experience in defining and enabling data quality standards for auditing, and monitoring

Job Responsibility

Collaborate with business stakeholders and other technical team members to acquire and migrate data sources
Determine solutions that are best suited to develop a pipeline for a particular data source
Develop data flow pipelines to extract, transform, and load data from various data sources
Efficient in ETL/ELT development using Azure cloud services and Snowflake
Work with modern data platforms including Snowflake to develop, test, and operationalize data pipelines
Provide clear documentation for delivered solutions and processes
Identify and implement internal process improvements for data management
Stay current with and adopt new tools and applications
Build cross-platform data strategy to aggregate multiple sources
Proactive in stakeholder communication, mentor/guide junior resources

Fulltime

Senior Data Engineer

We’re growing our team at ELEKS in partnership with a company, the UK’s largest ...

Location

Salary:

Not provided

ELEKS

Expiration Date

Until further notice

Requirements

Experience in Data Engineering, SQL, ETL(data validation + data mapping + exception handling) 4+ years
Hands-on experience with Databricks 2+ years
Experience with Python
Experience with AWS (e.g. S3, Redshift, Athena, Glue, Lambda, etc.)
At least an Upper-Intermediate level of English

Job Responsibility

Building Databases and Pipelines: Developing databases, data lakes, and data ingestion pipelines to deliver datasets for various projects
End-to-End Solutions: Designing, developing, and deploying comprehensive solutions for data and data science models, ensuring usability for both data scientists and non-technical users. This includes following best engineering and data science practices
Scalable Solutions: Developing and maintaining scalable data and machine learning solutions throughout the data lifecycle, supporting the code and infrastructure for databases, data pipelines, metadata, and code management
Stakeholder Engagement: Collaborating with stakeholders across various departments, including data platforms, architecture, development, and operational teams, as well as addressing data security, privacy, and third-party coordination

What we offer

Close cooperation with a customer
Challenging tasks
Competence development
Ability to influence project technologies
Team of professionals
Dynamic environment with low level of bureaucracy

Data Engineer (Python ETL)

VARTEQ

Location:

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

Job Description:

Job Responsibility:

Requirements:

Nice to have:

Additional Information:

Job Posted:
February 16, 2026

Looking for more opportunities? Search for other job offers that match your skills and interests.

Similar Jobs for Data Engineer (Python ETL)