CrawlJobs Logo

Data Engineer - II

aticaglobal.com Logo

Atica Global

Location Icon

Location:
India, Pune

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

The Data Engineer will design, develop, and maintain scalable data pipelines and ETL/ELT processes using tools like Airflow, Airbyte and PySpark. Collaborate with software engineers and analysts to ensure data availability and integrity for various applications.

Job Responsibility:

  • Design, develop, and maintain scalable data pipelines and ETL/ELT processes using tools like Airflow, Airbyte and PySpark
  • Collaborate with software engineers and analysts to ensure data availability and integrity for various applications
  • Design and implement robust data pipelines to extract, transform, and load (ETL) data from various sources
  • Utilize Airflow for orchestrating complex workflows and managing data pipelines
  • Implement batch processing techniques using Airflow/PySpark to handle large volumes of data efficiently
  • Develop ELT processes to optimize data extraction and transformation within the target data warehouse
  • Leverage AWS services (e.g., S3, RDS, Lambda) for data storage, processing, and orchestration
  • Ensure data security, reliability, and performance when utilizing AWS resources
  • Work closely with developers, analysts, and other stakeholders to understand data requirements and provide the necessary data infrastructure
  • Assist in troubleshooting and optimizing existing data workflows and queries
  • Create and maintain documentation for data pipelines, workflows, and processes
  • Adhere to best practices in data engineering, including version control and monitoring

Requirements:

  • Bachelor’s degree in computer science, Engineering, Mathematics, a related field, or equivalent practical experience
  • 3-5 years of experience in data engineering or a similar mid-level role
  • Proficiency in Python and SQL
  • experience with Java is a plus
  • Hands-on experience with AWS, Airbyte, DBT, PostgreSQL, MongoDB, Airflow, and Spark
  • Familiarity with data storage solutions such as PostgreSQL, MongoDB
  • Experience with BigQuery (setup, management and scaling)
  • Strong understanding of data modeling, ETL/ELT processes, and database systems
  • Experience with data extraction, batch processing and data warehousing
  • Excellent problem-solving skills and a keen attention to detail
  • Ability to work effectively in a collaborative team environment with software engineers and analysts

Nice to have:

experience with Java is a plus

What we offer:
  • Competitive salary and benefits package
  • Comprehensive Health Care benefits (best in the country, includes IPD+OPD, covers Employee, Spouse and two children)
  • Growth and advancement opportunities within a rapidly expanding company

Additional Information:

Job Posted:
December 06, 2025

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Data Engineer - II

New

Senior Data Engineer II

We are looking for a skilled Data Engineer to join our growing team. You will pl...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
alterdomus.com Logo
Alter Domus
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
  • 3+ years of experience as a Data Engineer or in a similar role working with cloud-based data platforms
  • Technical Skills: Cloud & Orchestration: Airflow (self-managed or managed services like Amazon MWAA) for workflow orchestration, DAG development, and scheduling
  • Familiarity with best practices for Airflow DAG structure, dependency management, and error handling
  • AWS Expertise: Hands-on experience with AWS Lake Formation, S3, Athena, and related services (e.g., Lambda, Glue, IAM)
  • Snowflake: Proficient in setting up data warehouses, configuring security, and optimizing queries on Snowflake
  • Data Ingestion & Transformation: Experience with Airbyte or similar tools for data ingestion
  • dbt or other SQL-based transformation frameworks for modular data processing
  • Programming: Proficiency in Python and/or Java/Scala for building data pipelines and custom integrations
  • Query Languages: Advanced knowledge of SQL for data manipulation and analysis
Job Responsibility
Job Responsibility
  • Data Pipeline Orchestration: Design, build, and maintain end-to-end data pipelines using Airflow (including managed services like Amazon MWAA) to orchestrate, schedule, and monitor batch/streaming workflows
  • Implement DAGs (Directed Acyclic Graphs) with retry logic, error handling, and alerting to ensure data quality and pipeline reliability
  • Data Ingestion & Transformation: Integrate data from various sources using Airbyte for ingestion and dbt for transformations in a scalable and modular fashion
  • Collaborate with Data Analysts and Data Scientists to implement transformations and business logic, ensuring data is analytics-ready
  • Data Modeling & Warehousing: Design and implement efficient data models for both structured and semi-structured data in AWS S3 (data lake) and Snowflake (data warehouse)
  • Ensure data schemas and transformations support advanced analytics, BI reporting, and machine learning use cases
  • Data Governance & Security: Utilize AWS Lake Formation APIs and best practices to maintain data security, access controls, and compliance
  • Work closely with IT security to establish robust encryption standards, audit trails, and identity/role-based access
  • Performance Optimization: Optimize AWS Athena queries and configurations (e.g., data partitioning) for performance and cost efficiency
  • Monitor and tune Airflow DAGs, Snowflake queries, and data transformations to improve throughput and reliability
What we offer
What we offer
  • Support for professional accreditations
  • Flexible arrangements, generous holidays, plus an additional day off for your birthday
  • Continuous mentoring along your career progression
  • Active sports, events and social committees across our offices
  • 24/7 support available from our Employee Assistance Program
  • The opportunity to invest in our growth and success through our Employee Share Plan
  • Plus additional local benefits depending on your location
Read More
Arrow Right
New

Data Engineer II

The purpose of the Data Engineer is to migrate and convert data from foreign dat...
Location
Location
Poland , Warsaw
Salary
Salary:
Not provided
sspinnovations.com Logo
SSP Innovations
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s Degree in Information Technology or related field
  • 1+ year of experience with managed telecommunications databases
  • 3+ years of experience in data conversion, data mapping, and data analysis
  • 3+ years of experience using ESRI, FME, or a combination of both
  • Intermediate database experience including SQL relationship statements
  • Intermediate Programming/Report Analysis experience
  • Communication skills, both oral and written
  • Strong organizational skills and attention to detail
  • Deadline driven, ability to work independently as well as a team player
Job Responsibility
Job Responsibility
  • responsible for supporting data migrations and data conversions from disparate systems into the 3-GIS Data Model
  • oversee data mapping exercises and work closely with customers to understand their data models and to provide mapping between systems
  • employ a variety of tools including but not limited to Python Scripting, Database Scripting languages such as SQL, advanced knowledge of ESRI applications, FME workbenches and internally developed tools, as the assignment dictates
  • exhibit strong communication skills to ensure customer understanding of data migrations and conversions and must work with various team members such as Solutions Engineers, Data Technicians, Sales Executives and Project Managers to ensure project success
  • must be detail oriented and maintain proper time management as projects generally have specific timelines and milestones required for project success
  • ability to provide references for users by writing and maintaining documentation
  • design, develop, and test data transformation, extraction, and migration activities
  • prepare technical reports by collecting, analyzing, and summarizing information
  • perform tasks efficiently while validating methodology
  • interact directly (face-to-face and remotely) with clients and project teams
  • Fulltime
Read More
Arrow Right

Senior Data Engineer II

As a Senior Data Engineer II at dbt Labs, you'll take the lead in designing, bui...
Location
Location
United States
Salary
Salary:
172000.00 - 207900.00 USD / Year
getdbt.com Logo
dbt Labs
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Expert in SQL and Python
  • 8+ years of experience as a data engineer
  • 10+ years of total experience in software engineering
  • Strong knowledge of data infrastructure and architecture design
  • Hands-on experience with modern orchestration tools like Airflow, Dagster, or Prefect
  • Hands-on experience with handling and ingesting large amounts of Parquet files
  • A bias for action—able to stay focused and prioritize effectively
Job Responsibility
Job Responsibility
  • Design, build, and manage scalable, reliable data pipelines that ingest product and event data into our data stores
  • Develop and maintain canonical datasets to track key product and business metrics—user growth, engagement, revenue, and more
  • Architect robust, reliable systems for large volume batch data processing
  • Drive decisions on data architecture, tooling, and engineering best practices
  • Enhance observability and monitoring of existing workflows and processes
  • Partner cross-functionally with teams across Infrastructure, Product, Marketing, Finance, and GTM to understand data needs and deliver impactful solutions
  • Provide product feedback by “dogfooding” new data infrastructure and AI technology
What we offer
What we offer
  • Unlimited vacation time with a culture that actively encourages time off
  • 401k plan with 3% guaranteed company contribution
  • Comprehensive healthcare coverage
  • Generous paid parental leave
  • Flexible stipends for: Health & Wellness, Home Office Setup, Cell Phone & Internet, Learning & Development, Office Space
  • Fulltime
Read More
Arrow Right

Data Engineer II

You will be part of a world-class Data Engineering team, where you will influenc...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s / Masters degree (or equivalent work experience) in a STEM field
  • 2+ Years of Experience in Data Engineering or related fields
  • Expertise in Python or other modern programming language development experience
  • Knowledge of advanced level capabilities in SQL and relational databases experience
  • Solid working knowledge of ETL (ELT), Data modeling, DataQuality, and Data Visualization/Dashboarding
  • Experience working with product analytics
Job Responsibility
Job Responsibility
  • influence product teams
  • inform Data Science and Analytics Platform teams
  • partner closely with data consumers and producers to ensure quality and usefulness of data assets
  • Defining metrics
  • Instrumenting logging
  • Acquiring/ingesting data
  • Architecting & modeling data
  • Transforming data
  • Ensuring data quality, governance, and enablement
  • Alerting, visualization, and reporting
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Data Engineer II

You will be part of a world-class Data Engineering team, where you will influenc...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s / Masters degree (or equivalent work experience) in a STEM field
  • 2+ Years of Experience in Data Engineering or related fields
  • Expertise in Python or other modern programming language development experience
  • Knowledge of advanced level capabilities in SQL and relational databases experience
  • Solid working knowledge of ETL (ELT), Data modeling, DataQuality, and Data Visualization/Dashboarding
  • Experience working with product analytics
Job Responsibility
Job Responsibility
  • influence product teams
  • inform Data Science and Analytics Platform teams
  • partner closely with data consumers and producers to ensure quality and usefulness of data assets
  • Defining metrics
  • Instrumenting logging
  • Acquiring/ingesting data
  • Architecting & modeling data
  • Transforming data
  • Ensuring data quality, governance, and enablement
  • Alerting, visualization, and reporting
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Data Engineer II

As a data engineer in the People DE team, you will have the opportunity to apply...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 2 years professional experience as a software engineer or data engineer
  • A BS in Computer Science or equivalent experience
  • Strong programming skills (some combination of Python, Java, and Scala preferred)
  • Experience with data modeling
  • Experience in modern software development practices (Agile, TDD, CICD)
  • Knowledge of data warehousing concepts
  • Experience writing SQL, structuring data, and data storage practices
  • Experience building data pipelines, platforms, micro services, and REST APIs
  • Experience with Spark, Hive, Airflow and other streaming technologies to process incredible volumes of streaming data
  • A willingness to accept failure, learn and try again
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Data Engineer II

PagerDuty is looking for a Data Engineer II to join their global Enterprise Data...
Location
Location
Portugal , Lisbon
Salary
Salary:
Not provided
https://www.pagerduty.com Logo
PagerDuty
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of experience working in data integration, pipelines, and data modeling
  • experience designing and deploying code in Data platforms in a cloud-based and Agile environment
  • knowledge and experience of relational databases and being capable of writing complex SQL
  • experience working with Cloud-based Data Warehousing Platform such as Snowflake or AWS, or Databricks
  • experience with Python and SQL
  • experience working with SQL, AWS S3, and ETL/Automation Workflow tools like Workato
Job Responsibility
Job Responsibility
  • translate business requirements into data models that are easy to understand
  • design, implement, and build pipelines that deliver data with measurable quality under the SLA
  • partner with business domain experts, data analysts, and engineering teams to build foundational data sets
  • be a champion of the overall strategy for data governance, security, privacy, quality, and retention
  • own and document foundational company metrics with a clear definition and data lineage
  • contribute to the data ecosystem and build a strong data foundation for the company
  • design, implement, and scale data pipelines that transform billions of records into actionable data models
  • provide hands-on technical support to build trusted and reliable domain-specific datasets and metrics
What we offer
What we offer
  • competitive salary
  • comprehensive benefits package
  • flexible work arrangements
  • company equity
  • ESPP (Employee Stock Purchase Program)
  • retirement or pension plan
  • generous paid vacation time
  • paid holidays and sick leave
  • Dutonian Wellness Days & HibernationDuty
  • paid parental leave
  • Fulltime
Read More
Arrow Right

Java Engineer II - Data Ingestion team

The Data Ingestion team is responsible for delivering and enhancing owned social...
Location
Location
Hungary
Salary
Salary:
Not provided
brandwatch.com Logo
Brandwatch
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong proficiency in Java and Spring Framework
  • Solid understanding of PostgreSQL and relational database principles
  • Hands-on experience with Kafka or similar event-driven messaging technologies
  • Familiarity with modern DevOps tools: Git, GitHub Actions, Jenkins, Docker, and Kubernetes
Job Responsibility
Job Responsibility
  • Enhance and maintain the backend stack of data ingestion
  • Collaborate with members of other product groups, fostering a culture of partnership and innovation. You’ll work as an engineer delivering back-end solutions of data ingestion from social platforms
  • Deliver well-tested and scalable solutions that meet technical and product needs
  • Share and discuss best practices proactively with your peers
  • Support product teams, such as handling changes in Social APIs
What we offer
What we offer
  • Friendly and welcoming environment focused on people, learning & development
  • 25 vacation days and extra vacation days after age and after children
  • Cafeteria benefit via SZEP card
  • Medicover private health insurance for employees and their family members
  • 37.5 hour work week - or less!
  • 10% of your time to work on anything you like, reading groups, tech talks
  • Flexible working and working from home
  • An extensive people development program, including access to Udemy
  • Fulltime
Read More
Arrow Right
Welcome to CrawlJobs.com
Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.