CrawlJobs Logo

Cloud Big-data Engineer

phasorsoft.com Logo

PhasorSoft Group

Location Icon

Location:
United States, Starkville ▼

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

45.00 USD / Hour ▼
Save Job
Save Icon
Apply Position

Job Description:

An expert with 4-5 years of experience in Hadoop ecosystem and cloud- (AWS ecosystem/Azure), relational data stores, Data Integration techniques, XML, Python, Spark, and ETL techniques.

Requirements:

  • 4-5 years of experience in Hadoop ecosystem and cloud (AWS ecosystem/Azure)
  • Experience working with in-memory computing using R, Python, Spark, PySpark, Kafka, and Scala
  • Experience in parsing and shredding XML and JSON, shell scripting, and SQL
  • Experience working with Hadoop ecosystem - HDFS, Hive
  • Experience working with AWS ecosystem - S3, EMR, EC2, Lambda Cloud Formation, Cloud Watch, SNS/SQS
  • Experience with Azure – Azure Data Factory (ADF)
  • Experience working with SQL and No SQL databases
  • Experience designing and developing data sourcing routines utilizing typical data quality functions involving standardization, transformation, rationalization, linking, and matching
  • Work Authorization: H1, GC, US Citizen

Additional Information:

Job Posted:
December 11, 2025

Work Type:
On-site work
Icon
View All Jobs In This Company
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Cloud Big-data Engineer

Data Engineer

Are you a Data Engineer based in Austin, Texas, who is inspired by working with ...
Location
Location
United States , Austin
Salary
Salary:
100000.00 - 135000.00 USD / Year
beezwax.net Logo
Beezwax Datatools
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4-6 years of hands-on data modeling and data engineering experience
  • Strong expertise in dimensional modeling and data warehousing
  • Database design and development experience with relational or MPP databases such as Postgres/ Oracle/ Teradata/ Vertica
  • Experience in design and development of custom ETL pipelines using SQL and scripting languages (Python/ Shell/ Golang)
  • Proficiency in advanced SQL and performance tuning
  • Hands on experience with Big-Data platforms like Spark, Dremio, Hadoop, MapReduce, Hive etc
  • Experience with Java, Scala and Python
  • Experience with cloud computing platforms like AWS, Google Cloud
  • Experience working with APIs
  • Ability to learn and adapt to new tools and technologies
What we offer
What we offer
  • Competitive compensation
  • Retirement plan with employer matching
  • Excellent healthcare package with vision and dental
  • Support for productivity and continued learning in the forms of hardware, software, learning materials, training, and conferences
  • Fulltime
Read More
Arrow Right
New

Senior Data Engineer

Senior Data Engineer – Dublin (Hybrid) Contract Role | 3 Days Onsite. We are see...
Location
Location
Ireland , Dublin
Salary
Salary:
Not provided
solasit.ie Logo
Solas IT Recruitment
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of experience as a Data Engineer working with distributed data systems
  • 4+ years of deep Snowflake experience, including performance tuning, SQL optimization, and data modelling
  • Strong hands-on experience with the Hadoop ecosystem: HDFS, Hive, Impala, Spark (PySpark preferred)
  • Oozie, Airflow, or similar orchestration tools
  • Proven expertise with PySpark, Spark SQL, and large-scale data processing patterns
  • Experience with Databricks and Delta Lake (or equivalent big-data platforms)
  • Strong programming background in Python, Scala, or Java
  • Experience with cloud services (AWS preferred): S3, Glue, EMR, Redshift, Lambda, Athena, etc.
Job Responsibility
Job Responsibility
  • Build, enhance, and maintain large-scale ETL/ELT pipelines using Hadoop ecosystem tools including HDFS, Hive, Impala, and Oozie/Airflow
  • Develop distributed data processing solutions with PySpark, Spark SQL, Scala, or Python to support complex data transformations
  • Implement scalable and secure data ingestion frameworks to support both batch and streaming workloads
  • Work hands-on with Snowflake to design performant data models, optimize queries, and establish solid data governance practices
  • Collaborate on the migration and modernization of current big-data workloads to cloud-native platforms and Databricks
  • Tune Hadoop, Spark, and Snowflake systems for performance, storage efficiency, and reliability
  • Apply best practices in data modelling, partitioning strategies, and job orchestration for large datasets
  • Integrate metadata management, lineage tracking, and governance standards across the platform
  • Build automated validation frameworks to ensure accuracy, completeness, and reliability of data pipelines
  • Develop unit, integration, and end-to-end testing for ETL workflows using Python, Spark, and dbt testing where applicable
Read More
Arrow Right

Applications Development Senior Group Manager

This role will be part of the Risk Data team and is a senior management level po...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong academic record, ideally with a Bachelor’s or Master’s degree in Computer Science or engineering or related technical discipline
  • Proven experience in enterprise application development with full stack technologies
  • Strong Architect and hands on technical experience in implementing large volume real time complex solutions in Big Data Platforms & Public Cloud platforms
  • Experience in Data architecture, strong Software development fundamentals, data structures, design patterns, object-oriented principles
  • Experience in design and delivery of multi-tiered applications and high performance server side components
  • Skills on system performance tuning, high performance, low latency, multithreading and experience with Java server side programming
  • Preferred experience in Handling high volumes of data and working with In-memory databases and Caching solutions
  • Experience of building and leading teams, ideally with a global resource profile and demonstrated ability to deliver large projects efficiently and on time
  • Significant experience in large Financial Services Technology services companies is expected for this position
  • Hands-on development, architecture and leadership experience in real-time data engineering platforms implementation
Job Responsibility
Job Responsibility
  • Lead the efforts in Institutional Data Platform (ICG) that span multiple businesses, products and functions
  • Delivery of Price Risk related Data initiatives and Capital reporting (GSIB) related deliverables
  • Establish strong relationships with the global business stakeholders and ensure transparency of project deliveries
  • Actively identify and manage risks and issues, working with disparate teams to create mitigation plans and follow-through to resolution
  • Adhere to all key Project Management (PMQC) & Engineering Excellence standards
  • Ensure timely communications to Senior Technology Management and Business Partners in Front Office, Middle Office & other Operations functions
  • Drive the design and development of system architecture, work with end-users of the systems, and enhance the quality of deliverables
  • Ensure staff follows Citi documented policy and procedures as well as maintain desktop procedures and supporting documentation for filings on a current basis and in comprehensive manner
  • Ensure change is managed with appropriate controls, documentation, and approvals including implementation of new and revised regulatory reporting requirements
  • Manage and maintain all disaster recovery plans, oversee appropriate testing, and provide permit-to-operate for new applications
What we offer
What we offer
  • 27 days annual leave (plus bank holidays)
  • A discretional annual performance related bonus
  • Private Medical Care & Life Insurance
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources
  • Fulltime
Read More
Arrow Right

Principal Software Engineer

Principal Software Engineer role at Hewlett Packard Enterprise to design, develo...
Location
Location
United States , San Jose
Salary
Salary:
148000.00 - 340500.00 USD / Year
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor or Masters degree in Computer science, Computer Engineering or a related field
  • 10+ years of experience in software engineering with a focus on Python, Go or Java
  • Strong understanding of RESTful API design and development
  • 2+ years of Experience working with large scale distributed systems based on either cloud technologies or Kubernetes
  • 2+ years of experience on event-driven technologies like Kafka and Apache Storm/Flink
  • 2+ years of experience in Big-data technologies like Apache spark/Databricks
  • Proficient in working with Redis and databases like Cassandra/Datastax
  • Must hold U.S. citizenship
Job Responsibility
Job Responsibility
  • Design, develop, and test software related to the cloud-based network configuration and reporting system
  • Solve complex problems and designing subsystems for Mist platform
  • Develop software for highly scalable and fault-tolerant cloud-scale distributed applications
  • Develop microservices using Python, and/or Go (golang)
  • Develop event-driven systems using Python and Java
  • Develop software for AIDE's real-time data pipeline and batch processing
  • Develop ETL pipelines aiding in training and inference of various ML models using big-data frameworks like Apache Spark
  • Build metrics, monitoring and structured logging into the product
  • Write unit, integration and functional tests
  • Participate in collaborative, DevOps style, lean practices
What we offer
What we offer
  • Health & Wellbeing benefits
  • Personal & Professional Development programs
  • Unconditional Inclusion environment
  • Comprehensive benefits suite supporting physical, financial and emotional wellbeing
  • Fulltime
Read More
Arrow Right
New

Senior C++ Developer

VarSome.com is the world’s leading website for professional human genetics. VarS...
Location
Location
Greece , Athens
Salary
Salary:
Not provided
saphetor.com Logo
Saphetor
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 4 years of commercial C++ development experience, using templates, STL containers, smart pointers, memory management and multithreading
  • BSc and/or MSc degree in Computer Science, Engineering or Mathematics
  • Hands-on experience with relational databases, optimizing queries and table schemas for optimal performance
  • Good experience with distributed cloud-based computing and big-data systems
  • Familiarity with agile methodologies and the ability to adapt to a fast-paced development environment, actively taking in code reviews, scrums, technical discussions etc
  • Very good understanding of fundamental application design principles and object-oriented design, in order to build & maintain a large high-quality code base
  • Excellent written & oral communication skills in English
Job Responsibility
Job Responsibility
  • Develop our core C++ applications and library for importing and analyzing genetic data
  • Write reusable, testable, and efficient code, including unit & regression tests
  • Take complete ownership of projects (ranging from a few days to a month) to deliver a working end-to-end implementation, including unit tests & testing
  • Optimize & architect our platform for maximum speed, high availability and scalability
  • Maintain & improve our internal high-performance clinical annotation tools and the custom databases built, optimized for genetics
  • Contribute to the documentation of software architecture, design and implementation details
What we offer
What we offer
  • A position in a fascinating healthcare growth domain, at the cutting edge of technology and research
  • A competitive compensation package combined with additional benefits
  • Endless learning opportunities, while transferring new technologies from academics to clinical practice all over the world
  • Fulltime
Read More
Arrow Right

Principal Software Engineer

Principal Software Engineer role at Hewlett Packard Enterprise to design, develo...
Location
Location
United States , San Jose
Salary
Salary:
148000.00 - 340500.00 USD / Year
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor or Masters degree in Computer science, Computer Engineering or a related field
  • 10+ years of experience in software engineering with a focus on Python, Go or Java
  • Strong understanding of RESTful API design and development
  • 2+ years of Experience working with large scale distributed systems based on either cloud technologies or Kubernetes
  • 2+ years of experience on event-driven technologies like Kafka and Apache Storm/Flink
  • 2+ years of experience in Big-data technologies like Apache spark/Databricks
  • Proficient in working with Redis and databases like Cassandra/Datastax
  • Excellent problem-solving and analytical skills
  • Strong communication and collaboration skills
Job Responsibility
Job Responsibility
  • Design, develop, and test software related to the cloud-based network configuration and reporting system
  • Solve complex problems and design subsystems for the Mist platform
  • Develop software for highly scalable and fault-tolerant cloud-scale distributed applications
  • Develop microservices using Python, and/or Go (golang)
  • Develop event-driven systems using Python and Java
  • Develop software for AIDE's real-time data pipeline and batch processing
  • Develop ETL pipelines aiding in training and inference of various ML models using big-data frameworks like Apache Spark
  • Build metrics, monitoring and structured logging into the product
  • Write unit, integration and functional tests
  • Participate in collaborative, DevOps style, lean practices
What we offer
What we offer
  • Health & Wellbeing benefits
  • Personal & Professional Development programs
  • Unconditional Inclusion environment
  • Comprehensive suite of benefits supporting physical, financial and emotional wellbeing
  • Fulltime
Read More
Arrow Right
New

Marketing Coordinator

We are looking for a highly organized and versatile Marketing Coordinator to joi...
Location
Location
United States , Lafayette
Salary
Salary:
27.00 - 32.00 USD / Hour
Xantrion
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous Marketing, Event and Social Media experience required, ideally within an IT Services & IT Consulting environment
  • Experience with marketing automation platforms, including HubSpot
  • PowerPoint and LinkedIn Platform Experience
  • Experience with Microsoft productivity suite
  • Proven experience successfully working remotely with the ability to operate independently
  • Excellent written communication & detail-orientation, especially when reviewing technical content or client-facing materials
  • Excellent project and time management skills with the ability to hit deadlines
Job Responsibility
Job Responsibility
  • Support the creation, editing, and distribution of marketing content & materials including case studies, articles, press releases, presentation decks, as well as website content & updates
  • Assist external Copywriters by posting approved case studies on Clutch and creating articles for postings
  • Assist SEO agency including updating Xantrion online profiles based on their requirements and providing relevant Hubspot data and reports
  • Assist with scheduling and posting approved social content on LinkedIn and other platforms to build authority and drive engagement. Respond to comments in a timely manner and monitor online reputation
  • Participate in weekly Sales & Marketing meetings ensuring the alignment of marketing efforts with sales goals and events
  • Participate in weekly Copy meetings, acting as a project coordinator to ensure deadlines are being met
  • Coordinate award data submissions and assist in performing company & competitive research as needed
  • Provide marketing and administrative support for industry, educational and marketing events, client webinars, and conferences
  • Manage event logistics including travel requirements, vendor coordination, registrations, marketing & event materials (signage, booth set-up, etc.), catering, AV and any other event needs
  • Assist Marketing agency with event giveaways including shipping & receiving and research support as needed
What we offer
What we offer
  • Xantrion will provide a cell phone, internet connection, and home office equipment allowance
  • Parttime
Read More
Arrow Right
New

Senior Data Engineer

As a Senior Data Engineer, you will be pivotal in designing, building, and optim...
Location
Location
United States
Salary
Salary:
102000.00 - 125000.00 USD / Year
wpromote.com Logo
Wpromote
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Information Technology, or a related field, or equivalent practical experience
  • 4+ years of experience in data engineering or a related field
  • Intermediate to advanced programming skills in Python
  • Proficiency in SQL and experience with relational databases
  • Strong knowledge of database and data warehousing design and management
  • Strong experience with DBT (data build tool) and test-driven development practices
  • Proficiency with at least 1 cloud database (e.g. BigQuery, Snowflake, Redshift, etc.)
  • Excellent problem-solving skills, project management habits, and attention to detail
  • Advanced level Excel and Google Sheets experience
  • Familiarity with data orchestration tools (e.g. Airflow, Dagster, AWS Glue, Azure data factory, etc.)
Job Responsibility
Job Responsibility
  • Developing data pipelines leveraging a variety of technologies including dbt and BigQuery
  • Gathering requirements from non-technical stakeholders and building effective solutions
  • Identifying areas of innovation that align with existing company and team objectives
  • Managing multiple pipelines across Wpromote’s client portfolio
What we offer
What we offer
  • Half-day Fridays year round
  • Unlimited PTO
  • Extended Holiday break (Winter)
  • Flexible schedules
  • Work from anywhere options*
  • 100% paid parental leave
  • 401(k) matching
  • Medical, Dental, Vision, Life, Pet Insurance
  • Sponsored life insurance
  • Short Term Disability insurance and additional voluntary insurance
  • Fulltime
Read More
Arrow Right
Welcome to CrawlJobs.com –
Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Australia Jobs Canada Jobs Poland Jobs Ireland Jobs Spain Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
© 2025 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy