This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
(Locals or Nearby resources only). You will work with technologies that include Python, AWS, Snowflake, Databricks, and Airflow. Tech stack includes Airflow, Spark, Databricks, Delta Lake, and Snowflake. This client requires that a background check be completed.
Job Responsibility:
Contribute to maintaining, updating, and expanding existing Core Data platform data pipelines
Build and maintain APIs to expose data to downstream applications
Develop real-time streaming data pipelines
Collaborate with product managers, architects, and other engineers to drive the success of the Core Data platform
Contribute to developing and documenting both internal and external standards and best practices for pipeline configurations, naming conventions, and more
Ensure high operational efficiency and quality of the Core Data platform datasets to ensure our solutions meet SLAs and project reliability and accuracy to all our stakeholders (Engineering, Data Science, Operations, and Analytics teams)
Requirements:
7+ years of data engineering experience developing large data pipelines
Proficiency in at least one major programming language (e.g. Python, Java, Scala)
Hands-on production environment experience with distributed processing systems such as Spark
Hands-on production experience with data pipeline orchestration systems such as Airflow for creating and maintaining data pipelines
Experience with at least one major Massively Parallel Processing (MPP) or cloud database technology (Snowflake, Databricks, Big Query)
Experience in developing APIs with GraphQL
Advance understanding of OLTP vs OLAP environments
Candidates must work W2, no Corp 2 Corp
US Citizen, Green Card Holder, H4-EAD, TN-Visa
Airflow
Apache Spark
Snowflake OR Databricks
Data Modeling
Nice to have:
Graph Database experience a plus
Realtime Event Streaming experience a plus
What we offer:
3 levels of medical insurance for you and your family
Dental insurance for you and your family
401k
Overtime
Sick leave policy: accrue 1 hour for every 30 hours worked up to 48 hours
Welcome to CrawlJobs.com – Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
We use cookies to enhance your experience, analyze traffic, and serve personalized content. By clicking “Accept”, you agree to the use of cookies.