Real-Time Data Engineer (Flink / Spark Streaming) Job at Bright Vision Technologies

Principal Data Engineer

PointClickCare is searching for a Principal Data Engineer who will contribute to...

Location

United States

Salary:

183200.00 - 203500.00 USD / Year

PointClickCare

Expiration Date

Until further notice

Requirements

Principal Data Engineer with at least 10 years of professional experience in software or data engineering, including a minimum of 4 years focused on streaming and real-time data systems
Proven experience driving technical direction and mentoring engineers while delivering complex, high-scale solutions as a hands-on contributor
Deep expertise in streaming and real-time data technologies, including frameworks such as Apache Kafka, Flink, and Spark Streaming
Strong understanding of event-driven architectures and distributed systems, with hands-on experience implementing resilient, low-latency pipelines
Practical experience with cloud platforms (AWS, Azure, or GCP) and containerized deployments for data workloads
Fluency in data quality practices and CI/CD integration, including schema management, automated testing, and validation frameworks (e.g., dbt, Great Expectations)
Operational excellence in observability, with experience implementing metrics, logging, tracing, and alerting for data pipelines using modern tools
Solid foundation in data governance and performance optimization, ensuring reliability and scalability across batch and streaming environments
Experience with Lakehouse architectures and related technologies, including Databricks, Azure ADLS Gen2, and Apache Hudi
Strong collaboration and communication skills, with the ability to influence stakeholders and evangelize modern data practices within your team and across the organization

Job Responsibility

Lead and guide the design and implementation of scalable streaming data pipelines
Engineer and optimize real-time data solutions using frameworks like Apache Kafka, Flink, Spark Streaming
Collaborate cross-functionally with product, analytics, and AI teams to ensure data is a strategic asset
Advance ongoing modernization efforts, deepening adoption of event-driven architectures and cloud-native technologies
Drive adoption of best practices in data governance, observability, and performance tuning for streaming workloads
Embed data quality in processing pipelines by defining schema contracts, implementing transformation tests and data assertions, enforcing backward-compatible schema evolution, and automating checks for freshness, completeness, and accuracy across batch and streaming paths before production deployment
Establish robust observability for data pipelines by implementing metrics, logging, and distributed tracing for streaming jobs, defining SLAs and SLOs for latency and throughput, and integrating alerting and dashboards to enable proactive monitoring and rapid incident response
Foster a culture of quality through peer reviews, providing constructive feedback and seeking input on your own work

What we offer

Benefits starting from Day 1!
Retirement Plan Matching
Flexible Paid Time Off
Wellness Support Programs and Resources
Parental & Caregiver Leaves
Fertility & Adoption Support
Continuous Development Support Program
Employee Assistance Program
Allyship and Inclusion Communities
Employee Recognition … and more!

Fulltime

Senior Data Engineer

We are looking for a Senior Data Engineer (SDE 3) to build scalable, high-perfor...

Location

India , Mumbai

Salary:

Not provided

Cogoport

Expiration Date

Until further notice

Requirements

6+ years of experience in data engineering, working with large-scale distributed systems
Strong proficiency in Python, Java, or Scala for data processing
Expertise in SQL and NoSQL databases (PostgreSQL, Cassandra, Snowflake, Apache Hive, Redshift)
Experience with big data processing frameworks (Apache Spark, Flink, Hadoop)
Hands-on experience with real-time data streaming (Kafka, Kinesis, Pulsar) for logistics use cases
Deep knowledge of AWS/GCP/Azure cloud data services like S3, Glue, EMR, Databricks, or equivalent
Familiarity with Airflow, Prefect, or Dagster for workflow orchestration
Strong understanding of logistics and supply chain data structures, including freight pricing models, carrier APIs, and shipment tracking systems

Job Responsibility

Design and develop real-time and batch ETL/ELT pipelines for structured and unstructured logistics data (freight rates, shipping schedules, tracking events, etc.)
Optimize data ingestion, transformation, and storage for high availability and cost efficiency
Ensure seamless integration of data from global trade platforms, carrier APIs, and operational databases
Architect scalable, cloud-native data platforms using AWS (S3, Glue, EMR, Redshift), GCP (BigQuery, Dataflow), or Azure
Build and manage data lakes, warehouses, and real-time processing frameworks to support analytics, machine learning, and reporting needs
Optimize distributed databases (Snowflake, Redshift, BigQuery, Apache Hive) for logistics analytics
Develop streaming data solutions using Apache Kafka, Pulsar, or Kinesis to power real-time shipment tracking, anomaly detection, and dynamic pricing
Enable AI-driven freight rate predictions, demand forecasting, and shipment delay analytics
Improve customer experience by providing real-time visibility into supply chain disruptions and delivery timeline
Ensure high availability, fault tolerance, and data security compliance (GDPR, CCPA) across the platform

What we offer

Work with some of the brightest minds in the industry
Entrepreneurial culture fostering innovation, impact, and career growth
Opportunity to work on real-world logistics challenges
Collaborate with cross-functional teams across data science, engineering, and product
Be part of a fast-growing company scaling next-gen logistics platforms using advanced data engineering and AI

Fulltime

Software Engineer, Data Infrastructure

The Data Infrastructure team at Figma builds and operates the foundational platf...

Location

United States , San Francisco; New York

Salary:

149000.00 - 350000.00 USD / Year

Figma

Expiration Date

Until further notice

Requirements

5+ years of Software Engineering experience, specifically in backend or infrastructure engineering
Experience designing and building distributed data infrastructure at scale
Strong expertise in batch and streaming data processing technologies such as Spark, Flink, Kafka, or Airflow/Dagster
A proven track record of impact-driven problem-solving in a fast-paced environment
A strong sense of engineering excellence, with a focus on high-quality, reliable, and performant systems
Excellent technical communication skills, with experience working across both technical and non-technical counterparts
Experience mentoring and supporting engineers, fostering a culture of learning and technical excellence

Job Responsibility

Design and build large-scale distributed data systems that power analytics, AI/ML, and business intelligence
Develop batch and streaming solutions to ensure data is reliable, efficient, and scalable across the company
Manage data ingestion, movement, and processing through core platforms like Snowflake, our ML Datalake, and real-time streaming systems
Improve data reliability, consistency, and performance, ensuring high-quality data for engineering, research, and business stakeholders
Collaborate with AI researchers, data scientists, product engineers, and business teams to understand data needs and build scalable solutions
Drive technical decisions and best practices for data ingestion, orchestration, processing, and storage

What we offer

equity
health, dental & vision
retirement with company contribution
parental leave & reproductive or family planning support
mental health & wellness benefits
generous PTO
company recharge days
a learning & development stipend
a work from home stipend
cell phone reimbursement

Fulltime

Senior Data Engineer

We are looking for a highly skilled Senior Data Engineer to join our team on a l...

Location

United States , Dallas

Salary:

Not provided

Robert Half

Expiration Date

Until further notice

Requirements

Bachelor's degree in Computer Science, Engineering, or a related discipline
At least 7 years of experience in data engineering
Strong background in designing and managing data pipelines
Proficiency in tools such as Apache Kafka, Airflow, NiFi, Databricks, Spark, Hadoop, Flink, and Amazon S3
Expertise in programming languages like Python, Scala, or Java for data processing and automation
Strong knowledge of both relational and NoSQL databases
Experience with Kubernetes-based data engineering and hybrid cloud environments
Familiarity with data modeling principles, governance frameworks, and quality assurance processes
Excellent problem-solving, analytical, and communication skills

Job Responsibility

Design and implement robust data pipelines and architectures to support data-driven decision-making
Develop and maintain scalable data pipelines using tools like Apache Airflow, NiFi, and Databricks
Implement and manage real-time data streaming solutions utilizing Apache Kafka and Flink
Optimize and oversee data storage systems with technologies such as Hadoop and Amazon S3
Establish and enforce data governance, quality, and security protocols
Manage complex workflows and processes across hybrid and multi-cloud environments
Work with diverse data formats, including Parquet and Avro
Troubleshoot and fine-tune distributed data systems
Mentor and guide engineers at the beginning of their careers

What we offer

Medical, vision, dental, and life and disability insurance
401(k) plan
Free online training

Fulltime

New