CrawlJobs Logo

Bigdata Consultant

oceanbluecorp.com Logo

Ocean Blue Solutions

Location Icon

Location:
United States, Columbus

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

The Technical Specialist will be responsible for Medicaid Enterprise Data Warehouse design, development, implementation, migration, maintenance and operation activities. Works closely with Data Governance and Analytics team. Will be one of the key technical resource for data warehouse projects for various Enterprise Data Warehouse projects and building critical Datamarts, data ingestion to Big Data platform for data analytics and exchange with State and Medicaid partners. This position is a member of Medicaid ITS and works closely with the Business Intelligence & Data Analytics team.

Job Responsibility:

  • Participate in Team activities, Design discussions, Stand up meetings and planning Review with team
  • Perform data analysis, data profiling, data quality and data ingestion in various layers using big data/Hadoop/Hive/Impala queries, PySpark programs and UNIX shell scripts
  • Follow the organization coding standard document, Create mappings, sessions and workflows as per the mapping specification document
  • Perform Gap and impact analysis of ETL and IOP jobs for the new requirement and enhancements
  • Create jobs in Hadoop using SQOOP, PYSPARK and Stream Sets to meet the business user needs
  • Create mockup data, perform Unit testing and capture the result sets against the jobs developed in lower environment
  • Updating the production support Run book, Control M schedule document as per the production release
  • Create and update design documents, provide detail description about workflows after every production release
  • Continuously monitor the production data loads, fix the issues, update the tracker document with the issues, Identify the performance issues
  • Performance tuning long running ETL/ELT jobs by creating partitions, enabling full load and other standard approaches
  • Perform Quality assurance check, Reconciliation post data loads and communicate to vendor for receiving fixed data
  • Participate in ETL/ELT code review and design re-usable frameworks
  • Create Remedy/Service Now tickets to fix production issues, create Support Requests to deploy Database, Hadoop, Hive, Impala, UNIX, ETL/ELT and SAS code to UAT environment
  • Create Remedy/Service Now tickets and/or incidents to trigger Control M jobs for FTP and ETL/ELT jobs on ADHOC, daily, weekly, Monthly and quarterly basis as needed
  • Model and create STAGE / ODS /Data Warehouse Hive and Impala tables as and when needed
  • Create Change requests, workplan, Test results, BCAB checklist documents for the code deployment to production environment and perform the code validation post deployment
  • Work with Hadoop Admin, ETL and SAS admin teams for code deployments and health checks
  • Create re-usable UNIX shell scripts for file archival, file validations and Hadoop workflow looping
  • Create re-usable framework for Audit Balance Control to capture Reconciliation, mapping parameters and variables, serves as single point of reference for workflows
  • Create PySpark programs to ingest historical and incremental data
  • Create SQOOP scripts to ingest historical data from EDW Oracle database to Hadoop IOP, created HIVE tables and Impala views creation scripts for Dimension tables
  • Participate in meetings to continuously upgrade the Functional and technical expertise

Requirements:

  • 8+ years of experience with Big Data, Hadoop on Data Warehousing or Data Integration projects
  • Analysis, Design, development, support and Enhancements of ETL/ELT in data warehouse environment with Cloudera Bigdata Technologies (with a minimum of 7 or more years’ experience in Hadoop, MapReduce, Sqoop, PySpark, Spark, HDFS, Hive, Impala, StreamSets, Kudu, Oozie, Hue, Kafka, Yarn, Python, Flume, Zookeeper, Sentry, Cloudera Navigator) along with Oracle SQL/PL-SQL, Unix commands and shell scripting
  • Strong development experience (minimum of 7 or more years) in creating Sqoop scripts, PySpark programs, HDFS commands, HDFS file formats (Parquet, Avro, ORC etc.), StreamSets pipeline creation, jobs scheduling, hive/impala queries, Unix commands, scripting and shell scripting etc.
  • Writing Hadoop/Hive/Impala scripts (minimum of 7 or more years’ experience) for gathering stats on table post data loads
  • Strong SQL experience (Oracle and Hadoop (Hive/Impala etc.))
  • Writing complex SQL queries and performed tuning based on the Hadoop/Hive/Impala explain plan results
  • Proven ability to write high quality code
  • Experience building datasets and familiarity with PHI and PII data
  • Expertise implementing complex ETL/ELT logic
  • Develop and enforce strong reconciliation process
  • Accountable for ETL/ELT design documentation
  • Good knowledge of Big Data, Hadoop, Hive, Impala database, data security and dimensional model design
  • Basic knowledge of UNIX/LINUX shell scripting
  • Utilize ETL/ELT standards and practices towards establishing and following centralized metadata repository
  • Good experience in working with Visio, Excel, PowerPoint, Word, etc.
  • Effective communication, presentation and organizational skills
  • Familiar with Project Management methodologies like Waterfall and Agile
  • Ability to establish priorities and follow through on projects, paying close attention to detail with minimal supervision
  • Required Education: BS/BA degree or combination of education & experience

Nice to have:

  • Demonstrate effective leadership, analytical and problem-solving skills
  • Required excellent written and oral communication skills with technical and business teams
  • Ability to work independently, as well as part of a team
  • Stay abreast of current technologies in area of IT assigned
  • Establish facts and draw valid conclusions
  • Recognize patterns and opportunities for improvement throughout the entire organization
  • Ability to discern critical from minor problems and innovate new solutions

Additional Information:

Job Posted:
December 11, 2025

Employment Type:
Fulltime
Work Type:
On-site work
Icon
View All Jobs In This Company
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Bigdata Consultant

Big Data Engineer

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4-8 years of experience in software development, building large scale distributed data processing systems or large-scale applications
  • Designing & developing Big Data solutions with at least one end to end implementation
  • Strong Hands-on experience in following technical skills: Apache Spark, Scala/ Java, XML/ JSON/ Parquet/ Avro, SQL, Linux, Hadoop Ecosystem (HDFS, Spark, Impala, HIVE, HBASE etc.), Kafka
  • Performance analysis, troubleshooting and issue resolution and Exposure to latest Cloudera offerings like Ozone, Iceberg
  • Intermediate level experience in Applications Development role
  • Consistently demonstrates clear and concise written and verbal communication
  • Demonstrated problem-solving and decision-making skills
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Design, development of BigData applications/ pipelines using Spark, Scala, SQL, Pyspark, Python, Java
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
  • Fulltime
Read More
Arrow Right

Data Analytics Senior Analyst

The Data Analytics Senior Analyst is a seasoned professional role. Applies in-de...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years experience using tools for statistical modeling of large data sets
  • Ability to effectively use complex analytical, interpretive and problem solving techniques
  • Demonstrated interpersonal, verbal and written communication skills
  • Bachelor’s/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flows
  • Coordinates and contribute to the objectives of data science initiatives and overall business through leveraging in-depth understanding of how areas collectively integrate within the sub-function
  • Assumes informal/formal leadership role through coaching and training of new recruits
  • Significantly influences decisions, work, and performance of all teams through advice, counsel and/or facilitating services to others in the business
  • Conducts strategic data analysis, identifies insights and implications and make strategic recommendations, develops data displays that clearly communicate complex analysis
  • Mines and analyzes data from various banking platforms to drive optimization and improve data quality
  • Develop automated control validations for the Bigdata systems, ensuring Quality and Control
  • Delivers analytics initiatives to address business problems with the ability to identify data required, assess time & effort required and establish a project plan
  • Strong expertise on Data analytics to identify different Patterns, duplicates, frequency, consistency, accuracy, completeness and referential integrity of data
  • Expertise in designing and developing advanced Data automation frameworks
  • Fulltime
Read More
Arrow Right

Senior Bigdata Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8 - 10 years of relevant experience
  • Experience in systems analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Programming Languages: Python, PySpark
  • Data Lake Table Format: Apache Iceberg
  • Data Orchestration: Apache Airflow
  • Data Visualization: Tableau
  • Big Data Processing: Apache Spark
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
What we offer
What we offer
  • Equal employment opportunity
  • Fulltime
Read More
Arrow Right

Applications Development Senior Programmer Analyst

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of relevant experience
  • Experience in systems analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Pyspark Developer

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of relevant experience in the Financial Service industry
  • Intermediate level experience in Tableau and Python Applications Development role
  • Consistently demonstrates clear and concise written and verbal communication
  • Demonstrated problem-solving and decision-making skills
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
  • 5-8 years of experience in Python
  • 3 year experience using PySpark
  • 3 year experience in developing Kafka data pipelines
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Applications Development Senior Programmer Analyst

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8-12 years of experience on following areas
  • Strong in Python programing
  • Strong in Object oriented programing & Data Structures
  • Good knowledge of design patterns
  • Experience with Python frameworks - flask, Django
  • Strong technical skills for Big data Pyspark & Hadoop ecosystem components (HDFS, Hbase, Hive, Pig)
  • Strong experience in pyspark
  • Strong knowledge of REST web services
  • Spark Performance tuning and optimization techniques
  • Databases knowledge should extend to PL SQL, SQL and Transact-SQL. Oracle is a plus
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications.
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision.
  • Can exercise independence of judgement and autonomy.
  • Acts as SME to senior stakeholders and /or other team members.
  • Fulltime
Read More
Arrow Right

Applications Development Senior Programmer Analyst

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8-13 years of experience on following areas
  • Strong in Python programming
  • Strong in Object oriented programming & Data Structures
  • Good knowledge of design patterns
  • Experience with Python frameworks - flask, Django
  • Strong technical skills for Big data Pyspark & Hadoop ecosystem components (HDFS, Hbase, Hive, Pig)
  • Strong experience in pyspark
  • Strong knowledge of REST web services
  • Spark Performance tuning and optimization techniques
  • Databases knowledge should extend to PL SQL, SQL and Transact-SQL. Oracle is a plus
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications.
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision.
  • Can exercise independence of judgement and autonomy.
  • Acts as SME to senior stakeholders and /or other team members.
  • Fulltime
Read More
Arrow Right

Python Developer

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of experience on following areas
  • Strong in Python programming
  • Strong in Object oriented programming & Data Structures
  • Good knowledge of design patterns
  • Experience with Python frameworks - flask, Django
  • Strong technical skills for Big data Pyspark & Hadoop ecosystem components (HDFS, Hbase, Hive, Pig)
  • Strong experience in pyspark
  • Strong knowledge of REST web services
  • Spark Performance tuning and optimization techniques
  • Databases knowledge should extend to PL SQL, SQL and Transact-SQL. Oracle is a plus
Job Responsibility
Job Responsibility
  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
What we offer
What we offer
  • Global Benefits
  • We bring the best to our people
  • We put our employees first and provide the best-in-class benefits they need to be well, live well and save well
  • Fulltime
Read More
Arrow Right
Welcome to CrawlJobs.com –
Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Australia Jobs Canada Jobs Poland Jobs Ireland Jobs Spain Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
© 2025 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy