CrawlJobs Logo

Data Architect/Databricks Consultant

genzeon.com Logo

Genzeon

Location Icon

Location:
India , Hyderabad

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

We are seeking a specialized Databricks Architect with deep expertise in cost optimization and migration strategies, particularly focused on transitioning away from Databricks platforms. The ideal candidate will have extensive experience in Spark cluster solutions and a proven track record of reducing Databricks operational costs while architecting successful migration paths to alternative platforms.

Job Responsibility:

  • Conduct comprehensive cost analysis and auditing of existing Databricks deployments across multiple workspaces
  • Develop and implement aggressive cost reduction strategies targeting 30-50% savings through cluster optimization
  • Design and deploy automated cost monitoring solutions with real-time alerts and budget controls
  • Optimize cluster configurations, auto-scaling policies, and job scheduling to minimize compute costs
  • Implement spot instance strategies and preemptible VM usage for non-critical workloads
  • Establish cost allocation frameworks and implement chargeback mechanisms for business unit accountability
  • Create cost governance policies and developer guidelines to prevent cost overruns
  • Analyze and optimize storage costs including Delta Lake table optimization and data lifecycle management
  • Lead strategic initiatives to migrate workloads away from Databricks to cost-effective alternatives
  • Assess existing Databricks implementations and create detailed migration roadmaps to target platforms
  • Design migration architectures for transitioning to open-source Spark on Kubernetes, EMR, or other platforms
  • Develop automated migration tools and frameworks to minimize business disruption
  • Create comprehensive migration strategies including data export, job conversion, and dependency mapping
  • Establish parallel running environments to ensure zero-downtime migrations
  • Lead post-migration validation and performance benchmarking against original Databricks solutions
  • Document lessons learned and create reusable migration playbooks for future projects
  • Design high-performance, cost-optimized Spark cluster architectures outside of Databricks ecosystem
  • Implement custom Spark solutions on Kubernetes, YARN, and standalone cluster managers
  • Optimize Spark job performance through advanced tuning of memory management, serialization, and parallelism
  • Develop custom Spark operators and applications for specialized business use cases
  • Troubleshoot complex Spark performance bottlenecks and implement optimization strategies
  • Create cluster auto-scaling solutions and dynamic resource allocation frameworks
  • Design fault-tolerant Spark architectures with disaster recovery and high availability
  • Implement monitoring and alerting for Spark cluster health and job performance metrics
  • Collaborate with finance teams to develop multi-year cost reduction roadmaps
  • Evaluate and recommend alternative platforms based on cost-benefit analysis
  • Create business cases for migration projects with detailed ROI calculations
  • Establish technical debt reduction strategies related to Databricks dependencies
  • Partner with procurement teams on contract negotiations and vendor management

Requirements:

  • 8+ years of experience in big data architecture with focus on cost optimization
  • 5+ years of hands-on Databricks experience with proven cost reduction achievements
  • Demonstrated experience architecting and executing complete platform migrations from Databricks to alternative solutions with successful outcomes
  • 6+ years of advanced Apache Spark development and cluster management experience
  • Track record of achieving significant cost savings (minimum 40%+) in cloud data platforms
  • Expert knowledge of Databricks pricing models, compute types, and cost drivers
  • Experience with FinOps practices and cloud cost management tools
  • Proven ability to implement automated cost controls and budget management systems
  • Knowledge of alternative platforms and their cost structures (EMR, HDInsight, GCP Dataproc, etc.)
  • Deep expertise in migrating complex data workloads between different Spark platforms
  • Advanced knowledge of Spark internals, catalyst optimizer, and performance tuning
  • Experience with Kubernetes-based Spark deployments and container orchestration
  • Proficiency in infrastructure-as-code for multi-cloud Spark cluster provisioning
  • Strong background in data pipeline migration and ETL/ELT conversion strategies
  • Expert-level proficiency in Scala, Python, and Java for Spark development
  • Advanced SQL skills and experience with multiple database technologies
  • Experience with open-source alternatives to Databricks (Apache Spark, Delta Lake OSS, MLflow OSS)
  • Knowledge of streaming platforms (Kafka, Kinesis, Pulsar) and real-time architectures
  • Proficiency with monitoring tools (Prometheus, Grafana, ELK stack)
  • Bachelor's degree in Computer Science, Engineering, Information Technology, or related field

Nice to have:

  • Databricks certifications combined with experience in competitive platforms
  • Cloud cost management certifications (AWS Cost Optimization, Azure Cost Management)
  • Experience with vendor negotiations and contract optimization
  • Background in building business cases for platform migrations
  • Knowledge of data governance during platform transitions
  • Experience with Apache Iceberg, Hudi, or other open table formats as Delta Lake alternatives
  • Master's degree preferred but not required with equivalent experience

Additional Information:

Job Posted:
December 08, 2025

Icon
Genzeon - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Data Architect/Databricks Consultant

New

Supplier Development Manager

Airbus Commercial Aircraft is looking for a Supplier Development Manager to join...
Location
Location
United States , Herndon; Atlanta; Washington, D.C.; Charlotte; Chicago; NYC; Philadelphia
Salary
Salary:
Not provided
airbus.com Logo
Airbus
Expiration Date
March 07, 2026
Flip Icon
Requirements
Requirements
  • A Bachelors' Degree in Engineering or a related field or an equivalent combination of education and experience
  • A minimum of eight (8) years in professional experience in Procurement, Supply Chain, and/or Program Management
  • Must have demonstrable experience applying quality management tools/techniques
  • Supplier Recovery and Crisis Management
  • Advanced Product Quality Planning (APQP) and Six Sigma
  • Practical Problem Solving
  • Lean methodologies
  • Transfers of Work between Production sites and/or to other countries
  • Must be authorized to work in the United States.
Job Responsibility
Job Responsibility
  • Supplier development management
  • Stakeholder management
  • Special projects
  • Securing aircraft deliveries on-time, with perfect quality
  • Traveling weekly to Airbus suppliers & their subtiers
  • Conducting assessments
  • Solving complex supply chain or quality management issues.
What we offer
What we offer
  • Competitive base salary
  • Incentive compensation which may include profit sharing schemes
  • Retirement savings plan
  • Ability to participate in an Employee Stock Ownership Plan (“ESOP”)
  • Paid time off including personal time, holidays
  • Generous paid parental leave program
  • Comprehensive insurance coverage including medical, prescription, dental, vision, life, disability
  • Employee Assistance Plan (“EAP”)
  • Other supplemental benefit coverages
  • Upskilling and development opportunities through our global Leadership University
  • Fulltime
Read More
Arrow Right
New

Junior Commercial Strategy Analyst

Hybrid role based in London. We’re looking for a Junior Commercial Strategy Anal...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
Uncapped
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Undergraduate degree and/or Masters in Mathematics, Statistics, Econometrics or a related analytical field
  • Strong analytical skills
  • Confident with Python and SQL
  • Builder mindset
  • Professional experience not required
Job Responsibility
Job Responsibility
  • Build Data Driven Strategies
  • Own Foundational Data Projects
  • Structure & Build Reporting
  • Generate Actionable Insights
  • Cross-Functional Collaboration
What we offer
What we offer
  • Unlimited holiday
  • Competitive compensation plan
  • Personal growth fund
  • Global Impact
  • Access to workspaces in Warsaw, London, Atlanta, and Toronto
  • Fulltime
Read More
Arrow Right
New

Certified Surgical Technologist

Location
Location
United States , Chicago
Salary
Salary:
25.30 - 37.95 USD / Hour
advocatehealth.com Logo
Advocate Health Care
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Graduate of surgical technologist program OR 2 years of surgical scrub experience
  • CPR certified
  • Certified Surgical Technologist certification will be obtained by the employee by sitting for the certification exam within 3 months of employment
  • Must be willing to work with exposures to body fluids, waste and chemicals, radiation, hazardous materials, anesthesia gases and life threatening diseases
  • On-call availability (nights, weekends, holidays)
  • Rotating shifts
  • Ability to move heavy patients and equipment and to stand for long periods of time
  • Flexibility to deal with stressful, changing environments
  • Ability to respond quickly to emergencies
  • Ability to communicate, cooperate and work well with others
Job Responsibility
Job Responsibility
  • Performs duties in the scrub role as assigned
  • Completely prepares sterile field in a timely manner
  • maintains the sterile field and boundaries prior to and during procedure
  • Complies with departmental policy and procedures regarding specimens, counts, and disposition of contaminated instruments
  • Assists the physician by sponging, suctioning, retracting, and cutting suture as required and instructed by the physician
  • Acts as preceptor, sharing knowledge in a supportive/educational manner
  • Maintains on-call availability
  • Assists in emergency situations as assigned, demonstrating appropriate actions and prioritizing to meet patient's condition
  • Maintains a safe work environment according to regulatory agency standards
  • Anticipates the needs of the physician as demonstrated by having all items required readily available on the sterile field
What we offer
What we offer
  • $6,000 sign-on bonus
  • Paid Time Off programs
  • Health and welfare benefits such as medical, dental, vision, life, and Short- and Long-Term Disability
  • Flexible Spending Accounts for eligible health care and dependent care expenses
  • Family benefits such as adoption assistance and paid parental leave
  • Defined contribution retirement plans with employer match and other financial wellness programs
  • Educational Assistance Program
  • Premium pay such as shift, on call, and more based on a teammate's job
  • Incentive pay for select positions
  • Opportunity for annual increases based on performance
  • Fulltime
Read More
Arrow Right
New

Senior Software Engineer- Platform Security

Join the Microsoft Device Ecosystem and Platform (MDEP) team, where we are build...
Location
Location
United States , Mountain View
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • 8+ years of software development experience, with significant experience contributing to or leading architectural design
  • Experience with AOSP security features such as device integrity, cryptography, Keymint, and Keystore
  • Expertise in device integrity mechanisms, key attestation processes, key provision strategies, and the management of trusted applications within AOSP
  • Ability to develop and manage trusted applications that operate within a TEE
  • Capability to design and develop robust security solutions that align with industry standards and best practices
  • Strong analytical and problem-solving skills, with the ability to troubleshoot complex software and security issues
  • Leadership skills to guide a team in the development of secure software solutions
  • Strong communication skills with proven track record of representing security technology with customers and device manufacturers
Job Responsibility
Job Responsibility
  • Design, develop, and maintain AOSP security features, applications, and frameworks using Java, Kotlin, C/C++, and other technologies
  • Work across drivers, tools, Linux Kernel, trusted applications, AOSP Framework, and AOSP services
  • Collaborate with developers, testers, product managers, and security experts to ensure the quality, performance, and security of the AOSP platform and applications
  • Research and implement the latest security best practices, standards, and technologies for AOSP development
  • Troubleshoot, debug, and fix security issues and vulnerabilities in AOSP applications and systems
  • Represent the Microsoft MDEP team in various forums with partners, providing industry-leading technical guidance
  • Participate in code reviews to ensure code quality and share knowledge with the team
  • Demonstrate excellent communication skills with the ability to articulate technical design, implementation, and issues clearly and concisely
  • Stay updated on AOSP operating system updates, security trends, and industry best practices
  • Fulltime
Read More
Arrow Right
New

Assistant Store Manager

Are you looking for an opportunity to progress in store management? We’re lookin...
Location
Location
United Kingdom , Edgware
Salary
Salary:
23241.40 GBP / Year
bhf.org.uk Logo
British Heart Foundation
Expiration Date
February 17, 2026
Flip Icon
Requirements
Requirements
  • Experience in a customer facing role
  • Supervisory experience
  • Commercially driven to encourage new ideas
  • Inclusive approach to developing teams
  • Passion for delivering exceptional customer service and achieving the highest retail standards
  • Results driven but with a recognition of right result, right way.
Job Responsibility
Job Responsibility
  • Support the Store Manager with the day to day running of the store and take full responsibility in their absence
  • Ensuring highest standard of customer service
  • Achieving targets
  • Maintaining a high standard of visual merchandising
  • Maximising sales through physical and digital channels
  • Supporting with the recruitment and development of volunteers
  • Achieving expectations within campaign activities
  • Working with the manager to generate stock
  • Work weekends and bank holidays on a rota basis.
What we offer
What we offer
  • 38 days annual leave (plus the option to buy and sell leave)
  • Wagestream - claim early access to your wages as you earn them
  • Holistic support leave of up to 10 additional days off each year
  • Enhanced family policies (maternity, paternity and adoption leave)
  • 25% staff discount
  • Health cash plan (Dental, Optical, Therapies, etc)
  • Access to a 24/7 Virtual GP and Employee Assistance Programme (EAP)
  • Pension with employer contribution of up to 10%
  • Cycle to work scheme
  • Discounts on gym memberships
  • Fulltime
Read More
Arrow Right
New

Desktop Support Analyst

IT Support Technician duties involve ensuring technology operates efficiently da...
Location
Location
United States , Las Vegas
Salary
Salary:
21.23 - 39.43 USD / Hour
aristocratgaming.com Logo
Aristocrat Gaming
Expiration Date
February 09, 2026
Flip Icon
Requirements
Requirements
  • High school diploma or equivalent experience
  • Associate degree in Information Technology or related field is helpful
  • A+, Network+, Security+ certification preferred
  • MCP certification or equivalent preferred
  • ITIL certification or experience is a plus
  • 1-2 yrs. Helpdesk/Call Center experience in a large Enterprise environment preferred
  • Technical knowledge across a broad range of IT disciplines
  • Strong verbal and written communication skills
  • Strong customer focus and attention to detail
  • Must be customer service oriented
Job Responsibility
Job Responsibility
  • Deliver exceptional, high-touch IT support to internal employees
  • Act as a trusted technical partner to the Global Aristocrat IT team
  • Manage the full lifecycle of incidents and service requests
  • Conduct sophisticated fixing and root cause analysis on local and remote systems
  • Contribute to the development and improvement of IT procedures, documentation, and knowledge bases
  • Provide expert-level support for corporate communication tools, devices, and platforms
  • Configure, deploy, and maintain end-user systems
  • Deliver responsive, multi-channel support via phone, email, walk-up, remote sessions, and virtual smart bar environments
  • Serve as a domain specialist and mentor for junior technicians
  • Partner with IT leadership to identify process enhancements
What we offer
What we offer
  • Health, dental, and vision insurance
  • Paid time off
  • 401(k) plan with employer matching
  • Global career opportunities
  • Fulltime
!
Read More
Arrow Right
New

Customer Support Collections Specialist

As a Customer Support Collections Specialist and be a key player in delivering e...
Location
Location
United States , Wilmington
Salary
Salary:
Not provided
barclays.co.uk Logo
Barclays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Working experience within collections or customer service
  • Moderate computer literacy
  • Verbal and written communication skills
Job Responsibility
Job Responsibility
  • Provision of excellent customer service, adherent to Barclays policies and procedures, by responding and resolving customer requests and inquiries effectively
  • Collaboration with internal stakeholders, including the relevant risk management teams, to align collection activities and comply with regulatory requirements and internal processes
  • Collaboration across functional projects and initiatives to support the broader business operations team
  • Support the business operation function team with day-to-day collection activities including customer service, risk management and compliance
  • Communication to customers with outstanding loans and/or credit card payments to negotiate payment arrangements and resolve any financial delinquencies
  • Maintenance of accurate and up-to-date records of all collection activities, including customer interactions, payment plans and account status updates
  • To collaborate with customers and negotiate repayment arrangements on outstanding loans/credit card repayments
  • Fulltime
Read More
Arrow Right
New

Parts Advisor

We are excited to announce a new opportunity for a dedicated and customer-focuse...
Location
Location
United Kingdom , Newton Abbot
Salary
Salary:
30000.00 GBP / Year
adamsmorey.com Logo
ADAMS MOREY LTD
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Excellent customer service and strong administration skills
  • A friendly, can do attitude, with the ability to work well under pressure
  • Some working knowledge of the motor vehicle sector or automotive trade
  • Experience working to performance targets
  • Previous experience in a similar role would be advantageous
  • Knowledge of Autoline, Kerridge or Keyloop would be an advantage
Job Responsibility
Job Responsibility
  • To provide excellent customer service, (must be customer focused)
  • Identify parts for front counter sales via phone, emails, Webshop orders, and walk ins, and process orders
  • Purchase parts at best price
  • Receipting parts
  • Parts Cross referencing
  • Processing credits
  • Checking off stock and accurately put away in correct bin locations when required
  • Pick Orders when required
What we offer
What we offer
  • 30 days holidays including 8 bank holidays
  • Company sick pay, that increases with length of service up to 13 weeks
  • Company pension with the option to salary sacrifice
  • Where salary sacrifice takes place, 50% NI saving reinvested into your pension pot
  • Genuine career development and pathways with DAF Academy
  • Award winning apprenticeship program
  • Apprentice equipment scheme
  • Employee Assistance program
  • Cycle to work scheme
  • My Staff shop – access to hundreds of discounts
  • Fulltime
Read More
Arrow Right
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Canada Jobs Australia Jobs Germany Jobs Ireland Jobs Poland Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
CrawlJobs Group
ITFlashcards AllDevBlogs
© 2026 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy