CrawlJobs Logo

Head of Inference Kernels

etched.com Logo

Etched

Location Icon

Location:
United States , San Jose

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

200000.00 - 300000.00 USD / Year

Job Description:

As a core member of the team, you will play a pivotal role in leading a high-performing team to build a suite of optimized kernels and implement highly optimized inference stacks for a variety of state-of-the-art transformer models (e.g., Llama-3, Llama-4, Deepseek-R1, Qwen-3, Stable Diffusion-3 etc.). You will be responsible for managing and scaling a high-performance team to pioneer novel model mapping strategies, while co-designing inference time algorithms (e.g., speculative and parallel decoding, prefill-decode disaggregation etc.).

Job Responsibility:

  • Architect Best-in-Class Inference Performance on Sohu: Deliver continuous batching throughput exceeding B200 by ≥10x on priority workloads
  • Develop Best-in-Performance Inference Mega Kernels: Develop complex, fused kernels that increase chip utilization and reduce inference latency, and validate these optimizations through benchmarking and regression-tested in production pipelines
  • Architect Model Mapping Strategies: Develop system level optimizations using a mix of techniques such tensor parallelism and expert parallelism for optimal performance
  • Hardware-Software Co-design of Inference-time Algorithmic Innovation: Develop and deploy production-ready inference-time algorithmic improvements (e.g., speculative decoding, prefill-decode disaggregation, KV cache offloading)
  • Build Scalable Team and Roadmap: Grow and retain a team of high-performing inference optimization engineers
  • Cross-Functional Performance Alignment: Ensure inference stack and performance goals are aligned with the software infrastructure teams, GTM and hardware teams for future generations of our hardware

Requirements:

  • Experience in designing and optimizing GPU kernels for deep learning on GPUs using CUDA, and assembly (ASM)
  • Experience with low-level programming to maximize performance for AI operations, leveraging tools like Compute Kernel (CK), CUTLASS, and Triton for multi-GPU and multi-platform performance
  • Deep fluency with transformer inference architecture, optimization levers, and full-stack systems (e.g., vLLM, custom runtimes)
  • History of delivering tangible perf wins on GPU hardware or custom AI accelerators
  • Solid understanding of roofline models of compute throughput, memory bandwidth and interconnect performance
  • Experienced in running large-scale workloads on heterogeneous compute clusters, optimizing for efficiency and scalability of AI workloads
  • Scopes projects crisply, sets aggressive but realistic milestones, and drives technical decision-making across the team
  • Anticipates blockers and shifts resources proactively

Nice to have:

  • Experience with implementation of state-of-the-art reasoning and chain-of-thought models at production scale
  • Experience with implementation of newer AI compute operations on hardware (e.g., flash attention, long-context attention variants and alternatives)
  • Analyzed and implemented strategies such as KV-cache offloading for efficient compute resource management
  • Familiarity with linear algebra (e.g. matrix decomposition, alternatives bases for vector spaces, matrix rank and its implications)
  • Managed lean, high-performing engineering teams and drove execution on timelines with high quality outcomes
What we offer:
  • Medical, dental, and vision packages with generous premium coverage
  • $500 per month credit for waiving medical benefits
  • Housing subsidy of $2k per month for those living within walking distance of the office
  • Relocation support for those moving to San Jose (Santana Row)
  • Various wellness benefits covering fitness, mental health, and more
  • Daily lunch + dinner in our office
  • significant equity package

Additional Information:

Job Posted:
February 18, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Head of Inference Kernels

New

Engineering Manager - Inference

We are looking for an Inference Engineering Manager to lead our AI Inference tea...
Location
Location
United States , San Francisco
Salary
Salary:
300000.00 - 385000.00 USD / Year
perplexity.ai Logo
Perplexity
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of engineering experience with 2+ years in a technical leadership or management role
  • Deep experience with ML systems and inference frameworks (PyTorch, TensorFlow, ONNX, TensorRT, vLLM)
  • Strong understanding of LLM architecture: Multi-Head Attention, Multi/Grouped-Query Attention, and common layers
  • Experience with inference optimizations: batching, quantization, kernel fusion, FlashAttention
  • Familiarity with GPU characteristics, roofline models, and performance analysis
  • Experience deploying reliable, distributed, real-time systems at scale
  • Track record of building and leading high-performing engineering teams
  • Experience with parallelism strategies: tensor parallelism, pipeline parallelism, expert parallelism
  • Strong technical communication and cross-functional collaboration skills
Job Responsibility
Job Responsibility
  • Lead and grow a high-performing team of AI inference engineers
  • Develop APIs for AI inference used by both internal and external customers
  • Architect and scale our inference infrastructure for reliability and efficiency
  • Benchmark and eliminate bottlenecks throughout our inference stack
  • Drive large sparse/MoE model inference at rack scale, including sharding strategies for massive models
  • Push the frontier with building inference systems to support sparse attention, disaggregated pre-fill/decoding serving, etc.
  • Improve the reliability and observability of our systems and lead incident response
  • Own technical decisions around batching, throughput, latency, and GPU utilization
  • Partner with ML research teams on model optimization and deployment
  • Recruit, mentor, and develop engineering talent
What we offer
What we offer
  • Equity
  • Health
  • Dental
  • Vision
  • Retirement
  • Fitness
  • Commuter and dependent care accounts
  • Fulltime
Read More
Arrow Right
New

Project Engineer - Commercial Construction - FIFO

Join a leading Indigenous-led commercial construction company dedicated to deliv...
Location
Location
Australia , Bulimba
Salary
Salary:
160000.00 - 190000.00 AUD / Year
https://www.randstad.com Logo
Randstad
Expiration Date
February 26, 2026
Flip Icon
Requirements
Requirements
  • Tertiary or trade Qualification in Construction
  • Minimum 2 years experience in a site/project Engineer role within the Australian commercial construction industry working for a head contractor
Job Responsibility
Job Responsibility
  • Contribute to site meetings and work collaboratively with upper management to ensure
  • Compile and complete documentation related to your designated workflow
  • Working within the engineering team managing individual responsibilities, assisting with documentation
  • Provide leadership in health, safety, quality and environmental management
  • Review project documents including estimates, design drawing, proposals and contracts per management direction
  • Stakeholder and resource management - Ensuring relevant labour is on site to deliver key project objectives, and ensuring quality is maintained throughout
  • Reporting directly to your site based Senior Project Engineer & Project Manager you will have all the support required but also show a capability to work autonomously as required
  • Manage project schedules, procurement schedules, and subcontractor coordination
  • Interpret engineering drawings and specifications to ensure site works meet all technical requirements
  • Implement and maintain rigorous ISO-certified systems for Safety (45001), Quality (9001), and Environment (14001)
What we offer
What we offer
  • All flights, site accommodation, uplift and LAFHA are provided
  • Gain experience in the highly secure Defence sector and specialized government infrastructure
  • Work within a values-driven organization that balances commercial excellence with community impact
Read More
Arrow Right
New

HR Project Specialist

We are looking for an HR Project Specialist - (Maternity Leave Cover) to join ou...
Location
Location
Portugal , Lisbon
Salary
Salary:
Not provided
airbus.com Logo
Airbus
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Ideally 2 /4 years of experience in HR
  • Ability to think and act with a project management mindset, structuring initiatives with clear timelines, tasks and deliverables
  • Proven ability to collaborate with diverse teams and manage multiple projects across departments
  • Passion for creating a great employee experience and fostering a strong company culture through effective teamwork
  • Strong stakeholder management, communication and interpersonal skills to work across teams
  • Continuous Improvement driven mindset to refine HR initiatives
  • Excellent verbal and written communication skills in English
  • Ability to act independently by prioritising work, problem-solving and taking ownership of tasks and responsibilities.
Job Responsibility
Job Responsibility
  • Onboarding experience - Manage and ensure a seamless onboarding journey that enhances new hire integration and engagement
  • Employee experience - Develop and implement initiatives that foster a positive, engaging and inclusive work environment
  • Trainee and internship programmes - Manage and enhance our trainee programs, ensuring alignment with broader business goals
  • HR project implementation - Lead and support additional HR projects as required, collaborating with other departments to ensure smooth execution and respect for timelines.
What we offer
What we offer
  • The chance to work in a multicultural environment
  • The opportunity to grow your skills and capabilities thanks to our Airbus learning offer (digital solutions, classroom sessions, certified training) and career opportunities across the globe
  • A hybrid working model, allowing you to combine onsite and offsite work
  • A modern office, well connected to public transportation
  • A motivated and fun crew to grow and build and shape the GBS together
  • An intense and exciting onboarding experience.
  • Fulltime
Read More
Arrow Right
New

Staff Pharmacist

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , Naples
Salary
Salary:
60.00 - 76.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
March 28, 2026
Flip Icon
Requirements
Requirements
  • Active Pharmacy License in the state in which you are employed
  • Not on the DEA Excluded Parties List
  • Immunization Certification through an accredited organization (e.g., APhA)
  • No pending felony charges or convictions for criminal offenses involving controlled substances
Job Responsibility
Job Responsibility
  • Patient Safety
  • Pharmacy Professional Practice
  • Regulatory Requirements
  • Quality Assurance
  • Customer Service
  • Personnel Management
  • Inventory Management
  • Financial Profitability
  • Loss Prevention
  • Workflow Management
What we offer
What we offer
  • Affordable medical plan options
  • a 401(k) plan (including matching company contributions)
  • an employee stock purchase plan
  • No-cost programs for all colleagues including wellness screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching
  • Benefit solutions that address the different needs and preferences of our colleagues including paid time off, flexible work schedules, family leave, dependent care resources, colleague assistance programs, tuition assistance, retiree medical access and many other benefits depending on eligibility
  • Fulltime
Read More
Arrow Right
New

Account Representative I

The Account Representative I is primarily responsible for building or expanding ...
Location
Location
United States , Springfield
Salary
Salary:
Not provided
allianceautomotive.co.uk Logo
Alliance Automotive UK LV Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Typically requires a bachelor's degree and zero (0) to two (2) years of experience or an equivalent combination
  • Knowledge of industrial products: power transmission, hydraulic, pneumatic, and industrial products
  • Excellent written and verbal communication skills
  • A proficient understanding of key sales principles and best practices
  • Ability to influence customers, while maintaining healthy relationships
  • Ability to take initiative and work with limited direction
  • Ability to multitask and manage time well
  • Ability to use Microsoft Office
  • Valid driver's license required
Job Responsibility
Job Responsibility
  • Increases revenue and profitability through generation of new business and further development of existing accounts
  • Generates new leads by networking, cold calling, researching various directories and internet sites
  • Routinely interacts face-to-face with customers to foster strong relationships and maintains satisfaction
  • Understands complex requirements from customers for preparation of customized quoting
  • Delivers effective sales presentations and ensures technical requirements are met
  • Assists with on-site troubleshooting of customers' concerns
  • Works with and coordinates vendor resources to build relationships and support sales
  • Performs other duties as assigned
What we offer
What we offer
  • options for healthcare coverage, 401(k), tuition, reimbursement, vacation, sick, and holiday pay
  • Fulltime
Read More
Arrow Right
New

Service Colleague

Our operation runs between 6am - 11pm, 7 days a week across our Supermarkets, Su...
Location
Location
United Kingdom
Salary
Salary:
12.60 GBP / Hour
asda.com Logo
Asda Express
Expiration Date
February 27, 2026
Flip Icon
Requirements
Requirements
  • You’re naturally friendly, enthusiastic, and ready to get stuck in
  • You’ll enjoy learning new skills across different departments
  • love helping customers find what they need
Job Responsibility
Job Responsibility
  • work across multiple departments to help keep our stores running smoothly
  • serving customers at the checkouts
  • preparing fresh pizzas
  • picking orders for home delivery
  • restocking shelves
  • offering excellent service, welcoming stores, and ‘extra special’ products
  • work together as one team to drive sales and deliver a great online and in-store shopping experience
What we offer
What we offer
  • Colleague discount: 15% off your shopping at Asda from day 1 for you and a nominated user
  • Discounted rates and special offers on Asda services such as Mobile, Pharmacy, Opticians, Personal Loan, Pet Insurance, Travel Insurance, Travel Money, Tyres, Breakdown Cover
  • Stream: access to flexible pay, income tracker, financial coaching, exclusive savings account and much more
  • Company pension
  • Wellbeing: including 24/7 virtual GP, 24/7 EAP service, as well as access to free counselling, legal, mortgage, cancer and bereavement support
  • Asda Allies Inclusion Networks – helping colleagues to make sure everybody is included and that our differences are recognised and celebrated
  • Excellent parental leave policies, including maternity & adoption leave, paternity leave, shared parental leave, neonatal care leave, and support for those doing fertility treatments
  • Colleague recognition programme
  • Special offers and discounts across a range of services and activities, from airport parking to theme parks and cinemas
  • Parttime
Read More
Arrow Right
New

Machinist I

Under direct supervision, sets up, operates, and adjusts various types of conven...
Location
Location
United States , Houston
Salary
Salary:
Not provided
allianceautomotive.co.uk Logo
Alliance Automotive UK LV Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or GED
  • Knowledge of various metals and measurements used in machining processes
  • Knowledge of machinist, CAD and/or CAM software
  • Knowledge of mathematics and its application
  • Knowledge of raw materials, production processes, quality control, costs, and other techniques for maximizing the effective manufacture and distribution of goods
  • Knowledge of monitoring gauges, dials, or other indicators to make sure a machine is working properly
  • Ability to safely use or learn to use piranha, metal shear, and metal breaks
  • Must be eligible to participate in the DoD Skillbridge program
  • Requires manual dexterity, arm-hand steadiness, and multi-limb coordination
  • Ability to lift 50 pounds or more and stand for extended periods of time
Job Responsibility
Job Responsibility
  • Sets up, operates, and adjusts various types of conventional or automatic machines including lathes, drill presses, grinders, and other precision equipment
  • Works from drawings and written instructions to create replacement and new parts
  • Machines parts to specifications, using machine tools such as lathes, milling machines, shapers, or grinders
  • Measures, examines or tests completed units to check for defects and ensure conformance to specifications
  • Calculates dimensions or tolerances
  • Monitors the feed and speed of machines
  • Maintains machine tools in proper operational condition
  • Studies sample parts, blueprints, drawings, or engineering information to determine methods or sequences of operations
  • Uses various bandsaws, cutoff wheels, table saws, and pedestal grinders
  • Lays out, measures and marks metal stock
What we offer
What we offer
  • Healthcare coverage
  • 401(k)
  • Tuition reimbursement
  • Vacation
  • Sick pay
  • Holiday pay
  • Fulltime
Read More
Arrow Right
New

Delivery Driver

As a Delivery Driver for Omnicare Pharmacy, you will deliver prescription orders...
Location
Location
United States , Santa Rosa
Salary
Salary:
16.00 - 31.46 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
March 27, 2026
Flip Icon
Requirements
Requirements
  • High School Diploma or GED is required
  • Possess geographical knowledge of service area
  • Flexibility required Nights and Weekends
  • Current, valid driver’s license in the state at the level required for the vehicle’s operation
  • Ability to drive for long periods of time
Job Responsibility
Job Responsibility
  • Provide timely and accurate delivery of medication, equipment, products and/or supplies into long term care facilities or patient’s homes
  • Load products/materials
  • Responsible for vehicle maintenance and alerting management for maintenance issues
  • Maintain and full compliance with all Federal, State and Local laws and regulations
  • Maintain highest professional and ethical standards
  • Following all applicable government regulations including HIPAA to ensure patient confidentiality
What we offer
What we offer
  • Competitive pay
  • Benefits starting 1st day of month following your start
  • Paid training
  • Fast track career progression with tuition reimbursement and utilizing our university partners, potentially a free higher education
  • Employee discount programs
  • Affordable medical plan options, a 401(k) plan (including matching company contributions), and an employee stock purchase plan
  • No-cost programs for all colleagues including wellness screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching
  • Benefit solutions that address the different needs and preferences of our colleagues including paid time off, flexible work schedules, family leave, dependent care resources, colleague assistance programs, tuition assistance, retiree medical access and many other benefits depending on eligibility
  • Parttime
Read More
Arrow Right