CrawlJobs Logo

Backend Software Engineer - Search, Crawler Team

perplexity.ai Logo

Perplexity

Location Icon

Location:

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

We are seeking an experienced Backend Software Engineer to join our Crawler team. In this role, you will design, develop, and operate systems that ingest, process, and manage web-scale data in support of our next generation of advanced search technologies. This is a critical, high-impact engineering position, requiring expertise across both backend and frontend components of our data acquisition stack.

Job Responsibility:

  • Take ownership of and lead projects focused on developing large-scale web crawlers, ingestion pipelines, and data processing systems
  • Build, maintain, and optimize core backend and frontend components for crawler services, including storage, retrieval, and UI dashboards for data management
  • Collaborate closely with Search and Infrastructure teams to ensure the reliable, high-quality ingestion and processing of billions of web pages
  • Architect and implement fullstack features and scalable distributed systems that handle high-load and real-time data operations
  • Rapidly iterate, experiment with novel approaches, and continuously enhance system performance, usability, and reliability

Requirements:

  • Minimum of 5 years of software development experience
  • Strong knowledge of data structures and algorithms in at least one of the following languages: Python, C++, Rust, or Go
  • Experience with large-scale web crawlers is highly desirable
  • Proven experience building, deploying, and optimizing high-load, distributed, and hardware-adjacent services
  • Deep understanding of cloud infrastructure, with hands-on experience in Kubernetes (K8s) and AWS
  • Demonstrated passion for writing clean, efficient, and scalable systems

Additional Information:

Job Posted:
February 21, 2026

Employment Type:
Fulltime
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Backend Software Engineer - Search, Crawler Team

New

Senior Software Engineer, Data Acquisition

The Data Acquisition team within the Foundations organization at OpenAI is respo...
Location
Location
United States , San Francisco
Salary
Salary:
293000.00 - 385000.00 USD / Year
openai.com Logo
OpenAI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS/MS/PhD in Computer Science or a related field
  • 6+ years of industry experience in software development
  • Experience with large web crawlers a plus
  • Strong expertise in large stateful distributed systems and data processing
  • Proficiency in Kubernetes, and Infrastructure-as-Code concepts
  • Willingness and enthusiasm for trying new approaches and technologies
  • Ability to handle multiple tasks and adapt to changing priorities
  • Strong communication skills, both written and verbal
Job Responsibility
Job Responsibility
  • Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search
  • Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability
  • Work closely with the legal team to handle any compliance or data privacy-related matters
  • Develop and deploy highly scalable distributed systems capable of handling petabytes of data
  • Architect and implement algorithms for data indexing and search capabilities
  • Build and maintain backend services for data storage, including work with key-value databases and synchronization
  • Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks
  • Conduct and analyze experiments on data to provide insights into system performance
What we offer
What we offer
  • Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
  • Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
  • 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend to fuel your professional growth
  • Daily meals in our offices, and meal delivery credits as eligible
  • Fulltime
Read More
Arrow Right
New

Box Truck Driver

Immediately hiring! No experience necessary! Unhappy with your current driving g...
Location
Location
United States of America , Frederick
Salary
Salary:
15.00 USD / Hour
avisbudgetgroup.com Logo
avis budget group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Valid Driver’s License (Class A, B, or C CDL)
  • Must be able to obtain DOT medical clearance
  • Must be 21 years of age and legally authorized to work in the United States
  • On-Site Requirement: This position requires regular, on-site presence and cannot be performed remotely
  • Ability to lift up to 50 pounds, continuously sit, stand, bend, reach, walk, kneel, get up and get down from a seated position, enter, exit, climb into, climb down from, and drive a variety of automobiles, cargo vans and box trucks in all weather conditions
Job Responsibility
Job Responsibility
  • Safely transporting box trucks and moving vans between rental locations, maintenance facilities, and other designated areas as directed by management
  • Moving vehicles from customer check-in to car wash
  • Positioning vehicles for rental
  • Assisting with maintenance needs
  • Following all company procedures
  • Ensuring the safe operation of vehicles at all times
  • Driving a Budget Truck on public access roads
What we offer
What we offer
  • Bi-weekly hourly wage (New York and Puerto Rico: weekly wage)
  • On-the-job training
  • Paid time off
  • Medical, Dental and Other Insurance
  • Flexible spending account opportunity to contribute up to $270 as a tax-free benefit for public transportation or parking expenses
  • Retirement benefits (401k)
  • Employee discounts, including discounted car rental and discounted prices on the purchase of Avis/Budget cars
  • Fulltime
Read More
Arrow Right
New

Store Delivery Driver

This is a Delivery Driver opportunity that truly delivers on being a NAPA brand ...
Location
Location
United States , Tampa
Salary
Salary:
Not provided
allianceautomotive.co.uk Logo
Alliance Automotive UK LV Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Agility to bend to floor-level shelves and reach to upper shelves (eight feet) with use of stool or ladder when necessary
  • Stamina to stand and walk for entire work shift
  • Lift up to 60 lbs of merchandise
  • Able to handle cash charge transactions correctly and core/part returns appropriately
  • Maintaining a distribution log or tracking system to record all deliveries/pickups made
  • Inspecting, protecting and maintaining company assets, merchandise, vehicles, building and people
  • Clear speaking and attentive listening skills
  • Driving throughout the metropolitan area using maps and directions
  • Able to be flexible with your schedule including evenings, weekends and holidays
  • Valid Driver’s License
Job Responsibility
Job Responsibility
  • Delivering parts to our Customers with a passion for developing relationships with our customers
  • Picks up parts from vendors, ensure stock room parts are accurately stocked and maintain/check inventory
  • Consistently focused on safety while driving and delivering our parts
  • Serving as a NAPA Brand Ambassador as you meet customers during your deliveries
  • Building long-term relationships with the customers you deliver to
  • Maintain store delivery truck through adherence maintained to safety checklists, ensuring vehicle is clean, and that basic maintenance is done (e.g., correct tire pressure)
  • Other duties as needed
What we offer
What we offer
  • Awesome people and brand
  • Outstanding health benefits and 401K
  • Stable company. Fortune 200 with a “family” feel
  • Family Culture where no 2 days or career paths are the same
  • Opportunity for accessing multiple career paths, ongoing development, with support from leaders and your team
  • Fulltime
Read More
Arrow Right
New

District Support Pharmacist

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , Graham
Salary
Salary:
60.00 - 76.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
March 08, 2026
Flip Icon
Requirements
Requirements
  • Active Pharmacist License in the state where the Store is located
  • Active National Provider Identifier (NPI)
  • Not on the DEA Excluded Parties list
  • Ability to travel within a reasonable radius to support market staffing as business needs require
  • Regular and predictable attendance, including nights and weekends
  • Ability to complete required training within designated timeframe
  • Attention and Focus: Ability to concentrate on a task over a period of time
  • Ability to pivot quickly from one task to another to meet patient and business needs
  • Ability to confirm prescription information and label accuracy, ensuring patient safety
  • Customer Service and Team Orientation: Actively look for ways to help people, and do so in a friendly manner
Job Responsibility
Job Responsibility
  • Traveling the district to fill pharmacist shifts as scheduled by the District Performance Coordinator (DPC)
  • overseeing the pharmacy and serving as the Pharmacy Manager’s proxy during bench shifts without overlap
  • Supporting safe and accurate prescription fulfillment by following—and directing the pharmacy team to follow—pharmacy workflow procedures and utilizing the safety guardrails at every workstation
  • Assumes Pharmacy Manager’s day-to-day duties when serving as the only or the primary pharmacist-on-duty
  • Contributing to positive patient experiences by showing empathy and genuine care and coaching the pharmacy team to do the same: demonstrating compassionate care, collaborating with the patient’s total healthcare team, and proactively resolving insurance and/or medication issues
  • Proactively offering and delivering immunizations to keep patients healthy
  • engaging and supporting Pharmacy Technicians to learn to immunize
  • Supporting the effective management of pharmacy inventory in all pharmacies worked by following—and guiding the pharmacy team to follow—all inventory best practices, with a special focus on protecting cold chain products for our patients and our business
  • Remaining flexible for both scheduling and business needs, while contributing to a safe, inclusive, and engaging team dynamic
  • Maintaining relevant clinical and technical skills for the job as the industry evolves (including but not limited to company-required trainings and CMEs)
What we offer
What we offer
  • Affordable medical plan options
  • a 401(k) plan (including matching company contributions)
  • an employee stock purchase plan
  • No-cost programs for all colleagues including wellness screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching
  • Benefit solutions that address the different needs and preferences of our colleagues including paid time off, flexible work schedules, family leave, dependent care resources, colleague assistance programs, tuition assistance, retiree medical access and many other benefits depending on eligibility
  • Fulltime
Read More
Arrow Right
New

B1.3 Licensed Engineer

Join the World Leader in Helicopter Solutions. Airbus Helicopters is the world’s...
Location
Location
Cyprus , Akrotiri
Salary
Salary:
Not provided
airbus.com Logo
Airbus
Expiration Date
March 02, 2026
Flip Icon
Requirements
Requirements
  • Experienced aircraft engineer with extensive rotary-wing experience
  • UK CAA Licensed Engineer
  • Military training or civilian aircraft maintenance qualification/apprenticeship
  • Strong understanding of Part-145 regulation
  • Methodical and confident fault-finding abilities
  • High commitment to safety and regulatory compliance
  • Ability to work closely with customers and develop strong working relationships
  • Flexible, pragmatic, and reliable in diverse working environments
  • Eligibility for DBS and Security Clearance (SC)
  • Compliance with all ITAR and Export Control regulations
Job Responsibility
Job Responsibility
  • Prepare aircraft for daily flying operations
  • Perform line and base maintenance activities
  • Carry out defect rectification and fault diagnosis
  • Undertake production duties as directed by the Chief Engineer
  • Support operational demands, including on-call periods and flexible shift patterns (early and late starts/finishes)
  • Fulltime
!
Read More
Arrow Right
New

Customer Service Representative

This is an outdoor combination role, where you will assist customers in differen...
Location
Location
United States of America , Vandalia
Salary
Salary:
14.50 USD / Hour
avisbudgetgroup.com Logo
avis budget group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Valid Driver’s License
  • Basic computer skills (typing, data entry)
  • Effective verbal communication skills
  • Willingness to work outdoors
  • Flexibility to work all shifts
  • Must be able to type, sit, stand, walk or move throughout rental lot and enter/exit vehicles for prolonged periods and drive a variety of vehicles
  • Must be 18 years of age and legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Assist customers in different parts of the rental or return process on our airport lots
  • This may include, depending on location, checking cars, entering information in a handheld device, processing rentals of our loyalty customers, assisting customers at our exit gates or checking-in our cars at return, while providing excellent customer service
What we offer
What we offer
  • Bi-weekly hourly wage (New York and Puerto Rico: weekly wage)
  • On the job training
  • Paid time off
  • Medical, dental and other insurance
  • Flexible spending account opportunity to contribute up to $270 as a tax-free benefit for public transportation or parking expenses
  • Retirement benefits (401k)
  • Employee discounts, including discounted car rental and discounted prices on the purchase of Avis/Budget cars
  • Parttime
Read More
Arrow Right
New

Entry Level Automotive Technician

Immediately hiring! We’re looking for responsible, detailed people who are as pa...
Location
Location
United States of America , Newark
Salary
Salary:
22.00 USD / Hour
avisbudgetgroup.com Logo
avis budget group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Valid Driver’s License
  • At least 6 months auto maintenance experience (ability to acquire 1 ASE certificate within first year)
  • Comfortable working in a mechanical shop with moderate or loud noise levels
  • Basic computer skills including typing, data entry
  • Must be able to lift up to 50 pounds, continuously stand, bend, reach, walk, sit and kneel frequently, enter, exit and drive vehicles
  • Must be 18 years of age and legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Perform preventative maintenance functions and light repairs on auto fleet of vehicles
  • oil, fluid, tire changes, repairs, rotations, and other basic light repairs, all in a safe manner
  • Depending on experience, may be assigned to assist experienced auto mechanics
What we offer
What we offer
  • Bi-weekly pay
  • On-the-job training to expand automotive or mechanical abilities, help to obtain ASE Certificates
  • Paid time off
  • Medical, dental, and other insurance
  • Flexible spending account opportunity to contribute up to $270 as a tax-free benefit for public transportation or parking expenses
  • Retirement benefits (401k)
  • Employee discounts, including discounted car rental and discounted prices on the purchase of Avis/Budget cars
  • Fulltime
Read More
Arrow Right
New

Medical Scribe

The purpose of a Medical Scribe (CIS or Medical Scribe) at Oak Street Health is ...
Location
Location
United States , Center Point
Salary
Salary:
17.00 - 28.46 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
February 28, 2026
Flip Icon
Requirements
Requirements
  • Knowledge of medical terminology and common medications, either from a pre-medical degree or prior clinical experience
  • Advanced listening and communication skills
  • Strong computer literacy and ability to learn new technical workflows
  • Fluency in Spanish, Polish, Russian, or other languages spoken by the communities we serve [required where indicated]
  • Ability to adapt to new workflows and to quickly learn new concepts and skills
  • Ability and willingness to take direction and be a member of a team providing patient care, including adapting to the provider's working style
  • Excellent job attendance including ability to work in-person in our clinics
  • Ability to commit to at least 1 year in role (2+ is ideal)
  • Ability to work approximately 40-45 hours per week during clinic hours (full time position) with predictable hours and break times
  • Compliance with hospital and Oak Street Health policies, including HIPAA
Job Responsibility
Job Responsibility
  • Documenting Patient Encounters ~ 80%: Joining the provider in the exam room to observe patient visits
  • Documenting patient encounters in a structured note, including the history of the present illness, assessment, plan, and physical exam
  • Assigning appropriate CPT and ICD-10 codes
  • Preparing After Visit Summaries
  • Consulting with provider to ensure accurate and specific documentation
  • Clinical Documentation Improvement ~ 10%: Requesting and reviewing medical records
  • Leveraging Oak Street's population health tools to support clinical documentation improvement
  • Preparing for and supporting Daily Huddles and Clinical Documentation Reviews
  • Consulting with provider on clinical documentation opportunities
  • Administrative support for your provider and care team ~ 10%: Placing orders and referrals
What we offer
What we offer
  • Affordable medical plan options
  • 401(k) plan (including matching company contributions)
  • Employee stock purchase plan
  • No-cost programs for all colleagues including wellness screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching
  • Paid time off
  • Flexible work schedules
  • Family leave
  • Dependent care resources
  • Colleague assistance programs
  • Tuition assistance
  • Fulltime
!
Read More
Arrow Right