CrawlJobs Logo

Software Engineer – Web Crawling

Woflow

Location Icon

Location:

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

Woflow is a technology startup creating products and solutions to support a high-growth, on-demand economy. Our flagship product is an end-to-end platform that allows our customers to request and receive merchant data (think structured menu data, images, store information, etc) through a combination of web applications and public APIs. Behind the scenes, that data is created through a series of ML/AI models and workflow products along with a fully automated distributed workforce management platform. We are the world’s first Merchant Data Platform. Our customers include food delivery companies, online ordering platforms, and ecommerce marketplaces. We provide the data infrastructure to help these companies scale and grow. We are looking for a Software Engineer with at least 3 years of experience to join our Web Crawling team. This team plays a critical role in our infrastructure, ensuring high-quality data collection at scale. The right candidate will work closely with both the Web Crawling and Application teams but will primarily focus on writing and optimizing web crawlers. Our ideal candidate is someone who enjoys solving complex web scraping challenges, has strong reverse engineering skills, and thrives in a fast-paced, high-growth environment. This role is a great opportunity for an engineer looking to make a significant impact at a company where data is the product.

Job Responsibility:

  • Develop, enhance, and maintain web crawlers and scraping infrastructure
  • Optimize scraping techniques to handle anti-bot mechanisms, performance, and security challenges
  • Collaborate with a geographically distributed team to identify and resolve issues
  • Ensure high availability, efficiency, and reliability of crawling operations
  • Integrate AI solutions to enhance automation and data extraction accuracy

Requirements:

  • 3+ years of experience in software engineering with a focus on web crawling and data extraction
  • Strong expertise in Node.js (preferred) for web crawling applications
  • Deep understanding of HTML, JavaScript, and reverse engineering techniques
  • Hands-on experience with Playwright, Puppeteer, and Cheerio for automation and scraping
  • Knowledge of security and performance best practices related to web crawling

Nice to have:

  • Experience with Apify or Crawlee for large-scale crawling solutions
  • Proficiency in TypeScript
What we offer:
  • Unlimited PTO
  • Comprehensive medical, dental, and vision insurance plans
  • STD, LTD, AD&D, and life insurance coverage
  • Free membership to TalkSpace, Teladoc and Health Advocate
  • Free annual membership to One Medical in participating regions
  • 401(k) retirement plan with company matching
  • Pre-tax commuter benefits
  • Free equipment: laptop and home office stipend

Additional Information:

Job Posted:
February 21, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Software Engineer – Web Crawling

New

Senior Software Engineer, Data Acquisition

The Data Acquisition team within the Foundations organization at OpenAI is respo...
Location
Location
United States , San Francisco
Salary
Salary:
293000.00 - 385000.00 USD / Year
openai.com Logo
OpenAI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS/MS/PhD in Computer Science or a related field
  • 6+ years of industry experience in software development
  • Experience with large web crawlers a plus
  • Strong expertise in large stateful distributed systems and data processing
  • Proficiency in Kubernetes, and Infrastructure-as-Code concepts
  • Willingness and enthusiasm for trying new approaches and technologies
  • Ability to handle multiple tasks and adapt to changing priorities
  • Strong communication skills, both written and verbal
Job Responsibility
Job Responsibility
  • Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search
  • Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability
  • Work closely with the legal team to handle any compliance or data privacy-related matters
  • Develop and deploy highly scalable distributed systems capable of handling petabytes of data
  • Architect and implement algorithms for data indexing and search capabilities
  • Build and maintain backend services for data storage, including work with key-value databases and synchronization
  • Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks
  • Conduct and analyze experiments on data to provide insights into system performance
What we offer
What we offer
  • Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
  • Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
  • 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend to fuel your professional growth
  • Daily meals in our offices, and meal delivery credits as eligible
  • Fulltime
Read More
Arrow Right
New

Software Engineer, Data Acquisition

The Data Acquisition team within the Foundations organization at OpenAI is respo...
Location
Location
United States , San Francisco
Salary
Salary:
293000.00 - 385000.00 USD / Year
openai.com Logo
OpenAI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS/MS/PhD in Computer Science or a related field
  • 4+ years of industry experience in software development
  • Strong expertise in large stateful distributed systems and data processing
  • Proficiency in Kubernetes, and Infrastructure-as-Code concepts
  • Willingness and enthusiasm for trying new approaches and technologies
  • Ability to handle multiple tasks and adapt to changing priorities
  • Strong communication skills, both written and verbal
Job Responsibility
Job Responsibility
  • Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search
  • Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability
  • Work closely with the legal team to handle any compliance or data privacy-related matters
  • Develop and deploy highly scalable distributed systems capable of handling petabytes of data
  • Architect and implement algorithms for data indexing and search capabilities
  • Build and maintain backend services for data storage, including work with key-value databases and synchronization
  • Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks
  • Conduct and analyze experiments on data to provide insights into system performance
What we offer
What we offer
  • Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
  • Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
  • 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend to fuel your professional growth
  • Daily meals in our offices, and meal delivery credits as eligible
  • Fulltime
Read More
Arrow Right

SEO Specialist

As an SEO Specialist, you will be responsible for maintaining and executing the ...
Location
Location
Philippines , Manila
Salary
Salary:
Not provided
staffvirtual.com Logo
STAFFVIRTUAL
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience in SEO with a focus on strategy and execution (2 to 4 years experience)
  • Strong understanding of search engine algorithms and ranking factors
  • Familiarity with SEO tools such as Ahrefs, Moz, SEMrush, etc.
  • Excellent analytical and problem-solving skills
  • Effective communication and collaboration skills
  • Strong written communication skills with attention to grammar and clarity
  • Detail-oriented with the ability to manage multiple tasks and deadlines
Job Responsibility
Job Responsibility
  • Local Search Optimization
  • Optimize and manage Google Business Profiles (GBP) for multiple locations
  • Improve local map pack rankings through strategic updates, category optimization, and posting
  • Keyword Research and Analysis
  • Conduct thorough keyword research to identify relevant search terms
  • Analyze keyword competitiveness and trends to inform content strategies
  • Perform audits and make strategic recommendations based on experience with the client
  • On-Page & Technical Local SEO
  • Collaborate with Content, Design, and Web Dev departments to optimize on-page elements such as meta tags, headers, content, and user experience
  • Implement structured data for local businesses, services, and FAQs
What we offer
What we offer
  • Competitive compensation and benefits package
  • HMO coverage starting on Day 1
  • Paid time offs
  • Training, career growth, and global exposure
  • A collaborative and supportive team culture
Read More
Arrow Right
New

Finance Controller - Operations

Job Description: SECURITY CLEARANCE: Eligibility to gain UK SC Security Clearanc...
Location
Location
United Kingdom , Belfast
Salary
Salary:
Not provided
airbus.com Logo
Airbus
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Eligibility to gain UK SC Security Clearance OR subject to BPSS and Export Control checks (including a criminal record check)
  • Educated to at least degree / masters level
  • CIMA qualification preferred
  • Experience in a relevant financial controlling / management accounting role
Job Responsibility
Job Responsibility
  • Take command of a $500m+ recurring cost base as a lead financial architect for the A220 Wing program in Belfast
  • Act as a strategic Business Partner to the Head of CDT
  • Drive performance
  • Navigate 5-year forecasts
  • Challenge business cases
  • Steer risk mitigation
  • Be the bridge between operational excellence and financial success
  • Overall business partner to the Head of CDT and integrated member of the A220 management team
  • Lead and coordinate A220 Wing controlling through streamlining of controlling processes such as budgeting, forecasting and reporting between sub-assembly areas
  • Evaluate project appraisals and business cases to assist decision-making
What we offer
What we offer
  • Competitive salary
  • Annual profit share
  • Contributory pension
  • Share options
  • 36 hour week
  • Flexible working around core hours and Friday afternoons off
  • Flexible working
  • Customised development solutions
  • Interview accommodation for applicants with disabilities
Read More
Arrow Right
New

Demand Process Owner CHC

Want a 3D Career? Join Norgine. At Norgine, our colleagues Dare themselves to be...
Location
Location
United Kingdom , Uxbridge
Salary
Salary:
Not provided
norgine.com Logo
Norgine
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Demand Planning experience at a similar level
  • Broader exposure to supply chain and manufacturing functions would be of benefit
  • Background in Pharma or FMCG working in the environment of multi-national SMEs or larger corporate bodies
  • Experience of working within a structured IBP/S&OP business process
  • Statistical forecasting experience
  • Strong analytical skills, able to be both precise and concise when handling large amounts of complex data
  • Excels at communication, negotiation and engagement, able to understand and articulate stakeholder requirements and acts with integrity
  • Progressive attitude towards change and a passion for continuous improvement
  • Able to thrive in a challenging environment with fast changing priorities
  • Extensive exposure to ERP systems, proficient in MS Excel (Power BI is a plus)
Job Responsibility
Job Responsibility
  • Running the Demand Planning cycle across multiple brands and ensuring that the consensus forecast is broadcast to Distribution Resource Planning in a timely manner
  • Proactively support Commercial Sales and Marketing functions in delivering a robust 24 month rolling forecast by gathering, analysing and reporting data, chairing monthly Demand Review Meetings, and being the Subject Matter Expert within Supply Chain on market trends and brand activations
  • Proactively support the S&OP team in understanding volatile movements in sales and forecasting in order to ensure supply risks are captured at the earliest possible stage, and ensuring that the flow of communication on any such risks goes back to Commercial functions
  • Manage the gates within S&OP for the implementation of NPDI, ensuring all key activities are completed on time to meet launch dates
  • Support the budget and quarterly update process through the provision of forecast and where relevant market intelligence
  • Have a continuous improvement mindset to review, develop and enhance planning processes in order to improve efficiency and align on best practice systemic ways of working, with a focus on root cause analysis and clear and consistent communication
  • Represent Norgine at meetings with external third party suppliers and customers
  • Build, develop and maintain relationships with stakeholders and peers across various business units including Commercial Sales and Marketing, Finance, and other Supply Chain functions
  • Other tasks as required by the business on an ad-hoc or ongoing basis
  • Fulltime
Read More
Arrow Right
New

Material Handler I

Under close supervision, the Material Handler performs physical and administrati...
Location
Location
United States , Fort Mill; Rock Hill; Charlotte; Pineville
Salary
Salary:
21.09 USD / Hour
allianceautomotive.co.uk Logo
Alliance Automotive UK LV Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or GED
  • Zero (0) to two (2) years of related experience or an equivalent combination
  • Ability to work as a team in a fast-paced environment
  • Ability to comprehend and carry-out instructions provided in oral, written, and diagram format
  • Basic computer skills, including information processing required
  • Ability to perform basic mathematical functions in addition, subtraction, multiplication, and division
  • Ability to regularly stand, walk, talk, hear, and lift
  • Ability to occasionally sit, stoop, kneel, crouch, climb, or crawl
  • Ability to regularly lift and/or move up to 25 pounds
  • Ability to frequently lift and/or move up to 50 pounds
Job Responsibility
Job Responsibility
  • Unloads and directs movement of shipments from shipping and receiving platform to storage and work areas
  • Verifies and records incoming and outgoing shipments within the Warehouse Management System to ensure shipments include correct product, quality, and quantity
  • Gathers, verifies, and places items into order picking container for outgoing shipments
  • Prepares packing slips, shipping labels, bills of lading, and other required documents
  • Ensures warehouse work areas and equipment are clean, orderly, and properly maintained
  • Replenishes primary stock shelves from reserve stock locations as needed
  • Operates a forklift
  • Works after hours as needed
  • May perform driving duties
  • Performs other duties as assigned
What we offer
What we offer
  • Healthcare coverage options
  • 401(k)
  • Tuition reimbursement
  • Vacation pay
  • Sick pay
  • Holiday pay
  • Fulltime
Read More
Arrow Right
New

Shift Supervisor

We’re building a world of health around every individual — shaping a more connec...
Location
Location
United States , Kihei
Salary
Salary:
18.00 - 27.00 USD / Hour
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
April 09, 2026
Flip Icon
Requirements
Requirements
  • Deductive reasoning ability, analytical skills and computer skills
  • Advanced communication skills and supervision skills
  • Ability to work a flexible schedule, including some early morning, overnight and weekend shifts, to work overtime as needed, and to respond to urgent issues at the store when they arise
Job Responsibility
Job Responsibility
  • Work effectively with store management and store crews
  • Supervise the store’s crew through assigning, directing and following up of all activities
  • Effectively communicate information both to and from store management and crews
  • Assist customers with their questions, problems and complaints
  • Promote CVS customer service culture
  • Handle all customer relations issues in accordance with company policy and promote a positive shopping experience
  • Maintain customer/patient confidentiality
  • Price merchandise
  • Stock shelves
  • Execute the displays, sign and inventory of weekly, promotional, and seasonal merchandise
What we offer
What we offer
  • Affordable medical plan options
  • 401(k) plan (including matching company contributions)
  • Employee stock purchase plan
  • No-cost programs for all colleagues including wellness screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching
  • Paid time off
  • Flexible work schedules
  • Family leave
  • Dependent care resources
  • Colleague assistance programs
  • Tuition assistance
  • Fulltime
Read More
Arrow Right
New

Legal Assistant

Are you ever wondering "What else is out there?" Are you currently looking for a...
Location
Location
Canada , Winnipeg
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
April 06, 2026
Flip Icon
Requirements
Requirements
  • Minimum of 3 years of working experience
  • Successful completion of a legal assistant program
  • Excellent organization and time management skills
  • Outstanding communication and interpersonal skills
  • Great MS Office Skills
  • Strong understanding of legal concepts and terminology
  • Ability to work independently but also as part of a team
Job Responsibility
Job Responsibility
  • Opening and closing files
  • Interacting with clients and other lawyers
  • Transcribing legal documents
  • Filing legal documents
  • Conflict, title and due diligence searches
  • Other administrative duties as assigned
What we offer
What we offer
  • Competitive compensation
  • Opportunity to work with some of Winnipeg's top law firms
  • Great company culture
  • Monday - Friday daytime hours
  • Great benefit packages are available
  • Potential growth opportunity
  • Fulltime
Read More
Arrow Right