Products

Crawlee

Build reliable web scrapers. Fast.

crawlee.dev Source★ 3,714Apache-2.0 DiscordData & Analytics / Web Scraping & Extraction

AI Benchmark
Desktop
Mobile
Monthly visits
38.3K
Down28.6%MoMOct 2026
Public revenue
Undisclosed
No public revenue evidence yet
Entry price
Free
No monthly subscription pricing
Creators
Marek Trunkát
No social account linkedCo-created with Martin Adámek, Szymon Marczak
Launched
4y ago
2022-08-23 · Est.

Traffic

Monthly visitsData through Oct 2026
Events3 changes

Marketing moves

Launches, search & content, referrals, promotions
  • Launched on Product HuntLaunch

    Crawlee for Python is now out of beta with new features!

    Sep 30, 2025 · 14mo agoLaunch page

  • Launched on Show HN (254 pts · 57 comments)Launch

    Crawlee for Python – a web scraping and browser automation library — Hey all, This is Jan, the founder of Apify ( ) — a full-stack web scraping platform. After the success of Crawlee for JavaScript ( ) and t…

    Jul 9, 2024 · 2y agoLaunch page

  • Launched on Product HuntLaunch

    Build reliable scrapers in Python

    Jul 9, 2024 · 2y agoLaunch page

  • Launched on Product HuntLaunch

    Build reliable web scrapers and robots, fast!

    Aug 23, 2022 · 4y agoLaunch page

  • Launched on Show HN (282 pts · 80 comments)Launch

    Crawlee – Web scraping and browser automation library for Node.js — Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're laun…

    Aug 23, 2022 · 4y agoLaunch page

Third-party reviews & case dossiers

Acquisition breakdowns, engineer reviews & open-source metrics
Show HN
2022-08-23Source
Post excerpt:
Hey HN, This is Jan, founder of Apify, a web scraping and automation platform. Drawing on our team's years of experience, today we're launching Crawlee [1], the web scraping and browser automation library for Node.js that's designed for the fastest development and maximum reliability in production. For details, see the short video [2] or read the announcemen
Console.dev
2023-03-16Source
What engineers like:
Uses TypeScript and integrates with (abstracts) common headless browsers like Puppeteer & Playwright which makes it easier to keep scripts up to date. Manages queuing, storing results, scaling & proxies. Extensive docs and examples. Open source.
Trade-offs & limitations:
Primarily designed to run locally (or in Docker), but there is a commercial SaaS option for hosting and scaling in the cloud.
Best suited for:
Developers using TypeScript and headless browsers should consider Crawlee for web scraping automation
Show HN
2024-07-09Source
Post excerpt:
Hey all, This is Jan, the founder of Apify ( https://apify.com/ ) — a full-stack web scraping platform. After the success of Crawlee for JavaScript ( https://github.com/apify/crawlee/ ) and the demand from the Python community, we're launching Crawlee for Python today! The main features are: - A unified programming interface for both HTTP (HTTPX with Beautif

Open Source & Self-Hosting

GitHub · MCP / Skill
Last commit 2026-11-06
GitHub Stars
★ 3,714
Forks
310
License
Apache-2.0
Primary language
JavaScript

Traffic sources

Oct 2026

Traffic breakdown

Channels

  • Organic Search37%
  • Referrals30%
  • Direct29%
  • AI Assistants3%

Countries

  • United States33%
  • Brazil14%
  • India14%
  • Vietnam4%
  • Indonesia4%

Search

Domain authority —Domain registered 2022-07-08

KeywordMonthly volume
apify962K
crawlee3,120
sitemap crawler350
cheeriocrawler300
crawlee, scrapy220

Traffic data: Vaneform

Pricing

Website does not list public plan prices.

Trial: None · Observed 2026-11-03 · Source page

Pages & changes

Tracked since 2022-08-23

Key pages

Weekly snapshots

Changes

  1. Nov 1 · 13d ago

    Adopted technology: Amazon CloudFrontFeatures

    —→Amazon CloudFront

    At the time: 38.3K visits/mo2 concurrent moves
  2. Nov 1 · 13d ago

    Removed technology: Product HuntFeatures

    Product Hunt→—

    At the time: 38.3K visits/mo2 concurrent moves
  3. Nov 1 · 13d ago

    Removed technology: Twitter AdsFeatures

    Twitter Ads→—

    At the time: 38.3K visits/mo2 concurrent moves

Design

First-screen visual analysis & :root CSS

Minimal Monochrome (Light canvas, balanced density) featuring centered headline + cta layout and code block / terminal / cli visuals.

First-screen preview · click to zoom

First-screen pitch & conversion flow

“Build reliable web scrapers. Fast.”

Centered Headline + CTA hero structure paired with download app / install extension CTA and no explicit trust bar.

Conversion path: Centered Headline + CTA + Code Block / Terminal / CLI → Download App / Install Extension

Palette · click to copy

5-D design score6.9 / 10
Typography
7.0
Color & Contrast
7.0
Layout & Spacing
7.0
Craft & Polish
7.0
Distinctiveness
6.0

Structure

Fonts
Inter / JetBrains Mono
Primary visual
Code Block / Terminal / CLI
Social proof
No Explicit Trust Bar
Navigation
Full-Width Top Bar
Theme & density
Light · Balanced
Corners & depth
Medium Radius · Hairline borders
Preview full AI design brief (includes :root CSS variables & layout rules)Style tokens only; never copy brand names, logos, copy, or images
# Design Brief: Minimal Monochrome (with Precision Swiss Grid)

Monochrome palette with a single accent color, hairline dividers, and generous whitespace.

## Design Tokens

```css
:root {
  --background: #ffffff;
  --surface: #f5f6f8;
  --foreground: #1b1d24;
  --muted-foreground: #6e7480;
  --accent: #f58220;
  --radius: 8px;
  --font-heading: "Inter", ui-sans-serif, system-ui, sans-serif;
  --font-body: "Inter", ui-sans-serif, system-ui, sans-serif;
  --font-mono: "JetBrains Mono", ui-monospace, monospace;
}
```

Color area ratio: Background #FFFFFF (~55%) · Surface #F5F6F8 (~20%) · Text #1B1D24 (~12%) · Muted Text #6E7480 (~8%) · Accent #F58220 (~5%). Background and surface dominate the canvas; reserve the accent color for primary buttons and active states.

## Layout & Conversion Structure

- Hero layout: Centered Headline + CTA
- Navigation: Full-Width Top Bar
- Theme & density: Light · Balanced
- Border radius & elevation: Medium Radius; no drop shadows—separate layers with hairline borders and whitespace
- Primary visual medium: Code Block / Terminal / CLI
- Conversion CTA: Download App / Install Extension
- Headline hook: Outcome & Speed Promise
- Hero components: Code & Terminal
- Spatial layout notes: Centered Headline + CTA hero structure paired with download app / install extension CTA and no explicit trust bar.
- Visual summary: Minimal Monochrome (Light canvas, balanced density) featuring centered headline + cta layout and code block / terminal / cli visuals.

## Avoid

- Multi-stop neon gradients
- Heavy drop shadows
- More than one competing accent color

Implement the UI following the design tokens and layout specifications above: use only these color variables, and reserve the accent color strictly for primary actions and active states. Adopt only this visual style—never copy any reference product’s brand name, logo, copy, or imagery.

Questions & evidence

How much website traffic does Crawlee get?
Vaneform estimates 38.3K website visits in 2026-10. Visits can include repeat visits; this estimate measures web traffic separately from paying customers or revenue.Source
What is Crawlee and what platforms does it support?
Crawlee is catalogued under Web Scraping & Extraction: Build reliable web scrapers. Fast.. Supported platforms: API.
Is Crawlee open source?
Crawlee has a public repository (https://github.com/apify/crawlee) under the Apache-2.0 license, recording 3,714 GitHub stars.Source

Related products in this category

Data & Analytics / Web Scraping & Extraction

These products share a catalog category. Compare customer tasks, features and deployment requirements before treating them as alternatives.