> Parel documentation. Index of every page: https://parel.si/docs/llms.txt. This page in HTML: https://parel.si/docs/tools/crawl_site

# crawl_site

> Up to twenty pages of one website in one call, each as clean text with its address. ₹0.56 per page; a result that finds nothing is ₹0.

## What it does

Reads a website from a start address and follows its links, returning up to limit pages (1 to 20, default 10) in one call: data.results[] with each page's url and raw_content as clean markdown, and data.base_url. Steer it with max_depth (how many clicks from the start, default 1), select_paths and exclude_paths (patterns such as "/blog/.*") and select_domains; links to other sites are not followed.

Use it to read a small site or one section of a big one at once; to list a site's addresses without reading them use [`map_site`](https://parel.si/docs/tools/map_site.md), for one page [`read_page`](https://parel.si/docs/tools/read_page.md), for a few known addresses [`read_pages`](https://parel.si/docs/tools/read_pages.md). Priced per page returned, so limit caps the cost; a site that answers no pages is free. Slow: five to thirty seconds.

India: works on Indian sites; pages come back in their own language.

## Inputs

Generated from the contract captured from staging on 2026-11-17.

| Field | Type | Required | Description |
|---|---|---|---|
| `url` | string | yes | The address to start from ("https://example.com/blog"). |
| `limit` | integer | no | How many pages to read, 1 to 20. Default 10. Each page returned is charged.; min 1, max 10 |
| `max_depth` | integer | no | How many clicks away from the start page to go, 1 to 5. Default 1.; min 1, max 5 |
| `max_breadth` | integer | no | How many links to follow from each page, up to 500. Default 20.; min 1, max 500 |
| `select_paths` | array of string | no | Only follow addresses matching these patterns (["/blog/.*"]). |
| `exclude_paths` | array of string | no | Skip addresses matching these patterns (["/tag/.*"]). |
| `select_domains` | array of string | no | Only follow these hosts, as patterns (["^docs\\.example\\.com$"]). |

## What drives the cost

Priced per page returned, up to 20 a call; set limit to what you need. A result that finds nothing is `class: "ok_free"`, ₹0. The timeout is 74 seconds.

## Price today

| | |
|---|---|
| Price today | **₹0.56** per page (`cost_paise: 56`) at ₹93.93 per US dollar on 17 November 2026; each one asked for is charged, ₹5.42 for the largest call |
| Hold before the call | ₹0.84 (`hold_paise: 84`) for one, ₹8.13 for the largest call; when the call settles the real price is charged and the unused part is released |
| ₹0 when | it fails, or the provider answers but there is nothing in it (an empty list, no email, no page): `class: "ok_free"`, ₹0 |
| Timeout | 74 s, then `upstream_timeout` (₹0, retryable) |
| Over MCP | `run_tool` with `tool: "crawl_site"` |
| Over REST | `POST https://api.parel.si/v1/run` with `"tool": "crawl_site"` |
| Confirmation | not needed (the estimate is under the ₹200 threshold) |
| Count field | `limit`: The hold covers the number of results you ask for in limit; leave it out and the worst case is held. |

## Request

```bash
curl https://api.parel.si/v1/run \
  -H "Authorization: Bearer $PAREL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tool":"crawl_site","input":{"url":"https://nodejs.org/en/about","limit":5,"max_depth":1}}'
```

## What comes back

`data`, as the live check of 2026-11-17 recorded it, trimmed:

```json
{
  "base_url": "https://nodejs.org/en/about",
  "results": [
    {
      "url": "https://nodejs.org/en/about",
      "raw_content": "[Skip to content](https://nodejs.org/en/about#main)\n\n[](https://nodejs.org/en)\n\n- [x] \n\n[Learn](https://nodejs.org/learn)[About](https://nodejs.org/en/about)[Download](https://nodejs.org/en/download)[Blog](https://nodejs.org/en/blog)[Docs](…"
    },
    {
      "url": "https://nodejs.org/en",
      "raw_content": "[Skip to content](https://nodejs.org/en#main)\n\n[](https://nodejs.org/en)\n\n- [x] \n\n[Learn](https://nodejs.org/learn)[About](https://nodejs.org/en/about)[Download](https://nodejs.org/en/download)[Blog](https://nodejs.org/en/blog)[Docs](https:…"
    },
    {
      "url": "https://nodejs.org/en/blog",
      "raw_content": "[Skip to content](#main)\n\n# Blog\n\nThe latest Node.js news, migration guides and events summaries\n\n[Everything](https://nodejs.org/en/blog/all)[Announcements](https://nodejs.org/en/blog/announcements)[Releases](https://nodejs.org/en/blog/release)[Vulnerabilities](https://nodejs.org/en/blog/vulnerability)[Migrati…"
    }
  ]
}
```

## Notes

- Priced per page returned, up to 20 a call; set limit to what you need.
- Five to thirty seconds. Pages that need a login or block automated readers come back missing, not as errors.
- Use select_paths to stay inside one section ("/blog/.*") instead of crawling the whole site.

## Siblings

- [`map_site`](https://parel.si/docs/tools/map_site.md)
- [`read_page`](https://parel.si/docs/tools/read_page.md)
- [`read_pages`](https://parel.si/docs/tools/read_pages.md)
