crawl_site

Up to twenty pages of one website in one call, each as clean text with its address. ₹0.56 per page; a result that finds nothing is ₹0.

What it does

Reads a website from a start address and follows its links, returning up to limit pages (1 to 20, default 10) in one call: data.results[] with each page's url and raw_content as clean markdown, and data.base_url. Steer it with max_depth (how many clicks from the start, default 1), select_paths and exclude_paths (patterns such as "/blog/.*") and select_domains; links to other sites are not followed.

Use it to read a small site or one section of a big one at once; to list a site's addresses without reading them use map_site, for one page read_page, for a few known addresses read_pages. Priced per page returned, so limit caps the cost; a site that answers no pages is free. Slow: five to thirty seconds.

India: works on Indian sites; pages come back in their own language.

Inputs

Generated from the contract captured from staging on 2026-11-17.

FieldTypeRequiredDescription
urlstringyesThe address to start from ("https://example.com/blog").
limitintegernoHow many pages to read, 1 to 20. Default 10. Each page returned is charged.; min 1, max 10
max_depthintegernoHow many clicks away from the start page to go, 1 to 5. Default 1.; min 1, max 5
max_breadthintegernoHow many links to follow from each page, up to 500. Default 20.; min 1, max 500
select_pathsarray of stringnoOnly follow addresses matching these patterns (["/blog/.*"]).
exclude_pathsarray of stringnoSkip addresses matching these patterns (["/tag/.*"]).
select_domainsarray of stringnoOnly follow these hosts, as patterns (["^docs\\.example\\.com$"]).

What drives the cost

Priced per page returned, up to 20 a call; set limit to what you need. A result that finds nothing is class: "ok_free", ₹0. The timeout is 74 seconds.

Price today

Price today₹0.56 per page (cost_paise: 56) at ₹93.93 per US dollar on 17 November 2026; each one asked for is charged, ₹5.42 for the largest call
Hold before the call₹0.84 (hold_paise: 84) for one, ₹8.13 for the largest call; when the call settles the real price is charged and the unused part is released
₹0 whenit fails, or the provider answers but there is nothing in it (an empty list, no email, no page): class: "ok_free", ₹0
Timeout74 s, then upstream_timeout (₹0, retryable)
Over MCPrun_tool with tool: "crawl_site"
Over RESTPOST https://api.parel.si/v1/run with "tool": "crawl_site"
Confirmationnot needed (the estimate is under the ₹200 threshold)
Count fieldlimit: The hold covers the number of results you ask for in limit; leave it out and the worst case is held.

Request

curl https://api.parel.si/v1/run \
  -H "Authorization: Bearer $PAREL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tool":"crawl_site","input":{"url":"https://nodejs.org/en/about","limit":5,"max_depth":1}}'

What comes back

data, as the live check of 2026-11-17 recorded it, trimmed:

{
  "base_url": "https://nodejs.org/en/about",
  "results": [
    {
      "url": "https://nodejs.org/en/about",
      "raw_content": "[Skip to content](https://nodejs.org/en/about#main)\n\n[](https://nodejs.org/en)\n\n- [x] \n\n[Learn](https://nodejs.org/learn)[About](https://nodejs.org/en/about)[Download](https://nodejs.org/en/download)[Blog](https://nodejs.org/en/blog)[Docs](…"
    },
    {
      "url": "https://nodejs.org/en",
      "raw_content": "[Skip to content](https://nodejs.org/en#main)\n\n[](https://nodejs.org/en)\n\n- [x] \n\n[Learn](https://nodejs.org/learn)[About](https://nodejs.org/en/about)[Download](https://nodejs.org/en/download)[Blog](https://nodejs.org/en/blog)[Docs](https:…"
    },
    {
      "url": "https://nodejs.org/en/blog",
      "raw_content": "[Skip to content](#main)\n\n# Blog\n\nThe latest Node.js news, migration guides and events summaries\n\n[Everything](https://nodejs.org/en/blog/all)[Announcements](https://nodejs.org/en/blog/announcements)[Releases](https://nodejs.org/en/blog/release)[Vulnerabilities](https://nodejs.org/en/blog/vulnerability)[Migrati…"
    }
  ]
}

Notes

  • Priced per page returned, up to 20 a call; set limit to what you need.
  • Five to thirty seconds. Pages that need a login or block automated readers come back missing, not as errors.
  • Use select_paths to stay inside one section ("/blog/.*") instead of crawling the whole site.

Siblings

This page as Markdown: /docs/tools/crawl_site.md. Something wrong or missing: hello@parel.si.