map_site

Every URL on a website, or under one path, in one call: the pages to read, before reading any of them. ₹0.20 per call; a site with no URLs is ₹0.

What it does

The URLs of a website in one call, from the sources that already list them (its sitemaps, the links on its home page and public web indexes), in data.rows[] with url, sources, last_seen and depth; data.meta says how many pages were read (pages_fetched) and whether a source failed (partial, warnings). include keeps only URLs under a path ("/team") or matching a glob ("/blog/*"); max_urls caps the list (default and maximum 5,000); mode: "full" adds the Wayback Machine and up to twenty seconds.

Use it before reading anything. When a list spans several pages (a team page, a blog, a catalogue), the map hands back every page's address in one call, and the next step is read_pages for up to ten of those pages in one call, or extract_structured for the entries as a table. It lists addresses only: it does not read the pages (read_page does), and it knows only the URLs the site publishes. In the live check on a 98-person team page split over five pages, the map returned the 99 person pages under /team and not the /team/page/2/ to /page/5/ list pages, which are not in the site's sitemap.

India: any public site; send the host as a bare domain.

Inputs

Generated from the contract captured from staging on 2026-11-05.

FieldTypeRequiredDescription
modefast or fullnofull adds the Wayback Machine, up to 20 s more. default "fast"
domainstringyesthe site's host; a leading https:// is dropped, www and the bare host are one site. e.g. "example.com"
includestringnokeep only matching URLs: a path prefix (/docs) or a glob with * over the path
max_urlsintegernodefault 5000; max 5000
include_subdomainsbooleannodefault false

What drives the cost

Nothing you can turn: flat per call whatever the size of the site or max_urls. The source bills by the pages its scan reads (robots.txt, the home page, each sitemap, each index reply); the live call read 16 pages and the source charged under a tenth of what is shown, so a call usually costs less than the price shown; a site with no URLs is class: "ok_free", ₹0. Usually five to thirty seconds; a large site can take over a minute, so the timeout is 120 s. A repeat within a week is answered from the source's cache.

Price today

Price today₹0.20 per call (cost_paise: 20) at ₹93.93 per US dollar on 17 November 2026
Hold before the call₹0.38 (hold_paise: 38); when the call settles the real price is charged and the unused part is released
₹0 whenit fails, or the provider answers but there is nothing in it (an empty list, no email, no page): class: "ok_free", ₹0
Timeout180 s, then upstream_timeout (₹0, retryable)
Over MCPrun_tool with tool: "map_site"
Over RESTPOST https://api.parel.si/v1/run with "tool": "map_site"
Confirmationnot needed (the estimate is under the ₹200 threshold)

Request

curl https://api.parel.si/v1/run \
  -H "Authorization: Bearer $PAREL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"tool":"map_site","input":{"domain":"americanindustrial.com","include":"/team","max_urls":200}}'

What comes back

data.rows[] and data.meta. A real answer recorded during the live check of 2026-11-04 (americanindustrial.com under /team: 100 rows in 22 s), trimmed to three rows:

{
  "meta": {
    "blocked": [],
    "elapsed_ms": 19796,
    "from_cache": false,
    "llm_calls": 0,
    "pages_fetched": 16,
    "partial": false,
    "warnings": [
      "module: site-scan: CC-MAIN-2026-43-index gave no urls (status 0 unavailable, 0 bytes)",
      "module: site-scan: CC-MAIN-2026-39-index gave no urls (status 0 unavailable, 0 bytes)",
      "module: site-scan: commoncrawl 0 · links 53 · sitemap 277 · union 100"
    ]
  },
  "recipe": "site-scan",
  "rows": [
    {
      "depth": 1,
      "last_seen": "2026-10-29T02:02:39+00:00",
      "sources": [
        "sitemap",
        "links"
      ],
      "url": "https://americanindustrial.com/team"
    },
    {
      "depth": 2,
      "last_seen": "2026-06-01T16:41:54+00:00",
      "sources": [
        "sitemap"
      ],
      "url": "https://americanindustrial.com/team/achintya-saxena"
    },
    {
      "depth": 2,
      "last_seen": "2026-06-02T21:25:08+00:00",
      "sources": [
        "sitemap"
      ],
      "url": "https://americanindustrial.com/team/adam-traina"
    },
    "… 97 more item(s) truncated"
  ],
  "version": 1
}

Notes

  • Flat per call. include narrows to a path prefix ("/team") or a glob ("/blog/*"); max_urls caps the list (default 5,000, the maximum here). No URLs is free.
  • Only URLs the site publishes: sitemaps, home-page links and public indexes. Paginated list pages (/page/2/) are often absent while every item's own page is present.
  • Usually five to thirty seconds; a large site can take over a minute, so the timeout is generous. A repeat within a week is answered from the source's cache.
  • Each row: url, sources (sitemap, links, commoncrawl), last_seen, depth.

Siblings

This page as Markdown: /docs/tools/map_site.md. Something wrong or missing: hello@parel.si.