POST /knowledge/crawls
Updated Oct 1, 20261 minute read
Start a crawl
Sends the crawler to a site. Every scraped page lands as an ordinary web document while the crawl runs. Scraping a site again updates the pages already in the library and asks for a live copy. Up to 3 crawls at once, and the same site cannot be crawled twice in parallel.
Part of Knowledge.
Request body
application/json, required.
| Field | Type | Required | Description |
|---|---|---|---|
url | string | Yes |
Responses
| Status | Description |
|---|---|
202 | Created. |
400 | The request is malformed, and the message names the field. |
401 | The bearer token is missing, revoked, or expired. |
409 | It already exists, or it was already settled. |
429 | Too many requests. Wait the seconds in Retry-After. Limits depend on the plan. See rate limits. |
202 body
| Field | Type | Required | Description |
|---|---|---|---|
id | string | No |
Example request
curl -X POST https://api.median.sh/v1/knowledge/crawls \
-H "Authorization: Bearer $BEARER_AUTH" \
-H "content-type: application/json" \
-d '{"url":"docs.example.com"}'