{"x402Version":2,"name":"Web Content Toolkit","description":"Fetch a public URL and extract its readable article, link-preview metadata, structured data (JSON-LD/Open Graph/microdata), sitemap URLs, HTTP/security response headers, trace its redirect chain, check its robots.txt crawl rules, extract and classify all its hyperlinks, or audit its on-page SEO.","version":"0.1.0","openapi":"https://web.openverbs.com/openapi.json","resources":[{"url":"https://web.openverbs.com/v1/extract","method":"POST","description":"Fetch a URL and return its main article as clean HTML, Markdown and plain text.","tags":["extract article text","readable content api","url to markdown","web page to text","html article extraction","fetch article content"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.006","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/unfurl","method":"POST","description":"Fetch a URL and return link-preview metadata from Open Graph, Twitter Card and standard meta tags.","tags":["link preview api","unfurl url metadata","open graph lookup","twitter card metadata","fetch page preview","url metadata extraction"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.006","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/structured","method":"POST","description":"Fetch a URL and pull out its machine-readable structured data: all JSON-LD blocks (schema.org), Open Graph and Twitter Card tags, standard named meta tags, and inline microdata items. Also returns the page title, canonical URL, language and the distinct schema.org types found. Ideal for turning a product, article, recipe, event or organization page into structured fields without bespoke scraping.","tags":["extract structured data","json ld extraction","schema org lookup","open graph structured data","microdata parser api","web page schema extraction"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.006","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/sitemap","method":"POST","description":"Discover a site's XML sitemaps (via robots.txt Sitemap: directives, an explicit sitemap URL, or the conventional /sitemap.xml) and parse them into a bounded list of URLs with their lastmod, changefreq and priority. Follows sitemap-index files to their child sitemaps. Fetches and URL counts are capped (with a `truncated` flag) to stay fast and memory-bounded. The reliable way for an agent to enumerate a site's pages.","tags":["parse sitemap","discover sitemap urls","robots sitemap lookup","website page enumeration","sitemap xml parser","list site urls"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.006","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/headers","method":"POST","description":"Fetch a URL (following redirects) and return the final response's HTTP headers, the server banner and content-type, plus a security-header audit reporting HSTS, Content-Security-Policy, X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Permissions-Policy and the cross-origin policies (with the missing ones listed). No body is downloaded or parsed.","tags":["http headers lookup","response headers api","check security headers","hsts csp inspector","server header check","fetch url headers"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.003","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/redirects","method":"POST","description":"Follow a URL's redirect chain (301/302/303/307/308) to its final destination and return every hop with its status and Location. Unwraps link shorteners and cloaked/tracking links so an agent can see where a link really goes before following it. Each hop is SSRF-validated and the chain is capped.","tags":["url redirect tracer","follow redirect chain","unshorten url api","expand short link","resolve final url","http redirect lookup"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.003","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/robots","method":"POST","description":"Fetch a site's robots.txt and evaluate crawl permissions: given a URL (plus optional extra paths) and a user-agent, return whether each path is allowed or disallowed, the rule that matched, the user-agent group, the crawl-delay and any declared sitemaps. Implements the Robots Exclusion Protocol (RFC 9309) with longest-match-wins, Allow-over-Disallow tie-breaking and * / $ wildcards.","tags":["robots.txt parser","robots txt checker","is url crawlable","crawl permission api","user-agent allow disallow","crawler compliance api"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.006","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/links","method":"POST","description":"Fetch a URL and extract every hyperlink on the page, each resolved to an absolute URL and classified as internal vs external (by registrable host), with its anchor text, rel attributes (nofollow, sponsored, ugc) and whether it opens in a new tab. Returns internal/external/nofollow counts. For SEO link audits, internal-linking analysis and outbound-link/nofollow compliance checks.","tags":["extract links api","page links api","internal external links","nofollow link checker","outbound links api","link audit api"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.006","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]},{"url":"https://web.openverbs.com/v1/seo-audit","method":"POST","description":"Fetch a public page and grade its on-page SEO: title and meta-description length, H1 and heading outline, canonical, meta-robots indexability, viewport/lang/charset, hreflang, Open Graph and Twitter Card completeness, image alt coverage and word count — plus optional focus-keyword placement and density. Returns a pass/warn/fail verdict per check and an overall 0–100 score.","tags":["on page seo audit","seo audit api","website audit","meta tag checker","on-page optimization","page seo analyzer"],"accepts":[{"scheme":"exact","network":"eip155:8453","price":"$0.006","payTo":"0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A","asset":"USDC"}]}]}