Scrapingdog

What is Scrapingdog?

Scrapingdog is a web scraping API. You forward the URL you want to scrape, with your API key, and get the page back while Scrapingdog manages proxies, browsers and blocking. Alongside the general scraping API it offers dedicated APIs that return parsed JSON for search engines, social platforms and e-commerce sites. (source: Scrapingdog homepage, Scrapingdog documentation, checked 2026-10-19)

What can Scrapingdog do?

• Web Scraping API: one GET request to https://api.scrapingdog.com/scrape with your API key and the target URL. Scrapingdog says it handles rotating proxies, CAPTCHA bypass and optional JavaScript rendering.
• Request options in the docs: JavaScript rendering, custom headers, premium residential proxies, geotargeting, sessions, CAPTCHA bypass, POST requests and screenshots.
• Search engine APIs: Google Search (organic results, ads, knowledge graph, AI overviews and other result features), plus Bing, DuckDuckGo, Brave, Yahoo and Baidu.
• Other dedicated APIs: Amazon, Walmart, eBay and Flipkart products; Google Maps, News, Scholar, Trends and Shopping; YouTube, TikTok and X profiles; job listings.
• LLM-ready output: the homepage says it can return a page as Markdown or JSON.

(source: Scrapingdog docs, Web Scraping API, Scrapingdog documentation, Scrapingdog homepage, checked 2026-10-19)

How would an AI agent use Scrapingdog?

An example research workflow, built only from capabilities Scrapingdog documents. DartzDabz hasn't tested it.

1. Add the Scrapingdog MCP server to Claude Code, Claude Desktop or Cursor, with your API key in the SCRAPINGDOG_API_KEY environment variable.
2. Ask the agent: "Find the top Google results for 'remote MCP servers' and summarize the first three pages."
3. The agent calls Scrapingdog's Google Search API and gets the organic results back as structured JSON.
4. It fetches each of the top result pages through the Web Scraping API, turning on JavaScript rendering for pages that need it.
5. It writes the summary and lists the URLs it read, so you can check them.

(source: Scrapingdog SDKs & MCP Server, Scrapingdog docs, Web Scraping API, Scrapingdog documentation, checked 2026-10-19)

How do you connect Scrapingdog to an agent?

• REST: plain GET requests with api_key and url query parameters. The docs show examples in cURL, Python, Node.js, PHP and Ruby.
• Official SDKs: Node.js (npm install scrapingdog-sdk) and Python (pip install scrapingdog). Scrapingdog says every method returns parsed, structured data.
• MCP server: launched with npx -y scrapingdog-mcp, for Claude Desktop, Claude Code, Cursor and other MCP clients.
• Keep the API key in an environment variable, as the SDK page recommends, never in your code.

(source: Scrapingdog docs, Web Scraping API, Scrapingdog SDKs & MCP Server, checked 2026-10-19)

MCP client config from Scrapingdog's SDKs page:

{
  "mcpServers": {
    "scrapingdog": {
      "command": "npx",
      "args": ["-y", "scrapingdog-mcp"],
      "env": { "SCRAPINGDOG_API_KEY": "APIKEY" }
    }
  }
}

Who is Scrapingdog for?

Developers and agent builders who want search-engine, e-commerce or social data as JSON without running their own proxies and headless browsers. Scrapingdog's homepage names uses such as SEO monitoring, rank tracking, competitor research and training LLMs with fresh web data. (source: Scrapingdog homepage, checked 2026-10-19)