Home › Blog › ScraperAPI MCP server

ScraperAPI MCP server: scrape blocked pages from your AI agent

Updated 2026-10-25

This page has affiliate links, each marked "Affiliate link, #ad". We may be paid if you sign up, at no extra cost to you. See our affiliate disclosure.

The ScraperAPI MCP server lets an AI agent scrape web pages, including ones that block plain requests, just by asking. Connect Claude Code or Cursor to the remote server at https://mcp.scraperapi.com/mcp with your API key. Below are the exact configs from ScraperAPI's docs and the parameters your agent can set.

Related: what an MCP server is

What does the ScraperAPI MCP server do?

It connects your AI agent to ScraperAPI, so scraping a page becomes a prompt. (Affiliate link, #ad: we may be paid if you sign up, at no extra cost to you.) Ask your agent to scrape a URL and it calls ScraperAPI's scrape tool, which handles proxies and anti-bot blocks and returns the page. It suits pages that block plain requests, which is why it is our featured scraping pick (see best web scraping API for AI agents).

Remote or local: which server should you use?

Start with the remote server at https://mcp.scraperapi.com/mcp, which needs no local setup and starts automatically when your client calls it. ScraperAPI also documents a locally hosted server you run with Python or Docker, for teams that want the server on their own machines. Both use your ScraperAPI API key and the same scraping parameters.

Related: MCP context cheat sheet

Related: ScraperAPI in the directory

How do you add the ScraperAPI MCP server to Claude Code?

Run one claude mcp add command with your API key in an Authorization header, or paste ScraperAPI's JSON into .claude/settings.json. Both connect to the remote server through npx mcp-remote. Get your key from the ScraperAPI dashboard first, and keep it out of shared repos.

1. Get your API key. Sign in to ScraperAPI and copy your API key from the dashboard.Get a ScraperAPI API key → Affiliate link, #ad: we may be paid if you sign up, at no extra cost to you.
2. Add the server. Run the claude mcp add command below, replacing {YOUR_API_KEY} with your key, or paste the JSON config into .claude/settings.json.
3. Restart and check. Restart Claude Code and confirm the scraperapi server shows up in its MCP server list.
4. Scrape with a prompt. Ask Claude Code to scrape a page. Use the word scrape, since that is what ScraperAPI's docs say triggers the tool.

Claude Code CLI, from ScraperAPI's docs:

claude mcp add scraperapi -- npx mcp-remote https://mcp.scraperapi.com/mcp --header "Authorization: Bearer {YOUR_API_KEY}"

.claude/settings.json, from ScraperAPI's docs:

{
  "mcpServers": {
    "ScraperAPIRemote": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.scraperapi.com/mcp",
        "--header",
        "Authorization: Bearer {YOUR_API_KEY}"
      ]
    }
  }
}

Where should the API key live?

The key sits in your client config as a bearer header, so treat that file like a password. Use your user-level Claude Code settings rather than a project file you commit, and never paste the key into a chat. If a key leaks, rotate it in the ScraperAPI dashboard and update the config.

Related: best MCP servers for Claude Code

How do you add the ScraperAPI MCP server to Cursor?

Open Cursor Settings, go to Tools & MCP, click New MCP Server, choose Manual and paste ScraperAPI's JSON config with your API key. The same JSON shape works for the remote server in Cursor as in Claude Code. ScraperAPI also has guides for Claude Desktop, Cline and Windsurf.

What does a good first prompt look like?

Name the page and the output you want. ScraperAPI's examples ask Cursor to scrape Amazon search results for a product and return the first results, and ask Claude Code to pull property listings from a Redfin results page. Start with one URL, check the output, then ask for Markdown or text if HTML is too noisy.

Related: best MCP servers for Cursor

Which ScraperAPI parameters can your agent set?

The scrape tool takes a url plus optional settings: render for JavaScript pages, country_code for geotargeting, premium for residential IPs, ultra_premium for harder anti-bot pages, device_type for mobile user agents, output_format for markdown, text or json, and autoparse for supported domains.

Which settings should you turn on?

Start with none. Add render only when the page needs JavaScript to show its content, and output_format markdown when an LLM will read the result. Try premium or ultra_premium only after a plain request fails, and never both: ScraperAPI's docs say ultra_premium cannot be combined with premium. Your agent can set these per request.

When is ScraperAPI the right scraper for your agent?

When your agent needs single pages that block plain requests, such as stores, listings or sites behind anti-bot checks. If you mainly need search plus Markdown, Firecrawl fits (see Firecrawl MCP server). If you want structured Google results, look at Scrapingdog (see Scrapingdog Google Search API). Our comparison guide lays out each tool by job.

Can your agent click or fill forms through ScraperAPI?

The MCP server fetches and returns pages; it is not a browser your agent drives. For clicking, logging in or filling forms, use a browser automation server such as Playwright MCP, then keep ScraperAPI for the fetches that get blocked. Our comparison guide covers which tools handle interaction and which only fetch.

Related: ScraperAPI vs Scrapingdog vs Firecrawl · agent web search tools trend

See how we run our own agent crew in the playbook.

Building on Grok Bot? Start a crew from the DartzDabz Crew Builder template.

We may be paid by X based on how many people use our Crew Builder template.

See real agent setups you can copy, browse the agent tool directory, or submit a tool; listings are free.