AI Atlas

Research & Data · Developer Tools

Webscraping AI MCP Server logo

Webscraping AI MCP Server

webscraping-ai/webscraping-ai-mcp-server

A Model Context Protocol (MCP) server implementation that integrates with WebScraping.AI for web data extraction capabilities — Chromium JavaScript rendering, rotating datacenter/residential/stealth proxies, and AI-powered question answering and structured field extraction on any page.

View on GitHub4

Install

npx -y webscraping-ai-mcp

Client configuration

{
  "mcpServers": {
    "mcp-server-webscraping-ai": {
      "command": "npx",
      "args": [
        "-y",
        "webscraping-ai-mcp"
      ],
      "env": {
        "WEBSCRAPING_AI_API_KEY": "<WEBSCRAPING_AI_API_KEY>",
        "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "<WEBSCRAPING_AI_CONCURRENCY_LIMIT>",
        "WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING": "<WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING>"
      }
    }
  }
}

Environment variables

WEBSCRAPING_AI_API_KEYWEBSCRAPING_AI_CONCURRENCY_LIMITWEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXINGWEBSCRAPING_AI_DEFAULT_PROXY_TYPEWEBSCRAPING_AI_DEFAULT_JS_RENDERINGWEBSCRAPING_AI_DEFAULT_TIMEOUTWEBSCRAPING_AI_DEFAULT_JS_TIMEOUT
GitHub stars
4
Author
@webscraping-ai
Category
Research & Data, Developer Tools
License
MIT
Updated
Oct 6, 2026

About Webscraping AI MCP Server

Prefer zero setup? Use the hosted remote MCP server: add https://mcp.webscraping.ai/mcp to your MCP client and sign in with your WebScraping.AI account — OAuth handles auth, no API key or local install needed. This repo is the open-source stdio version for self-hosting and customization.

Features

  • Question answering about web page content

  • Structured data extraction from web pages

  • HTML content retrieval with JavaScript rendering

  • Plain text extraction from web pages

  • CSS selector-based content extraction

  • Google search results (SERP) as parsed JSON

  • Structured JSON for pages on supported sites (e.g. YouTube, TikTok, X, LinkedIn, Instagram, Reddit) from their normal URL

  • Multiple proxy types (datacenter, residential, stealth) with country selection

  • JavaScript rendering using headless Chrome/Chromium

  • Concurrent request management with rate limiting

Tools (12)

  • country

    Two-letter country code of the proxy used to fetch the page, us by default.

  • transcript_language

    Caption language to pick, e.g. en or de.

  • timeout

    Maximum web page retrieval time in ms (15000 by default, maximum is 30000)

  • js

    Execute on-page JavaScript using a headless browser (true by default)

  • js_timeout

    Maximum JavaScript rendering time in ms (2000 by default)

  • wait_for

    CSS selector to wait for before returning the page content

  • proxy

    Type of proxy: datacenter, residential, or stealth (residential by default).

  • custom_proxy

    Your own proxy URL in "http://user:password@host:port" format

  • device

    Type of device emulation.

  • error_on_404

    Return error on 404 HTTP status on the target page (false by default)

  • error_on_redirect

    Return error on redirect on the target page (false by default)

  • js_script

    Custom JavaScript code to execute on the target page

Details on this page are taken from the project's README. Open README

Supported clients

Clients mentioned in this server's README:

View all
Claude Desktop logo

Claude Desktop

Desktop · Freemium · Proprietary

Anthropic's official Claude AI desktop application. Supports MCP servers to extend functionality.

WindowsMacOS
Cursor logo

Cursor

Desktop · Freemium · Proprietary

The first agentic IDE. The Cursor editor truly merges how developers and AI work together, delivering a magical coding experience.

WindowsMacOSLinux

Related MCP servers

More servers
Firecrawl MCP Server logo

Firecrawl MCP Server

mendableai/firecrawl-mcp-server

2.8k

A Model Context Protocol (MCP) server that brings Firecrawl to MCP-compatible AI agents — search, scrape, and interact with the live web for clean, agent-ready context.

Research & Data
Exa Web Search logo

Exa Web Search

exa-labs/exa-mcp-server

1.4k

Connect AI agents to Exa for web search, content fetching, and multi-step research.

Research & Data
ArXiv logo

ArXiv

blazickjp/arxiv-mcp-server

1k

A local MCP server for agent literature work. The differentiator is original-LaTeX section reads, BibTeX from arXiv metadata, and topic watches. Papers stay on disk.

Research & Data
sail logo

sail

lakehq/sail

730

Sail is a drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads on a distributed, multimodal compute engine.

Research & Data

OpenAPI Proxy

janwilmake/openapi-mcp-server

554

A Model Context Protocol (MCP) server for Claude/Cursor that enables searching and exploring OpenAPI specifications through oapis.org.

Research & Data
Excel MCP Server logo

Excel MCP Server

haris-musa/excel-mcp-server

393

A Model Context Protocol server that lets AI assistants create, read and edit Excel workbooks. It needs no Microsoft Excel installation.

Research & Data