Research & Data · Developer Tools

WEB-SCRAPING-MCP logo

WEB-SCRAPING-MCP

MaitreyaM/WEB-SCRAPING-MCP

This project provides an MCP (Model Context Protocol) server that uses the crawl4ai library to perform web scraping and intelligent content extraction tasks.

Install

docker run -it --rm -p 8002:8002 --env-file .env crawl4ai-mcp-server
GitHub stars
2
Category
Research & Data, Developer Tools
License
MIT
Updated
Oct 6, 2026

About WEB-SCRAPING-MCP

It allows AI agents (like Claude, or agents built with LangChain/LangGraph) to interact with web pages, retrieve content, search for specific text, and perform LLM-based extraction based on natural language instructions.

Features

  • Exposes MCP tools for web interaction:

  • Configurable via environment variables (API keys).

  • Includes Docker configuration (Dockerfile) for easy, self-contained deployment.

  • Communicates over Server-Sent Events (SSE) on port 8002 by default.

Tools (3)

  • scrape_url

    Scrape a webpage and return its content in Markdown format.

  • extract_text_by_query

    Extract relevant text snippets from a webpage that contain a specific search query.

  • smart_extract

    Intelligently extract specific information from a webpage using the configured LLM (currently requires Google Gemini API key) based on a natural language…

Details on this page are taken from the project's README. Open README

Related MCP servers

More servers
Firecrawl MCP Server logo

Firecrawl MCP Server

mendableai/firecrawl-mcp-server

2.8k

A Model Context Protocol (MCP) server that brings Firecrawl to MCP-compatible AI agents — search, scrape, and interact with the live web for clean, agent-ready context.

Research & Data
Exa Web Search logo

Exa Web Search

exa-labs/exa-mcp-server

1.4k

Connect AI agents to Exa for web search, content fetching, and multi-step research.

Research & Data
ArXiv logo

ArXiv

blazickjp/arxiv-mcp-server

1k

A local MCP server for agent literature work. The differentiator is original-LaTeX section reads, BibTeX from arXiv metadata, and topic watches. Papers stay on disk.

Research & Data
sail logo

sail

lakehq/sail

730

Sail is a drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads on a distributed, multimodal compute engine.

Research & Data

OpenAPI Proxy

janwilmake/openapi-mcp-server

554

A Model Context Protocol (MCP) server for Claude/Cursor that enables searching and exploring OpenAPI specifications through oapis.org.

Research & Data
Excel MCP Server logo

Excel MCP Server

haris-musa/excel-mcp-server

393

A Model Context Protocol server that lets AI assistants create, read and edit Excel workbooks. It needs no Microsoft Excel installation.

Research & Data