Fetch URL MCP Server

A web content fetcher MCP server that converts HTML to clean, AI and human readable markdown.

Overview

The Fetch URL MCP Server provides a standardized interface for fetching public web content and transforming it into Markdown enriched with structured metadata. It validates URLs, applies noise removal heuristics, and caches results for reuse. The server supports both inline and task-based execution modes, making it suitable for a wide range of client applications and LLM interactions.

Key Features

• fetch-url validates public HTTP(S) URLs, fetches the page, and returns cleaned Markdown plus structured metadata.
• The tool advertises optional task support and emits progress updates while fetching and transforming larger pages.
• GitHub, GitLab, Bitbucket, and Gist page URLs are rewritten to raw-content endpoints when possible before fetch.
• internal://instructions and internal://cache/{namespace}/{hash} expose built-in guidance and cached Markdown as MCP resources.
• HTTP mode adds host/origin validation, auth, rate limiting, health checks, OAuth protected-resource metadata, and cached-download URLs.

Requirements

• Node.js >=24 (from package.json)
• Docker is optional if you want to run the published container image.

Quick Start

Use this standard MCP client configuration:

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

Client Configuration

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

code --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

code-insiders --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

extensions:
  fetch-url-mcp:
    name: fetch-url-mcp
    cmd: npx
    args: ['-y', '@j0hanz/fetch-url-mcp@latest']
    enabled: true
    type: stdio
    timeout: 300

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

claude mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
      "env": {}
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "amp.mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

amp mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

codex mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest

[mcp_servers.fetch-url-mcp]
command = "npx"
args = ["-y", "@j0hanz/fetch-url-mcp@latest"]

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "context_servers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
      "env": {}
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

Use Cases

• Fetch documentation pages, blog posts, or reference material into Markdown before sending them to an LLM.
• Retrieve repository-hosted content from GitHub, GitLab, Bitbucket, or Gists and let the server rewrite page URLs to raw endpoints when possible.
• Reuse cached Markdown through internal://cache/{namespace}/{hash} or bypass the cache with forceRefresh for time-sensitive pages.
• Use task mode for large pages or slower sites when the inline response would otherwise be truncated or delayed.

Architecture

[MCP Client]
  ├─ stdio -> `src/index.ts` -> `startStdioServer()` -> `createMcpServer()`
  └─ HTTP (`--http`) -> `src/index.ts` -> `startHttpServer()` -> HTTP dispatcher
       ├─ `GET /health`
       ├─ `GET /.well-known/oauth-protected-resource`
       ├─ `GET /.well-known/oauth-protected-resource/mcp`
       ├─ `GET /mcp/downloads/{namespace}/{hash}`
       └─ `POST|GET|DELETE /mcp`

`createMcpServer()`
  ├─ registers tool: `fetch-url`
  ├─ registers prompt: `get-help`
  ├─ registers resources:
  │    - `internal://instructions`
  │    - `internal://cache/{namespace}/{hash}`
  ├─ enables capabilities: completions, logging, resources, prompts, tasks
  └─ installs task handlers, log-level handling, and shutdown cleanup

`fetch-url` execution
  ├─ validate input with `fetchUrlInputSchema`
  ├─ normalize URL and block local/private targets unless allowed
  ├─ rewrite supported code-host URLs to raw endpoints when possible
  ├─ fetch and cache content via the shared pipeline
  ├─ transform HTML into Markdown in the transform worker path
  └─ validate `structuredContent` with `fetchUrlOutputSchema`

Request Lifecycle

[Client] -- initialize {protocolVersion, capabilities} --> [Server]
[Server] -- {protocolVersion, capabilities, serverInfo} --> [Client]
[Client] -- notifications/initialized --> [Server]
[Client] -- tools/call {name, arguments} --> [Server]
[Server] -- {content: [{type, text}], structuredContent?, isError?} --> [Client]

MCP Surface

Tools

fetch-url

Fetch public webpages and convert HTML into AI-readable Markdown. The tool is read-only, does not execute page JavaScript, can bypass the cache with forceRefresh, and supports optional task mode for larger or slower fetches.

The response is returned as MCP text content and, when validation succeeds, as structuredContent containing url, resolvedUrl, finalUrl, title, metadata, markdown, fromCache, fetchedAt, contentSize, and truncated.

1. [Client] -- tools/call {name: "fetch-url", arguments} --> [Server]
2. [Server] -- dispatch("fetch-url") --> [src/tools/fetch-url.ts]
3. [Handler] -- validate(fetchUrlInputSchema) --> normalize / fetch / transform
4. [Handler] -- validate(fetchUrlOutputSchema) --> assemble content + structuredContent
5. [Server] -- result or tool error --> [Client]

Resources

Prompts

MCP Capabilities

Tool Annotations

Structured Output

• fetch-url publishes an explicit outputSchema and returns structuredContent when the assembled response passes validation.

Configuration

HTTP Endpoints

Security

Development

Build and Release

• The repository includes release automation under .github/workflows/.
• Dockerfile and docker-compose.yml are available for container-based packaging and local runs.
• npm run prepublishOnly runs the release gate: lint, type-check, and build.

Troubleshooting

• For stdio mode, avoid writing logs to stdout; keep logs on stderr.
• For HTTP mode, verify MCP protocol headers and endpoint routing.
• Update client snippets when client MCP configuration formats change.

Credits

Contributing and License

• License: MIT
• Contributions are welcome via pull requests.