Linked Web Provider

Linked Web Provider is an MCP server that offers comprehensive web interaction capabilities for AI assistants, providing advanced browsing, content extraction, and web automation features beyond basic URL fetching. This MCP bridge enables language models to navigate web pages, interact with forms, extract structured data from complex layouts, and perform multi-step web workflows, serving as a general-purpose web interaction layer for AI-driven automation tasks.

When to Use This MCP Server

Connect this server when...

• You need AI assistants to interact with web pages beyond simple content fetching, including form submission and navigation
• Your workflow involves extracting structured data from complex web layouts with tables, lists, and nested elements
• You want to automate multi-step web workflows like filling forms, clicking buttons, and capturing results
• You need to process web content from sites that require JavaScript rendering for content visibility
• You are building web data extraction pipelines that need intelligent parsing of diverse page structures

Consider alternatives when...

• You only need basic HTTP fetching without page interaction (use the simpler fetch MCP server)
• Your web automation needs require full browser recording and playback capabilities
• You need authenticated access to specific platforms that have their own dedicated MCP servers

Quick Start

Copy
# .mcp.json configuration
{
  "mcpServers": {
    "web-provider": {
      "command": "npx",
      "args": ["-y", "@mcp/web-provider-server"],
      "env": {
        "HEADLESS": "true"
      }
    }
  }
}

Connection setup:

1. Ensure Node.js 18+ is installed on your system
2. The server may require Chromium/Puppeteer for JavaScript rendering
3. Add the configuration above to your .mcp.json file
4. Restart your MCP client to activate the web provider

Example tool usage:

# Navigate and extract data
> Go to the product listing page and extract all product names, prices, and ratings

# Submit a form
> Fill in the search form with "AI tools" and return the search results

# Multi-step workflow
> Navigate to the registration page, fill in the form fields, and capture the confirmation

Core Concepts

Concept	Purpose	Details
Page Navigation	URL browsing	Load web pages with full rendering support including JavaScript execution and dynamic content
Content Extraction	Data retrieval	Extract text, tables, links, images, and structured data from rendered web page DOM
Form Interaction	Input automation	Fill form fields, select options, click buttons, and submit forms programmatically
Session Management	State persistence	Maintain browser session state (cookies, local storage) across multiple page interactions
Headless Rendering	Background processing	Run a headless browser for JavaScript rendering without displaying a visible browser window

Architecture:

+------------------+       +------------------+       +------------------+
|  Web Pages       |       |  Web Provider    |       |  AI Assistant    |
|  (Internet)      |<----->|  MCP Server      |<----->|  (Claude, etc.)  |
|                  | HTTP  |  + Headless      | stdio |                  |
|                  |       |  Browser Engine  |       |                  |
+------------------+       +------------------+       +------------------+
        |
        v
+------------------------------------------------------+
|  Navigate > Render > Extract > Interact > Return      |
+------------------------------------------------------+

Configuration

Parameter	Type	Default	Description
HEADLESS	boolean	true	Run browser in headless mode without visible window
viewport_width	integer	1280	Browser viewport width in pixels for page rendering
viewport_height	integer	720	Browser viewport height in pixels for page rendering
navigation_timeout	integer	30000	Maximum time in milliseconds to wait for page navigation to complete
block_resources	string[]	[]	Resource types to block during loading (images, fonts, stylesheets) for faster extraction

Best Practices

1. Use headless mode for server environments. Keep the HEADLESS flag enabled unless you need to debug page interactions visually. Headless mode uses fewer resources and is appropriate for production AI assistant workflows running in background environments.

2. Block unnecessary resources for faster extraction. When you only need text content, configure block_resources to skip loading images, fonts, and stylesheets. This significantly speeds up page loading and reduces bandwidth usage for data extraction tasks.

3. Wait for dynamic content before extraction. JavaScript-rendered pages may take time to load dynamic content after the initial page load. Allow sufficient navigation timeout for single-page applications that load data asynchronously before attempting to extract content.

4. Manage sessions carefully for multi-step workflows. When performing multi-step web interactions, ensure session state is maintained between steps. Cookies and authentication tokens need to persist across page navigations for workflows that require login or session continuity.

5. Respect website terms of service and rate limits. Automated web interaction should comply with the target website's terms of service. Avoid aggressive scraping patterns, excessive request rates, and accessing content behind authentication without authorization.

Common Issues

JavaScript-rendered content not visible in extraction. Ensure the headless browser engine is properly installed and configured. If using Puppeteer, Chromium must be available on the system. Check that the navigation timeout is long enough for the page's JavaScript to execute and render dynamic content.

Form submission fails with unexpected results. Web forms may have hidden fields, CSRF tokens, or JavaScript validation that must be satisfied before submission. Ensure all required fields are populated and any hidden inputs are included. Some forms require specific interaction sequences.

Memory usage grows during extended browsing sessions. The headless browser consumes memory for each page and tab. Close pages after extracting data to free resources. For long-running extraction sessions across many pages, periodically restart the browser instance to prevent memory accumulation.