When scraping web data, you run into very different page types: some are plain static HTML, while others require scrolling, pagination, or clicks before data appears. You do not want to start with an overly heavy setup, and you also want a workflow that stays reliable, so you need a stable set of options chosen by page complexity.
如果你追求稳定优先,不要先看“能不能抓”,而要先看页面类型。纯静态 HTML 页面,reclaimergold-mcp-web-scraper 最省事;有翻页、滚动、截图或交互,tai-dt-selenium-mcp-server 更稳;想走礼貌抓取、提取链接表格等结构化内容,eitan3-scrapy-mcp 更合适;如果你已经在 Rube MCP 体系里,且希望把抓取流程托管出去,composiohq-composio-skills-scrapingbee-automation 和 composiohq-composio-skills-zenrows-automation 更适合批量化与自动化编排。
| Name | Type | Stars | Verdict | |
|---|---|---|---|---|
| MCP Web Scraper | MCP | — | 适合静态网页 HTML 抓取与结构化信息提取;优点是目标清晰、流程轻;缺点是描述里只强调静态网页,不适合需要复杂浏览器交互的页面。 | |
| Selenium MCP Server | MCP | — | 适合需要翻页、滚动、截图或智能提取的动态页面;优点是覆盖真实浏览器操作场景更全面;缺点是相对更重,不适合所有简单静态抓取。 | |
| scrapy-mcp | MCP | — | 适合礼貌抓取网页并提取链接、表格与结构化内容;优点是定位明确,兼顾结构化提取与受限异步爬取;缺点是候选信息未强调浏览器级交互,对强前端动态页未必是首选。 | |
| scrapingbee-automation | SKILL | ★ 68k | 适合已在 Rube MCP 中编排网页抓取任务与数据流程;优点是便于自动化执行抓取流程;缺点是它是通过 Rube MCP 的 skill,更依赖既有自动化体系。 | |
| zenrows-automation | SKILL | ★ 68k | 适合希望通过 Rube MCP 自动化执行网页抓取与数据采集任务的场景;优点是面向网页抓取任务本身,便于批量流程化;缺点是同样属于 skill 方案,不如直接型 mcp 那样强调本地逐步调试。 |
适合静态网页 HTML 抓取与结构化信息提取;优点是目标清晰、流程轻;缺点是描述里只强调静态网页,不适合需要复杂浏览器交互的页面。
适合需要翻页、滚动、截图或智能提取的动态页面;优点是覆盖真实浏览器操作场景更全面;缺点是相对更重,不适合所有简单静态抓取。
适合礼貌抓取网页并提取链接、表格与结构化内容;优点是定位明确,兼顾结构化提取与受限异步爬取;缺点是候选信息未强调浏览器级交互,对强前端动态页未必是首选。
适合已在 Rube MCP 中编排网页抓取任务与数据流程;优点是便于自动化执行抓取流程;缺点是它是通过 Rube MCP 的 skill,更依赖既有自动化体系。
适合希望通过 Rube MCP 自动化执行网页抓取与数据采集任务的场景;优点是面向网页抓取任务本身,便于批量流程化;缺点是同样属于 skill 方案,不如直接型 mcp 那样强调本地逐步调试。
Start with page complexity. Try `reclaimergold-mcp-web-scraper` for static pages first. If the target needs scrolling, pagination, screenshots, or browser interaction, move to `tai-dt-selenium-mcp-server`. If you care more about polite crawling plus extracting links and tables, use `eitan3-scrapy-mcp`. If you already work inside Rube MCP workflows and want batch automation, compare `composiohq-composio-skills-scrapingbee-automation` and `composiohq-composio-skills-zenrows-automation`.
If the content is already available as static HTML, you usually do not need `tai-dt-selenium-mcp-server` first. Browser automation is better for real page interactions. For static content plus structured extraction, `reclaimergold-mcp-web-scraper` is often more direct and lighter.
If you want scraping embedded in Rube MCP automation, `composiohq-composio-skills-scrapingbee-automation` and `composiohq-composio-skills-zenrows-automation` fit better because they are designed to automate scraping tasks through Rube MCP. If you need more direct, controllable, page-by-page debugging, `eitan3-scrapy-mcp`, `reclaimergold-mcp-web-scraper`, and `tai-dt-selenium-mcp-server` behave more like hands-on scraping tools.
The most common mistake is misjudging the page type. If you treat an interactive page as static, `reclaimergold-mcp-web-scraper` may seem unreliable. If you send every simple page to `tai-dt-selenium-mcp-server`, the workflow becomes unnecessarily heavy. Another pitfall is when the target is really more of a managed scraping or broad structured-data workflow, where `composiohq-composio-skills-scrapingbee-automation` or `composiohq-composio-skills-zenrows-automation` can be a better fit.