scrape-do-automation

通过 Rube MCP 自动执行 Scrape Do 抓取任务并获取结构化网页数据。

来源

GitHub

更新于

2026-06-07

// 安全评估低风险

仅提示词，不执行代码
开源可审计

正在进行安全审计…

凭证密钥
网络外发
代码执行
数据访问
来源供应链

// 安装

复制安装指令，让 AI 自动完成配置 · 推荐新手

请帮我安装 askskill 上的 "scrape-do-automation" 技能：
1. 下载 https://raw.githubusercontent.com/ComposioHQ/awesome-claude-skills/master/composio-skills/scrape-do-automation/SKILL.md
2. 保存为 ~/.claude/skills/scrape-do-automation/SKILL.md
3. 装好后重载技能，告诉我可以用了

// 下载

下载 SKILL.md机读安装清单 ↗

// 用法示例

批量抓取竞品页面

输入

先搜索可用的 Scrape Do 相关工具及其最新参数 schema，然后用合适的工具批量抓取这 20 个竞品官网页面，提取标题、描述、价格信息和主要 CTA，最终输出为结构化表格。

预期产出

返回按页面整理的结构化数据表，包含核心字段与抓取结果摘要。

提取电商商品信息

输入

先搜索 Scrape Do 工具的最新 schema，再抓取这个电商分类页及前 3 页商品链接，提取每个商品的名称、价格、评分、评论数和商品链接，并去重输出 JSON。

预期产出

输出去重后的商品 JSON 列表，字段完整且便于后续分析。

监测网页内容变化

输入

先搜索当前可用的 Scrape Do 工具与参数 schema，然后抓取这个目标页面，与我提供的上次抓取结果对比，列出新增、删除和修改的内容，按变化类型生成摘要。

预期产出

输出变更对比结果，包含差异明细和按类型汇总的变化摘要。

// 文档

Scrape Do Automation via Rube MCP

Automate Scrape Do operations through Composio's Scrape Do toolkit via Rube MCP.

Toolkit docs: composio.dev/toolkits/scrape_do

Prerequisites

Rube MCP must be connected (RUBE_SEARCH_TOOLS available)
Active Scrape Do connection via RUBE_MANAGE_CONNECTIONS with toolkit scrape_do
Always call RUBE_SEARCH_TOOLS first to get current tool schemas

Setup

Get Rube MCP: Add https://rube.app/mcp as an MCP server in your client configuration. No API keys needed — just add the endpoint and it works.

Verify Rube MCP is available by confirming RUBE_SEARCH_TOOLS responds
Call RUBE_MANAGE_CONNECTIONS with toolkit scrape_do
If connection is not ACTIVE, follow the returned auth link to complete setup
Confirm connection status shows ACTIVE before running any workflows

Tool Discovery

Always discover available tools before executing workflows:

RUBE_SEARCH_TOOLS
queries: [{use_case: "Scrape Do operations", known_fields: ""}]
session: {generate_id: true}

This returns available tool slugs, input schemas, recommended execution plans, and known pitfalls.

Core Workflow Pattern

Step 1: Discover Available Tools

RUBE_SEARCH_TOOLS
queries: [{use_case: "your specific Scrape Do task"}]
session: {id: "existing_session_id"}

Step 2: Check Connection

RUBE_MANAGE_CONNECTIONS
toolkits: ["scrape_do"]
session_id: "your_session_id"

Step 3: Execute Tools

RUBE_MULTI_EXECUTE_TOOL
tools: [{
  tool_slug: "TOOL_SLUG_FROM_SEARCH",
  arguments: {/* schema-compliant args from search results */}
}]
memory: {}
session_id: "your_session_id"

Known Pitfalls

Always search first: Tool schemas change. Never hardcode tool slugs or arguments without calling RUBE_SEARCH_TOOLS
Check connection: Verify RUBE_MANAGE_CONNECTIONS shows ACTIVE status before executing tools
Schema compliance: Use exact field names and types from the search results
Memory parameter: Always include memory in RUBE_MULTI_EXECUTE_TOOL calls, even if empty ({})
Session reuse: Reuse session IDs within a workflow. Generate new ones for new workflows
Pagination: Check responses for pagination tokens and continue fetching until complete

Quick Reference

Operation	Approach
Find tools	`RUBE_SEARCH_TOOLS` with Scrape Do-specific use case
Connect	`RUBE_MANAGE_CONNECTIONS` with toolkit `scrape_do`
Execute	`RUBE_MULTI_EXECUTE_TOOL` with discovered tool slugs
Bulk ops	`RUBE_REMOTE_WORKBENCH` with `run_composio_tool()`
Full schema	`RUBE_GET_TOOL_SCHEMAS` for tools with `schemaRef`