$ loading_
Migrate HDInsight Spark, Hive, and Oozie workloads to Microsoft Fabric.
Copy the install command and let the AI configure it · recommended for beginners
Please install the "hdinsight-migration" skill from askskill: 1. Download https://raw.githubusercontent.com/microsoft/skills-for-fabric/main/skills/hdinsight-migration/SKILL.md 2. Save it as ~/.claude/skills/hdinsight-migration/SKILL.md 3. Reload skills and tell me it's ready
Please migrate this Azure HDInsight Spark code to Microsoft Fabric: remove HiveContext and standalone SparkContext initialization, use the pre-instantiated SparkSession, convert WASB/ABFS paths to OneLake abfss URLs, and explain the reason for each change.
Returns Fabric-ready Spark code with explanations of the key migration changes.
Please migrate the following Hive DDL to Microsoft Fabric Lakehouse: convert STORED AS ORC, external table definitions, and storage paths into Delta Lake-compatible table schemas and locations; also note any Hive metastore semantic differences.
Outputs converted Delta Lake table definitions and lists syntax and storage mapping differences.
I am retiring an HDInsight cluster. Please migrate this Oozie workflow/coordinator to Microsoft Fabric Pipeline: identify matching activities for spark, hive, shell, and sqoop actions, propose schedule triggers, and explain which file and credential operations should be replaced with notebookutils.
Provides a Fabric Pipeline mapping plan, trigger recommendations, and a script replacement checklist.
Update Check — ONCE PER SESSION (mandatory) The first time this skill is used in a session, run the check-updates skill before proceeding.
- GitHub Copilot CLI / VS Code: invoke the
check-updatesskill.- Claude Code / Cowork / Cursor / Windsurf / Codex: compare local vs remote package.json version.
- Skip if the check was already performed earlier in this session.
CRITICAL NOTES
- To find workspace details (including its ID) from a workspace name: list all workspaces, then use JMESPath filtering
- To find item details (including its ID) from workspace ID, item type, and item name: list all items of that type in that workspace, then use JMESPath filtering
- HDInsight has no
mssparkutilsordbutilsequivalent —notebookutilsis net-new capability being introducedHiveContextandSQLContextare legacy Spark 1.x/2.x APIs — Fabric uses Spark 3.xSparkSessionexclusivelywasb://paths are deprecated and require a Storage Account key or SAS — replace with OneLake shortcuts
Read these companion documents before executing migration tasks:
az rest, az login, token acquisition, Fabric REST via CLIFor notebook and Lakehouse creation, see spark-authoring-cli. For Fabric Warehouse DDL/DML authoring, see sqldw-authoring-cli.
| Topic | Reference |
|---|---|
| Migration Workload Map | § Migration Workload Map |
| SparkSession & Context API Changes | § SparkSession API Changes |
| WASB / ABFS → OneLake Path Migration | path-migration.md |
| Hive DDL → Delta Lake / Lakehouse Schemas | hive-to-delta.md |
| Oozie → Fabric Pipelines | § Oozie → Fabric Pipelines |
Introducing notebookutils | § Introducing notebookutils |
| Before/After Code Patterns | code-patterns.md |
| Spark Configuration Differences | § Spark Configuration Differences |
| Must / Prefer / Avoid | § Must / Prefer / Avoid |
| Authentication & Token Acquisition | COMMON-CORE.md § Authentication |
| Lakehouse Management | SPARK-AUTHORING-CORE.md § Lakehouse Management |
| HDInsight Component | Fabric Target | Notes |
|---|---|---|
| Spark cluster (notebooks, scripts) | Fabric Spark (Lakehouse / Notebooks / SJD) | No persistent cluster — Starter Pool or Custom Pool provides on-demand Spark |
| Hive / HiveServer2 | Lakehouse SQL Endpoint + Lakehouse schemas | Delta Lake replaces Hive metastore; schemas provide namespace equivalent |
| HBase | Fabric Warehouse or Azure Cosmos DB (separate from Fabric) | HBase has no direct Fabric equivalent — assess workload access patterns |
| Oozie workflows | Fabric Data Pipelines | Map Oozie actions to Fabric activities; see § Oozie → Fabric Pipelines |
| YARN Resource Manager | Fabric Spark monitoring (Spark UI, Monitoring Hub) | No YARN — Fabric manages compute automatically |
| Ambari | Fabric Monitoring Hub + Admin Portal | Cluster health, capacity, and job monitoring |
…
Create, edit, deploy, and optimize Power BI semantic models and DAX.
Build end-to-end Bronze, Silver, and Gold lakehouse pipelines in Microsoft Fabric.
Search Microsoft Fabric items across workspaces and return locations and IDs.
Ask natural-language questions about Power BI reports and get business answers.
Migrate Azure Synapse Spark workloads to Microsoft Fabric with code replacements.
Migrate Databricks notebooks, jobs, and paths into Microsoft Fabric.