$ ~/registry/skill/microsoft-plugin-skills-microsoft-foundry-finetuning

SKILL

finetuning

Name: finetuning
Author: microsoft

Fine-tune, deploy, and evaluate models in Azure AI Foundry.

Stars

★ 232

Source

GitHub

Updated

2026-07-13

// securityCaution

Prompt only, no code
Open source
Community-trusted· 222

Summary

The material indicates an open-source, prompt/document-oriented fine-tuning skill with no declared required secrets or fixed remote endpoints, sourced from GitHub with some community adoption, so overall risk is relatively low. Caution is still warranted because the docs mention workflows such as training submission, deployment, evaluation, and large file uploads, which may involve cloud-side data handling and script execution in practice, but the current evidence does not justify a high-risk rating.

CredentialsLow risk

The material explicitly states there are no required secrets or environment variables; as a prompt-only skill, it does not itself appear to request or expose credentials. Real Azure AI Foundry operations may rely on an existing cloud login session, but the provided material shows no sign of extra credential collection or misuse.

Network egressCaution

Although no fixed remote host is declared, the described functions include training job submission, deployment, evaluation, and large file uploads, which reasonably imply normal communication with Azure AI Foundry/Azure APIs and possible upload of training or evaluation data to relevant cloud services. The material shows no red flag of sending data to unrelated or unknown third-party endpoints.

Code executionCaution

The README lists multiple Python scripts (such as submit_training, deploy_model, and evaluate_model), indicating the repository can be used to run local scripts and initiate cloud operations; however, the system also marks this skill as prompt-only, so the skill itself appears to be guidance rather than an automatic executor. No sign of unusual system privilege requests or stealthy execution is present.

Data accessCaution

The docs cover dataset preparation, format conversion, synthetic data generation, quality scoring, and large file uploads, implying that real workflows may handle user-provided local training data and evaluation outputs. Based on the current material, there is no stated access beyond what is relevant to fine-tuning, and no sign of excessive authorization requests.

Supply chainLow risk

The source is an auditable open-source GitHub repository associated with Microsoft, with some community adoption (222 stars), which materially lowers supply-chain risk. The license and maintenance status are not explicit in the material, leaving some uncertainty, but not enough to constitute a high-risk signal.

Recommendations

Verify the latest commits, maintenance activity, and actual license before use.
If you run the documented scripts, review their network calls, input/output paths, and Azure resource operations first.
Do not upload sensitive or regulated training data to the cloud without prior minimization and sanitization.
Test training and deployment workflows in an isolated subscription/resource group with limited permissions.

Audited by: gpt-5.4 · 2026-06-17

// install

Copy the install command and let the AI configure it · recommended for beginners

Please install the "finetuning" skill from askskill:
1. Download https://raw.githubusercontent.com/microsoft/GitHub-Copilot-for-Azure/main/plugin/skills/microsoft-foundry/finetuning/SKILL.md
2. Save it as ~/.claude/skills/finetuning/SKILL.md
3. Reload skills and tell me it's ready

// download

Download SKILL.mdInstall manifest ↗

// examples

Prepare SFT Training Data

Prompt

I want to run supervised fine-tuning in Azure AI Foundry. Help me convert a batch of customer support Q&A into an SFT-ready training dataset, validate required fields, inspect sample quality and length distribution, detect dirty data, and give upload recommendations.

Expected

A SFT-ready dataset format guide, cleaning suggestions, quality check results, and an upload checklist.

Submit a DPO Fine-Tuning Job

Prompt

Guide me through launching a DPO fine-tuning job in Azure AI Foundry, including dataset requirements, recommended training parameters, job submission steps, common error troubleshooting, and deployment steps after training completes.

Expected

A complete DPO fine-tuning plan with parameter guidance, submission steps, troubleshooting, and deployment instructions.

Evaluate and Calibrate a Fine-Tuned Model

Prompt

I have finished fine-tuning. Help me design an evaluation plan to compare the base model and the fine-tuned model, explain how to calibrate the grader, choose evaluation metrics, interpret the results, and decide whether the model is ready for production.

Expected

An evaluation and grader calibration plan with metrics, comparison methods, result interpretation, and production-readiness advice.

// docs

Fine-Tuning on Azure AI Foundry

Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.

When to Use

Use this sub-skill when the user asks about:

Fine-tuning a model (SFT, DPO, or RFT)
Preparing, validating, or formatting training data
Submitting, monitoring, or diagnosing training jobs
Calibrating graders or pass thresholds for RFT
Deploying or evaluating a fine-tuned model
Choosing between training types (SFT vs DPO vs RFT)
Distillation, synthetic data generation, or dataset quality scoring
Large file uploads for training data
Cleaning up fine-tuning resources (files, deployments)

Do NOT use for: General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

Workflows

Stage	Guide
Quick start	workflows/quickstart.md
Full pipeline	workflows/full-pipeline.md
Create data	workflows/dataset-creation.md
Iterate	workflows/iterative-training.md
Diagnose	workflows/diagnose-poor-results.md

References

Topic	File
SFT vs DPO vs RFT	references/training-types.md
Hyperparameters	references/hyperparameters.md
Data formats	references/dataset-formats.md
Grader design (RFT)	references/grader-design.md
Reward hacking	references/reward-hacking.md
Agentic RFT (tools)	references/agentic-rft.md
Deployment	references/deployment.md
Training curves	references/training-curves.md
Evaluation	references/evaluation.md
Vision fine-tuning	references/vision-fine-tuning.md
Large file uploads	references/large-file-uploads.md
Platform gotchas	references/platform-gotchas.md

Scripts

Script	Purpose
`scripts/submit_training.py`	Submit SFT/DPO/RFT jobs
`scripts/monitor_training.py`	Poll job until completion
`scripts/calibrate_grader.py`	Find optimal RFT pass_threshold
`scripts/check_training.py`	Analyze curves, list checkpoints
`scripts/deploy_model.py`	Deploy via ARM REST API
`scripts/evaluate_model.py`	LLM judge evaluation
`scripts/convert_dataset.py`	Convert between SFT/DPO/RFT formats
`scripts/generate_distillation_data.py`	Generate synthetic training data
`scripts/score_dataset.py`	Quality scoring on training data
`scripts/cleanup.py`	Delete old files and deployments
`scripts/validate/`	Data validators (SFT, DPO, RFT) + stats

Rules

Always baseline first — evaluate the base model before fine-tuning
Validate data before submitting — run scripts/validate/validate_sft.py
Calibrate RFT graders — target 25-50% failure rate on the base model
Evaluate checkpoints — don't blindly deploy the final one
Measure token cost alongside accuracy when comparing models

Quick Reference

Task	Command
Validate SFT data	`python scripts/validate/validate_sft.py data.jsonl`
Submit SFT job	`python scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft`
Monitor job	`python scripts/monitor_training.py --job-id ftjob-xxx`
Analyze curves	`python scripts/check_training.py --job-id ftjob-xxx`

…

View full docs ↗

Discover Azure OpenAI capacity, quotas, and best regions for deployment.

microsoftGet→

$ loading_