Chat Performance Testing

When to use

• Before/after modifying chat rendering code (chatListRenderer.ts, chatInputPart.ts, markdown rendering)
• When changing the streaming response pipeline or SSE processing
• When modifying disposable/lifecycle patterns in chat components
• To compare performance between two VS Code releases
• In CI to gate PRs that touch chat UI code

Quick start

# Run perf regression test (compares local dev build vs VS Code 1.115.0):
npm run perf:chat -- --scenario text-only --runs 3

# Run all scenarios with no baseline (just measure):
npm run perf:chat -- --no-baseline --runs 3

# Compare two local builds (apples-to-apples):
npm run perf:chat -- --build /path/to/build-A --baseline-build /path/to/build-B --runs 5

# Build a local production package and compare against a release:
npm run perf:chat -- --production-build --baseline-build 1.115.0 --runs 5

# Run memory leak check (10 messages in one session):
npm run perf:chat-leak

# Run leak check with more messages for accuracy:
npm run perf:chat-leak -- --messages 20 --verbose

Perf regression test

Script: scripts/chat-simulation/test-chat-perf-regression.js npm: npm run perf:chat

Launches VS Code via Playwright Electron, opens the chat panel, sends a message with a mock LLM response, and measures timing, layout, and rendering metrics. By default, downloads VS Code 1.115.0 as a baseline, benchmarks it, then benchmarks the local dev build and compares.

Key flags

Comparing two remote builds

# Compare 1.110.0 against 1.115.0 (no local build needed):
npm run perf:chat -- --build 1.110.0 --baseline-build 1.115.0 --runs 5

Comparing two local builds

Both --build and --baseline-build accept local paths to VS Code executables. This enables apples-to-apples comparisons between any two builds:

# Compare two dev builds (e.g. feature branch vs main):
npm run perf:chat -- \
  --build .build/electron/Code\ -\ OSS.app/Contents/MacOS/Code\ -\ OSS \

…