Instance Storage & Sessions

The Pattern

Problem: You run multiple instances of something (task runners, user sessions, build jobs). Each needs isolated storage, lifecycle tracking, and its own subdirectory tree. You also have expensive initialization (loading bundles, connecting to APIs) that should happen once, not per instance.

Approach: A filesystem-backed instance store with per-instance directories, per-instance file locking, and a session factory that prepares once and creates many.

Pattern proven in production across multiple Python CLI tools and web services.

Key Design Decisions

1. Instance-rooted directory layout

All data for a single instance lives under one directory. Example layout (adapt to your needs):

~/.my-tool/
  instances/{instance_id}/
    instance.json         — lifecycle metadata (status, timestamps)
    workspace/            — code, git repo, tests
    logs/                 — pipeline node logs, checkpoint.json
    events/               — events.jsonl, state.json, input-requests/
    sidecar-data/         — sidecar service data (databases, repos)
    output/               — artifacts (branch_name.txt, pr_url.txt)

This is defined as:

def get_instance_dir(instance_id: str) -> Path:
    d = get_data_root() / "instances" / instance_id
    d.mkdir(parents=True, exist_ok=True)
    return d

# Subdirectory names used by other modules:
WORKSPACE_DIR = "workspace"
LOGS_DIR = "logs"
EVENTS_DIR = "events"

The design choice of instance-rooted (not type-rooted) matters: you can ls, tar, or rm -rf one instance directory to inspect, archive, or delete everything about that instance. Compare to the alternative where workspace files live in workspaces/{id}/ and logs in logs/{id}/ — that's harder to reason about.

2. Single env var override for all data paths

def get_data_root() -> Path:
    env_dir = os.environ.get("MY_TOOL_DATA_DIR")
    root = Path(env_dir) if env_dir else Path.home() / ".my-tool"
    root.mkdir(parents=True, exist_ok=True)
    return root

One env var redirects ALL instance data. This is critical for:

• Testing — point to a temp directory per test
• Docker — mount a host volume at a known path
• Multi-tenant — separate data by project

3. Per-instance file locking with defaultdict(threading.Lock)

The InstanceStore uses a per-instance lock to prevent concurrent writes to the same instance's JSON:

class InstanceStore:
    def __init__(self, root: str | Path) -> None:
        self._root = Path(root)
        # Note: locks are never pruned — one Lock (~100 bytes) per instance_id seen.
        self._locks: defaultdict[str, threading.Lock] = defaultdict(threading.Lock)

    def create_instance(self, instance_id, name, params) -> dict:
        with self._locks[instance_id]:
            ...

    def update_instance(self, instance_id, **changes) -> dict | None:
        with self._locks[instance_id]:
            data = self._read_instance(instance_id)
            ...

Why defaultdict instead of a single global lock: a global lock would serialize ALL instance writes. Per-instance locks allow concurrent writes to different instances while serializing writes to the same instance.

4. Atomic writes with tempfile.mkstemp + os.replace

Every instance mutation is written atomically:

def _write_instance(self, instance_id, data):
    path = self._instance_path(instance_id)
    path.parent.mkdir(parents=True, exist_ok=True)
    content = json.dumps(data, ensure_ascii=False, default=str)
    fd, tmp_path = tempfile.mkstemp(dir=path.parent, suffix=".tmp")
    try:
        os.write(fd, content.encode("utf-8"))
        os.close(fd)
        Path(tmp_path).replace(path)        # atomic
    except BaseException:
        with contextlib.suppress(OSError):
            os.close(fd)
        Path(tmp_path).unlink(missing_ok=True)
        raise

…