Skip to content

IPFS Worker

This document explains what the Sage IPFS Worker does, why it exists, and how it fits into the broader architecture. It's a mental model for contributors, not an exhaustive API reference.

For the authoritative route list, see packages/ipfs-worker/src/router.ts. For the public API surface, see Web API Reference.

What the Worker Is

The worker is Sage's materialization + protocol-translation layer. It sits between the on-chain protocol (registries, governance, CIDs) and the surfaces that humans and agents actually use (HTTP discovery, git clone, marketplace feeds).

The worker reads canonical pointers from the protocol, fetches content from IPFS/R2, and serves it through developer-friendly interfaces. It tracks author reputation from A2A purchases. It handles authentication and billing for paid features.

Why the Worker Exists

On-chain registries store pointers (DAO → manifest CID), but pointers aren't very useful by themselves. To install a prompt library, you need to: resolve the pointer to a manifest, fetch the manifest, parse it, fetch each prompt CID, and assemble the results. Doing this per-request against chain RPC and IPFS gateways would be slow, expensive, and unreliable.

The worker pre-computes and caches this work. It materializes pointers into ready-to-serve artifacts — searchable indexes, git repositories, marketplace feeds — so that consumers get fast, reliable access without needing to understand the underlying protocol.

This is a pragmatic centralization choice. The worker is not the source of truth (CIDs are still verifiable against the chain), but it is an availability dependency. If the worker is down, discovery and installation degrade even though the canonical state is unaffected. We accept this tradeoff because the alternative — requiring every consumer to run their own materialization — would make the protocol unusable for most audiences.

How the Worker Stores State

KV (Fast Index/Cache)

KV is used for hot lookups that must be cheap during installs and search: DAO/library pointer caches, discovery indexes, and feature-specific metadata. KV is eventually consistent and optimized for reads.

Durable Objects (Stateful Materialization)

Durable Objects handle state that needs transactional guarantees: git object storage for synthesized repositories, author reputation tracking (TrendScores), credits and billing (CreditsLedger), and other materialization caches.

R2 + IPFS Gateways

The worker fetches content by CID, preferring R2 (fast object storage) when available and falling back to IPFS gateways. Paid pinning affects availability and performance, not canonicity — the CID is the truth regardless of where the content is hosted.

Key Route Families

These are conceptual groupings; exact paths and parameters evolve. Always check packages/ipfs-worker/src/router.ts for the current surface.

  • Health + diagnostics: GET /health for basic service checks
  • IPFS helpers: GET /ipfs/* for CID fetch/proxy used by the app/SDK
  • Git smart HTTP: GET/POST /git/* serves git protocol for Claude/Codex installs, backed by on-chain manifest pointers (DAO libraries) or content CIDs (immutable URLs)
  • Marketplace: GET /marketplace* for Claude Code marketplace discovery and install descriptors
  • Discovery/search: GET /discover/* for search, trending, tags
  • Reputation: GET /reputation/* for author metrics, leaderboards, and achievement badges

What Typically Changes When Adding a Feature

Most new features touch the worker in one of four ways:

  1. New or changed routes in router.ts
  2. New KV index keys or DO storage layout (plus invalidation rules)
  3. Subgraph/indexer updates if the worker needs new cached facts at read time
  4. Web app/API proxy changes if the UI consumes new endpoints

The general pattern: if you need a new fact available at request time without chain RPC, add it to the subgraph, cache it in KV via the sync cron, and read from KV in the route handler.

How This Connects