s3-orchestrator

Database Schema

Entity-relationship diagram of the PostgreSQL metadata store. Hover over any table for column details and usage context.

Legend

SymbolMeaning
PKPrimary key column
FKForeign key reference
||--o{One-to-many relationship
BIGSERIALAuto-incrementing surrogate key
text_pattern_opsB-tree index optimized for LIKE prefix queries

Table Summary

TablePurposePrimary Key
backend_quotasBackend registry with storage quota trackingbackend_name
object_locationsMaps objects to backends (supports replication)(object_key, backend_name)
multipart_uploadsIn-progress multipart upload stateupload_id
multipart_partsIndividual parts within a multipart upload(upload_id, part_number)
backend_usageMonthly API/bandwidth counters per backend(backend_name, period)
cleanup_queueRetry queue for failed orphan deletionsid (auto-increment)
cleanup_dlqDead-letter for cleanup_queue rows that exhausted retriesid (auto-increment)
pending_objectsIn-flight PUT intents (PUT-before-COMMIT write-path crash recovery)intent_id (UUID)
notification_outboxDurable webhook event delivery queueid (auto-increment)

Schema Migrations

MigrationDescription
00001_init_schemaAll six tables, indexes, and foreign keys
00002_multipart_metadataAdd metadata JSONB column to multipart_uploads
00003_add_encryptionAdd encryption columns to object_locations and multipart_parts
00004_add_orphan_bytesAdd orphan_bytes to backend_quotas and size_bytes to cleanup_queue
00005_add_content_hashAdd content_hash to object_locations for integrity verification
00006_add_indexes_and_tablesamplePerformance indexes on multipart_uploads(backend_name) and object_locations(object_key, created_at)
00007_notification_outboxAdd notification_outbox table for durable webhook event delivery
00008_pending_objectsAdd pending_objects table for the PUT-before-COMMIT write-path pattern
00009_cleanup_dlqAdd cleanup_dlq table so retry-exhausted cleanup rows surface for operator action
00010_multipart_upload_encryptionAdd encryption_key and key_id columns to multipart_uploads so every part of an encrypted upload shares one wrapped DEK
00011_cleanup_queue_claimAdd claimed_at and claimed_by to cleanup_queue; replace the partial index with idx_cleanup_queue_claim (next_retry, created_at) WHERE attempts < 10; supports the ClaimPendingCleanups FOR UPDATE SKIP LOCKED worker pattern that prevents cross-instance double-processing