Quickstart

Get the S3 Orchestrator running locally in under a minute. The only prerequisites are Go, Docker, and Make.

Start the orchestrator

git clone https://github.com/afreidah/s3-orchestrator.git
cd s3-orchestrator
make run

This starts three MinIO instances via Docker Compose, then launches the orchestrator pointing at them. The included config.yaml uses embedded SQLite (no external database needed) and is pre-configured for this environment — no manual setup required.

What make run does

1. make dev-deps — runs docker compose -f docker-compose.test.yml up -d to start:
   • MinIO 1 on localhost:19000 (bucket: backend1)
   • MinIO 2 on localhost:19002 (bucket: backend2)
   • MinIO 3 on localhost:19004 (bucket: backend3)
   • A setup container that creates the MinIO buckets
   • PostgreSQL and Redis are also started for integration tests, but the default config.yaml uses embedded SQLite so neither is required for local development
2. go run ./cmd/s3-orchestrator -config config.yaml — compiles and starts the server on port 9000

Test it

Upload and retrieve an object using the AWS CLI:

aws --endpoint-url http://localhost:9000 \
    s3 cp /etc/hostname s3://photos/test.txt \
    --region us-east-1

aws --endpoint-url http://localhost:9000 \
    s3 cp s3://photos/test.txt - \
    --region us-east-1

The config defines three virtual buckets (photos, documents, backups) with these credentials:

Configure the AWS CLI profile to match:

aws configure --profile orchestrator
# AWS Access Key ID: photoskey
# AWS Secret Access Key: photossecret
# Default region name: us-east-1
# Default output format: json

Then use it:

alias s3o='aws --profile orchestrator --endpoint-url http://localhost:9000'
s3o s3 cp ./myfile.txt s3://photos/myfile.txt
s3o s3 ls s3://photos/

Web dashboard

The dashboard is enabled at http://localhost:9000/ui/ with credentials admin / admin.

Metrics

Prometheus metrics are available at http://localhost:9000/metrics.

Validate the config

Check the configuration file without starting the server:

go run ./cmd/s3-orchestrator validate -config config.yaml

Clean up

Stop the Docker containers and remove build artifacts:

make clean

make clean stops the containers but Docker volumes persist (MinIO data). SQLite data is stored locally in dev-data.db. To remove everything including stored data:

docker compose -f docker-compose.test.yml down -v

Troubleshooting

• make run fails immediately — is Docker running? Check with docker ps.
• Port 9000 already in use — another service is on port 9000. Stop it or change listen_addr in config.yaml.
• Container health checks failing — check backing service logs: docker compose -f docker-compose.test.yml logs
• Stale data from a previous run — clean volumes: docker compose -f docker-compose.test.yml down -v

Next steps

• Admin Guide — production deployment, backend configuration, operational procedures
• User Guide — client setup for AWS CLI, rclone, boto3, and Go SDK
• README — architecture, features, and full configuration reference