s3-orchestrator

backend

import "github.com/afreidah/s3-orchestrator/internal/backend"

Package backend defines the ObjectBackend interface for S3-compatible storage providers and provides the S3Backend implementation (AWS SDK v2) with per-backend circuit breaker wrapping.

Index

Variables

ErrCopyNotSupported signals that a backend (or a decorator wrapping one) cannot perform a server-side copy. Callers should fall back to materialized copy. Returned by decorators like CircuitBreakerBackend when their wrapped backend does not implement BackendCopier.

var ErrCopyNotSupported = errors.New("backend does not support native copy")

func IsCopyPhase

func IsCopyPhase(err error, phase CopyPhase) bool

IsCopyPhase reports whether err is (or wraps) a CopyError whose Phase matches phase. Returns false on nil err or unrelated errors.

func IsNotFound

func IsNotFound(err error) bool

IsNotFound returns true if the error chain contains a typed response whose HTTPStatusCode() reports 404. A 404 from DELETE means the object is already absent on the backend, which is the desired end state - callers can treat it as idempotent success rather than as a failure worth retrying.

type BackendCopier

BackendCopier is the optional capability for backends that support a server-side CopyObject. Backends that implement this method enable the orchestrator’s same-backend fast path: instead of GET-then-PUT through the proxy, a single CopyObject call asks the backend to copy bytes internally, eliminating proxy CPU, memory, and network cost.

Callers should check ok via a type assertion and fall back to materialized copy when the backend does not implement this interface or when it returns ErrCopyNotSupported (e.g., a decorator forwarding to an underlying backend that lacks native copy).

type BackendCopier interface {
    CopyObject(ctx context.Context, srcKey, dstKey, contentType string, metadata map[string]string) (etag string, err error)
}

type CircuitBreakerBackend

CircuitBreakerBackend wraps an ObjectBackend with circuit breaker protection. All S3 operations are guarded: when the circuit is open, calls immediately return ErrBackendUnavailable without touching the real backend.

type CircuitBreakerBackend struct {
    *breaker.CircuitBreaker
    // contains filtered or unexported fields
}

func NewCircuitBreakerBackend

func NewCircuitBreakerBackend(real ObjectBackend, name string, threshold int, timeout time.Duration) *CircuitBreakerBackend

NewCircuitBreakerBackend wraps a backend with per-backend circuit breaker logic. The breaker is wired to the telemetry hook so transitions surface on the standard CircuitBreaker* metrics and the BackendCircuit* notification events.

func (*CircuitBreakerBackend) CopyObject

func (cb *CircuitBreakerBackend) CopyObject(ctx context.Context, srcKey, dstKey, contentType string, metadata map[string]string) (string, error)

CopyObject forwards a server-side copy through the circuit breaker when the wrapped backend implements BackendCopier. When it does not, returns ErrCopyNotSupported so the caller falls back to materialized copy. CopyObject failures count toward the same breaker as other operations so a misbehaving backend’s native copy path trips the breaker just like its PutObject/GetObject path.

func (*CircuitBreakerBackend) DeleteObject

func (cb *CircuitBreakerBackend) DeleteObject(ctx context.Context, key string) error

DeleteObject removes an object from the backend with circuit breaker protection.

func (*CircuitBreakerBackend) GetObject

func (cb *CircuitBreakerBackend) GetObject(ctx context.Context, key string, rangeHeader string) (*GetObjectResult, error)

GetObject retrieves an object from the backend with circuit breaker protection.

func (*CircuitBreakerBackend) HeadObject

func (cb *CircuitBreakerBackend) HeadObject(ctx context.Context, key string) (*HeadObjectResult, error)

HeadObject retrieves object metadata with circuit breaker protection.

func (*CircuitBreakerBackend) PutObject

func (cb *CircuitBreakerBackend) PutObject(ctx context.Context, key string, body io.Reader, size int64, contentType string, metadata map[string]string) (string, error)

PutObject uploads an object to the backend with circuit breaker protection.

func (*CircuitBreakerBackend) Unwrap

func (cb *CircuitBreakerBackend) Unwrap() ObjectBackend

Unwrap returns the underlying ObjectBackend. This is needed for code that type-asserts to a concrete type or to a narrow interface (e.g. the reconciler’s objectLister, which extends ObjectBackend with ListObjects).

type CopyError

CopyError tags a stream-copy failure with the phase that produced it and preserves the underlying error for errors.Is / errors.As walks. Error renders as “<phase>: <underlying>” so log output matches the historical string-prefix shape.

type CopyError struct {
    Phase CopyPhase
    Err   error
}

func (*CopyError) Error

func (e *CopyError) Error() string

Error renders the wrapped failure with the phase prefix.

func (*CopyError) Unwrap

func (e *CopyError) Unwrap() error

Unwrap exposes the wrapped error to errors.Is / errors.As callers.

type CopyPhase

CopyPhase identifies which leg of a stream copy failed.

type CopyPhase string

CopyPhase values. Read failures are recoverable by trying another source; write failures terminate the attempt because the destination rejected the data.

const (
    CopyPhaseRead  CopyPhase = "read"
    CopyPhaseWrite CopyPhase = "write"
)

type GetObjectResult

GetObjectResult holds the response from a GetObject call.

type GetObjectResult struct {
    Body         io.ReadCloser
    Size         int64
    ContentType  string
    ETag         string
    ContentRange string
    LastModified time.Time
    Metadata     map[string]string
}

type HeadObjectResult

HeadObjectResult holds the response from a HeadObject call.

type HeadObjectResult struct {
    Size         int64
    ContentType  string
    ETag         string
    LastModified time.Time
    Metadata     map[string]string
}

type ListedObject

ListedObject holds metadata for a single object returned by S3 ListObjects.

type ListedObject struct {
    Key       string
    SizeBytes int64
}

type ObjectBackend

ObjectBackend defines the interface for object storage operations.

type ObjectBackend interface {
    PutObject(ctx context.Context, key string, body io.Reader, size int64, contentType string, metadata map[string]string) (etag string, err error)
    GetObject(ctx context.Context, key string, rangeHeader string) (*GetObjectResult, error)
    HeadObject(ctx context.Context, key string) (*HeadObjectResult, error)
    DeleteObject(ctx context.Context, key string) error
}

type S3Backend

S3Backend implements ObjectBackend using AWS SDK v2.

type S3Backend struct {
    // contains filtered or unexported fields
}

func NewS3Backend

func NewS3Backend(ctx context.Context, cfg *config.BackendConfig) (*S3Backend, error)

NewS3Backend creates a new S3-compatible backend client. Uses BaseEndpoint to direct requests to the configured provider instead of AWS. Each backend gets a dedicated HTTP transport with tuned connection pool settings. ctx is used for the default-chain credential probe (IMDS, SSO, STS); the resulting provider continues to refresh in the background after Init.

func (*S3Backend) CopyObject

func (b *S3Backend) CopyObject(ctx context.Context, srcKey, dstKey, contentType string, metadata map[string]string) (string, error)

CopyObject performs a server-side copy from srcKey to dstKey within the same backend bucket. Used by the proxy’s same-backend copy fast path to avoid materializing the object through the orchestrator.

When contentType or metadata is provided the SDK sends MetadataDirective=REPLACE so the destination uses the supplied values; otherwise the directive defaults to COPY and S3 preserves the source’s content type and user metadata. The CopySource string is URL-encoded because S3 requires escaping for keys containing slashes or other reserved characters.

func (*S3Backend) DeleteObject

func (b *S3Backend) DeleteObject(ctx context.Context, key string) error

DeleteObject removes an object from the backend.

func (*S3Backend) GetObject

func (b *S3Backend) GetObject(ctx context.Context, key string, rangeHeader string) (*GetObjectResult, error)

GetObject retrieves an object from the backend. When rangeHeader is non-empty (e.g. “bytes=0-99”), it is passed through to S3 and the response includes a contentRange value (e.g. “bytes 0-99/1000”) for 206 Partial Content responses.

func (*S3Backend) HeadObject

func (b *S3Backend) HeadObject(ctx context.Context, key string) (*HeadObjectResult, error)

HeadObject retrieves object metadata without the body.

func (*S3Backend) ListObjects

func (b *S3Backend) ListObjects(ctx context.Context, prefix string, fn func([]ListedObject) error) error

ListObjects iterates all objects in the backend bucket with the given prefix, calling fn for each page of results. Uses ListObjectsV2 pagination internally; per-page metric emission happens inside nextListPage, so the outer span has no recorder.

func (*S3Backend) PutObject

func (b *S3Backend) PutObject(ctx context.Context, key string, body io.Reader, size int64, contentType string, metadata map[string]string) (string, error)

PutObject uploads an object to the backend.

Generated by gomarkdoc