s3-orchestrator

encryption

import "github.com/afreidah/s3-orchestrator/internal/encryption"

Package encryption provides envelope encryption for object payloads. It frames plaintext into chunked AES-GCM segments addressable by byte range and supports pluggable key providers (config-embedded, file, and Vault transit).

Index

Constants

HeaderSize and related constants used by this package.

const (
    // HeaderSize is the fixed size of the encryption header.
    HeaderSize = 32

    // NonceSize is the AES-GCM nonce length.
    NonceSize = 12

    // TagSize is the AES-GCM authentication tag length.
    TagSize = 16

    // ChunkOverhead is the per-chunk overhead: nonce + tag.
    ChunkOverhead = NonceSize + TagSize
)

func PackKeyData

func PackKeyData(baseNonce, wrappedDEK []byte) []byte

PackKeyData packs the base nonce and wrapped DEK into a single byte slice for storage in the database. Format: baseNonce (12B) || wrappedDEK.

func ParseHeader

func ParseHeader(r io.Reader) (chunkSize int, baseNonce []byte, err error)

ParseHeader reads and validates the 32-byte encryption header from r. Returns the chunk size and base nonce encoded in the header.

func UnpackKeyData

func UnpackKeyData(data []byte) (baseNonce, wrappedDEK []byte, err error)

UnpackKeyData splits stored key data into the base nonce and wrapped DEK.

type ConfigKeyProvider

ConfigKeyProvider wraps DEKs using an AES-256-GCM key derived from a base64-encoded master key supplied in the configuration file or via environment variable.

type ConfigKeyProvider struct {
    // contains filtered or unexported fields
}

func NewConfigKeyProvider

func NewConfigKeyProvider(masterKeyB64, keyID string) (*ConfigKeyProvider, error)

NewConfigKeyProvider creates a provider from a base64-encoded 256-bit key. The keyID identifies this key for rotation tracking; defaults to “config-0”.

func (*ConfigKeyProvider) KeyID

func (p *ConfigKeyProvider) KeyID() string

KeyID returns the identifier for this config-based master key.

func (*ConfigKeyProvider) UnwrapDEK

func (p *ConfigKeyProvider) UnwrapDEK(_ context.Context, wrappedDEK []byte, _ string) ([]byte, error)

UnwrapDEK decrypts a wrapped DEK using the inline master key.

func (*ConfigKeyProvider) WrapDEK

func (p *ConfigKeyProvider) WrapDEK(_ context.Context, dek []byte) ([]byte, string, error)

WrapDEK encrypts the DEK with AES-256-GCM using the inline master key.

type EncryptResult

EncryptResult holds the output of an encryption operation, including the ciphertext stream and metadata to store in the database.

type EncryptResult struct {
    // Body is the ciphertext stream (header + encrypted chunks).
    Body io.Reader

    // CiphertextSize is the total size of the ciphertext output.
    CiphertextSize int64

    // WrappedDEK is the encrypted DEK to store in the database.
    WrappedDEK []byte

    // KeyID identifies which master key wrapped the DEK.
    KeyID string

    // BaseNonce is the 12-byte nonce embedded in the header, needed for
    // range-based decryption without fetching the header from the backend.
    BaseNonce []byte
    // contains filtered or unexported fields
}

func (*EncryptResult) RawDEK

func (r *EncryptResult) RawDEK() []byte

RawDEK returns the plaintext DEK so the caller can reuse it on retry via EncryptWithDEK, avoiding an extra KeyProvider round-trip. Callers must not persist or log the returned bytes.

type Encryptor

Encryptor provides encrypt and decrypt operations using envelope encryption with a pluggable KeyProvider for DEK wrapping.

Per-stream byte buffers (plaintext staging, framed ciphertext, nonce, header) come from bufPool. Reusing the same buffer set across streams keeps the encrypt + decrypt hot path allocation-free once the pool is warm; under load the orchestrator was previously dominated by these per-stream allocations (PR #885).

type Encryptor struct {
    // contains filtered or unexported fields
}

func NewEncryptor

func NewEncryptor(provider KeyProvider, chunkSize int) (*Encryptor, error)

NewEncryptor creates an Encryptor with the given key provider and chunk size. The chunk size must be positive. Config validation enforces stricter bounds (4KB-1MB, power of 2); this guard catches programming errors.

The buffer pool is seeded with chunkBuffers sized to chunkSize so every borrowing reader gets a buffer set that fits a worst-case full chunk without growth.

func (*Encryptor) ChunkSize

func (e *Encryptor) ChunkSize() int

ChunkSize returns the configured plaintext chunk size.

func (*Encryptor) CiphertextSize

func (e *Encryptor) CiphertextSize(plaintextSize int64) int64

CiphertextSize returns the total ciphertext size for a given plaintext size using this Encryptor’s chunk size.

func (*Encryptor) Decrypt

func (e *Encryptor) Decrypt(ctx context.Context, body io.Reader, wrappedDEK []byte, keyID string) (io.Reader, error)

Decrypt unwraps the DEK and returns a streaming plaintext reader for the entire encrypted object. The body must start at the beginning of the ciphertext (including the header). Buffers come from the per-Encryptor pool and are returned automatically when the reader reaches io.EOF.

func (*Encryptor) DecryptRange

func (e *Encryptor) DecryptRange(ctx context.Context, body io.Reader, wrappedDEK []byte, keyID string, rng *RangeResult, baseNonce []byte) (io.Reader, int64, error)

DecryptRange unwraps the DEK and returns a streaming plaintext reader for a range of the encrypted object. The body should contain only the ciphertext chunks identified by CiphertextRange (no header). Returns the plaintext reader and the number of plaintext bytes it will produce.

func (*Encryptor) Encrypt

func (e *Encryptor) Encrypt(ctx context.Context, body io.Reader, plaintextSize int64) (*EncryptResult, error)

Encrypt generates a random DEK, wraps it with the KeyProvider, and returns a streaming ciphertext reader along with encryption metadata. The plaintext is read from body and its MD5 digest is computed on the fly for ETag generation.

func (*Encryptor) EncryptWithDEK

func (e *Encryptor) EncryptWithDEK(body io.Reader, plaintextSize int64, dek, wrappedDEK []byte, keyID string) (*EncryptResult, error)

EncryptWithDEK encrypts using a previously wrapped DEK, skipping the KeyProvider.WrapDEK call. A fresh base nonce is generated per call, so the ciphertext is unique even though the same DEK is reused. This is safe because AES-GCM nonce uniqueness is per (key, nonce) pair - see the SAFETY INVARIANT comment in chunk.go.

Intended for write failover retries where the Vault round-trip for key wrapping has already been paid on the first attempt.

func (*Encryptor) GenerateAndWrapDEK

func (e *Encryptor) GenerateAndWrapDEK(ctx context.Context) (dek, wrappedDEK []byte, keyID string, err error)

GenerateAndWrapDEK produces a fresh 256-bit DEK and wraps it via the KeyProvider, returning the unwrapped DEK along with the wrapped form and key ID. Used by callers that need a wrapped DEK ahead of any actual encryption work - notably CreateMultipartUpload, which persists the wrapped DEK on the upload row so every subsequent UploadPart can reuse it without making its own WrapDEK round-trip.

func (*Encryptor) Provider

func (e *Encryptor) Provider() KeyProvider

Provider returns the underlying KeyProvider.

type FileKeyProvider

FileKeyProvider wraps DEKs using a raw 32-byte key read from a file on disk. Suitable for bare-metal and systemd deployments where keys are provisioned by configuration management tools.

type FileKeyProvider struct {
    // contains filtered or unexported fields
}

func NewFileKeyProvider

func NewFileKeyProvider(path, keyID string) (*FileKeyProvider, error)

NewFileKeyProvider creates a provider by reading a raw 32-byte key from the given file path. The keyID defaults to “file-0” if empty.

func (*FileKeyProvider) KeyID

func (p *FileKeyProvider) KeyID() string

KeyID returns the identifier for this file-based master key.

func (*FileKeyProvider) UnwrapDEK

func (p *FileKeyProvider) UnwrapDEK(_ context.Context, wrappedDEK []byte, _ string) ([]byte, error)

UnwrapDEK decrypts a wrapped DEK using the file-based master key.

func (*FileKeyProvider) WrapDEK

func (p *FileKeyProvider) WrapDEK(_ context.Context, dek []byte) ([]byte, string, error)

WrapDEK encrypts the DEK with AES-256-GCM using the file-based master key.

type KeyProvider

KeyProvider wraps and unwraps per-object DEKs using a master key. Each implementation corresponds to a different key source (config, file, Vault).

type KeyProvider interface {
    // WrapDEK encrypts a plaintext DEK with the master key and returns the
    // wrapped bytes along with the key identifier used for wrapping.
    WrapDEK(ctx context.Context, dek []byte) (wrappedDEK []byte, keyID string, err error)

    // UnwrapDEK decrypts a wrapped DEK using the key identified by keyID.
    UnwrapDEK(ctx context.Context, wrappedDEK []byte, keyID string) (dek []byte, err error)

    // KeyID returns the identifier for the current master key.
    KeyID() string
}

func NewKeyProviderFromConfig

func NewKeyProviderFromConfig(cfg *config.EncryptionConfig) (KeyProvider, error)

NewKeyProviderFromConfig creates the appropriate KeyProvider from the encryption configuration. Returns a MultiKeyProvider when previous keys are configured for rotation support.

type MultiKeyProvider

MultiKeyProvider wraps a primary provider with fallback providers for key rotation. WrapDEK always uses the primary key; UnwrapDEK resolves the correct provider by keyID, falling back to previous keys for objects encrypted before a rotation.

type MultiKeyProvider struct {
    // contains filtered or unexported fields
}

func NewMultiKeyProvider

func NewMultiKeyProvider(primary KeyProvider, previous []KeyProvider) *MultiKeyProvider

NewMultiKeyProvider creates a rotation-aware provider. New DEKs are wrapped with the primary key; existing DEKs are unwrapped using whichever key originally wrapped them.

func (*MultiKeyProvider) KeyID

func (p *MultiKeyProvider) KeyID() string

KeyID returns the primary key identifier.

func (*MultiKeyProvider) UnwrapDEK

func (p *MultiKeyProvider) UnwrapDEK(ctx context.Context, wrappedDEK []byte, keyID string) ([]byte, error)

UnwrapDEK decrypts a wrapped DEK by resolving the provider for the given keyID. Returns an error if the keyID matches neither the primary key nor any previous rotation key - this indicates metadata corruption or a key that was removed from previous_keys config before all objects were migrated.

func (*MultiKeyProvider) WrapDEK

func (p *MultiKeyProvider) WrapDEK(ctx context.Context, dek []byte) ([]byte, string, error)

WrapDEK encrypts a DEK using the primary (current) master key.

type RangeResult

RangeResult holds the translated ciphertext range and the slice offsets needed to extract the requested plaintext bytes after decryption.

type RangeResult struct {
    // BackendRange is the HTTP Range header value for the backend request
    // (e.g., "bytes=32-65595"). Empty string means fetch the entire object.
    BackendRange string

    // StartChunk is the zero-based index of the first chunk to decrypt.
    StartChunk uint64

    // SliceStart is the byte offset within the first decrypted chunk where
    // the requested plaintext range begins.
    SliceStart int64

    // SliceLen is the number of plaintext bytes to return from the
    // decrypted output.
    SliceLen int64
}

func CiphertextRange

func CiphertextRange(start, end int64, chunkSize int) (*RangeResult, error)

CiphertextRange translates a plaintext byte range into the corresponding ciphertext range. The start and end parameters are inclusive byte offsets into the plaintext (matching HTTP Range semantics).

type VaultKeyProvider

VaultKeyProvider wraps and unwraps DEKs via the Vault Transit encrypt and decrypt endpoints. A background goroutine renews the token before expiry (static token mode) or re-reads it from a file (Nomad workload identity mode). The provider is safe for concurrent use.

type VaultKeyProvider struct {
    // contains filtered or unexported fields
}

func NewVaultKeyProvider

func NewVaultKeyProvider(cfg *config.VaultTransitConfig) (*VaultKeyProvider, error)

NewVaultKeyProvider creates a provider backed by Vault Transit. A background goroutine manages token lifecycle: for token_file configs it re-reads the file each tick; for static tokens it calls RenewSelf. Call Close to stop the renewal goroutine.

func (*VaultKeyProvider) Close

func (p *VaultKeyProvider) Close()

Close stops the background token renewal goroutine.

func (*VaultKeyProvider) KeyID

func (p *VaultKeyProvider) KeyID() string

KeyID returns a composite identifier of the form “vault:{mount}/{key}”.

func (*VaultKeyProvider) UnwrapDEK

func (p *VaultKeyProvider) UnwrapDEK(ctx context.Context, wrappedDEK []byte, _ string) ([]byte, error)

UnwrapDEK sends a Vault Transit ciphertext blob for decryption and returns the recovered plaintext DEK.

func (*VaultKeyProvider) WrapDEK

func (p *VaultKeyProvider) WrapDEK(ctx context.Context, dek []byte) ([]byte, string, error)

WrapDEK sends the plaintext DEK to Vault Transit for encryption and returns the Vault ciphertext blob as the wrapped DEK.

Generated by gomarkdoc