Module storage_s3 

S3-compatible object storage adapter (feature = “object-storage”) S3-compatible object storage adapter.

Implements StoragePort for AWS S3, MinIO, Cloudflare R2, DigitalOcean Spaces, and any S3-compatible endpoint.

Also implements ScrapingService so objects stored in S3 can be used as pipeline input sources.

Feature gate

Requires feature = "object-storage".

Key structure

Objects are stored under {prefix}/{pipeline_id}/{node_name}/{record_id}.json for easy browsing and lifecycle rules.

Authentication

Reads AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from the environment (or from the S3StorageConfig). For non-AWS providers set endpoint to your custom S3-compatible URL (MinIO, R2, Spaces, Backblaze B2, etc.).

Structs

S3Storage

S3-compatible object storage adapter implementing StoragePort.

S3StorageConfig

Configuration for the S3 storage adapter.