S3

Many Kubedoop products read and write object storage: Hive keeps table data there, Trino queries it, Spark checkpoints to it. Rather than repeating endpoint and credential settings in every product resource, Kubedoop models S3 as two cluster resources that products point at.

Two objects

Object	Describes
S3Connection	Where the object store is and how to reach it: host, port, TLS, region, addressing style, credentials
S3Bucket	A bucket name, plus the connection that bucket lives on

Both belong to the s3.kubedoop.dev/v1alpha1 API group.

Which one a product asks for depends on how it addresses storage. A product handed a whole bucket takes an S3Bucket; a product that decides bucket names itself — Hive Metastore, for instance — takes an S3Connection directly.

Inline or reference, never both

Wherever Kubedoop accepts S3 configuration it accepts the same pair of fields: inline to define the object in place, or reference to name an existing object in the same namespace. They are mutually exclusive, and resolution fails if you set both or neither:

invalid S3 connection: inline and reference are mutually exclusive
invalid S3 connection: neither inline nor reference is set

Use reference as soon as a second product talks to the same object store, so the endpoint is defined once.

S3Connection

apiVersion: s3.kubedoop.dev/v1alpha1
kind: S3Connection
metadata:
  name: minio
spec:
  host: minio.default.svc.cluster.local
  port: 9000
  pathStyle: true
  region: us-east-1
  credentials:
    secretClass: minio-credentials

Field	Required	Default	Notes
host	yes	—	Hostname of the object store. No scheme; the scheme follows tls
port	no	—	Omitted entirely when unset, so the endpoint uses the scheme's default port
credentials	yes	—	SecretClass supplying the access keys, see below
pathStyle	no	false	Addressing style, see below
tls	no	—	Presence switches the endpoint to https
region	no	us-east-1	Signing region

The endpoint is assembled from these fields: the scheme is https when tls is set and http otherwise, so there is no separate "use TLS" switch to keep in sync.

Credentials

credentials.secretClass names a SecretClass that must publish exactly two keys:

Key	Read by products as
ACCESS_KEY	AWS_ACCESS_KEY_ID
SECRET_KEY	AWS_SECRET_ACCESS_KEY

The Secret Operator mounts them as files under /kubedoop/secret/<volume>/, and products source them into the environment:

export AWS_ACCESS_KEY_ID="$(cat /kubedoop/secret/s3-credentials/ACCESS_KEY)"
export AWS_SECRET_ACCESS_KEY="$(cat /kubedoop/secret/s3-credentials/SECRET_KEY)"

Credentials may additionally carry a scope, which narrows what the issued credential covers — see Authentication for how SecretClass scopes work.

pathStyle: the field that breaks MinIO

pathStyle selects how the bucket is addressed:

Value	Resulting URL	Correct for
false (default)	https://<bucket>.<host> — virtual host	AWS S3
true	https://<host>/<bucket> — path	MinIO, Ceph RGW, most self-hosted backends

The default is right for AWS and wrong for most in-cluster deployments. MinIO in particular serves path-style only. With the default, the client resolves <bucket>.<host> — warehouse.minio for an in-cluster MinIO — which does not exist in DNS.

Nothing rejects this at admission. The resource applies cleanly, the pods start, and the failure only surfaces on the first object access. Set pathStyle: true for MinIO, Ceph RGW and similar backends.

TLS

Setting tls switches the endpoint to https. The nested verification decides how the server certificate is checked:

spec:
  host: s3.example.com
  tls:
    verification:
      server:
        caCert:
          secretClass: tls   # a SecretClass that issues the CA certificate

verification	Behaviour
server.caCert.secretClass	Verify against the CA published by that SecretClass
server.caCert.webPki: {}	Verify against the system's public CA bundle
none: {}	Do not verify the certificate

none disables verification entirely and should stay out of production.

S3Bucket

An S3Bucket is a bucket name plus a connection, and the connection is itself an inline-or-reference pair:

apiVersion: s3.kubedoop.dev/v1alpha1
kind: S3Bucket
metadata:
  name: warehouse
spec:
  bucketName: warehouse
  connection:
    reference: minio

bucketName is the name on the object store; the resource's own metadata.name is only what other Kubedoop resources refer to. They do not have to match.

A bucket with no connection fails resolution:

invalid S3 bucket "warehouse": no connection is set

Using it from a product

Products expose the same inline-or-reference pair. Hive Metastore takes a connection under clusterConfig.s3:

apiVersion: hive.kubedoop.dev/v1alpha1
kind: HiveMetastore
metadata:
  name: hive
spec:
  clusterConfig:
    s3:
      reference: minio

Swapping reference: minio for an inline: block with the same fields as an S3Connection spec is equivalent, and appropriate when only one product uses that endpoint.

Related

• Roles and role groups
• Resource management