Storage¶
Each Warehouse stores its data in one location, described by the Warehouse's storage profile. Lakekeeper accesses that location with the Warehouse's storage credential and gives query engines access to the tables in it. Profile and credential are set when the Warehouse is created, and Lakekeeper validates them before it saves them.
Supported Storage¶
Each storage has its own page with its configuration parameters, credentials and setup steps:
| Storage | Profile type |
Vended credentials | Remote signing | System identity | LoQE |
|---|---|---|---|---|---|
| S3: AWS, S3-compatible storage, Cloudflare R2, Alibaba Cloud OSS | s3 |
STS, where the storage offers it | Yes | AWS | Yes |
| STACKIT Object Storage | stackit |
STS on a credentials group | Yes | No | Yes |
| Azure Data Lake Storage Gen2 | adls |
SAS tokens | No | Yes | Read-only |
| OneLake (Microsoft Fabric) | onelake |
User-delegated SAS tokens | No | Yes | No |
| Google Cloud Storage | gcs |
Downscoped STS tokens | No | Yes | Yes |
A system identity is the identity the Lakekeeper process runs as, such as an instance profile or a managed identity. Warehouses that use it need no stored credential; each storage page explains how to enable it.
LoQE reads storage only with vended credentials, because DuckDB does not support remote signing.
Locations¶
Table and view locations must use the scheme of the Warehouse's storage, which most query engines expect. Lakekeeper assigns it to tables created without a location. A location the client provides must use it, or one of the alternative schemes below when allow-alternative-protocols is enabled.
| Storage | Scheme | With allow-alternative-protocols |
|---|---|---|
| S3 and STACKIT | s3:// |
S3 also accepts s3a:// and s3n:// |
| ADLS and OneLake | abfss:// |
ADLS also accepts wasbs:// |
| GCS | gs:// |
- |
Alternative protocols are meant for registering legacy Hadoop-based tables. Tables with s3a:// paths are not accessible outside the Java ecosystem.
How Clients Access Data¶
Lakekeeper gives query engines access to table data in one of two ways:
- Vended credentials: Lakekeeper issues temporary credentials and returns them with the table. It downscopes them to the table's location and ensures that no two table locations in a Warehouse overlap.
- Remote signing (S3 and STACKIT): the client sends the headers of each S3 request to Lakekeeper's sign endpoint. Lakekeeper checks that the request is allowed, signs it with its own credentials and returns the signature headers. The client then sends the request to the storage itself.
Clients choose with the X-Iceberg-Access-Delegation header, which takes the Iceberg REST values vended-credentials and remote-signing, plus Lakekeeper's client-managed:
client-managedreturns neither credentials nor signing information, so the client uses its own credentials.vended-credentialsorremote-signinguses that method if the storage profile enables it.- With both values or no header, Lakekeeper vends credentials if the profile enables it, and falls back to remote signing otherwise.
- A profile that disables both returns no credentials, whatever the header says.
If a client does not implement the method Lakekeeper offers — DuckDB, for example, does not support remote signing — it needs its own storage credentials. For S3, region, endpoint and path-style settings are still returned in the table config, but no storage-credentials entry is returned.
Disabling Credential Vending and Remote Signing¶
Each storage profile turns the methods on or off for its Warehouse. Disabled methods are never offered to clients, whatever the request headers say:
| Storage | Credential vending | Remote signing |
|---|---|---|
| S3 and STACKIT | sts-enabled |
remote-signing-enabled |
| ADLS and OneLake | sas-enabled |
- |
| GCS | sts-enabled |
- |
Remote signing applies to Generic Tables as well as Iceberg tables.
CORS¶
LoQE, the in-browser query console, reads and writes table data directly from object storage, so the bucket must return a CORS (Cross-Origin Resource Sharing) policy that allows requests from the Lakekeeper origin. This applies to S3, STACKIT, Google Cloud Storage and ADLS, which LoQE only reads; LoQE does not support OneLake. It also applies only when the Warehouse vends credentials, because LoQE cannot use remote signing. Storage validation reports a cors-origin-allowed warning when the Lakekeeper origin is not allowed.
Each storage page has the policy and how to apply it: S3, STACKIT, Google Cloud Storage, ADLS.
Storage Layout¶
The storage layout controls how namespace and table directories are structured under the Warehouse location. It is set with the storage-layout field of the storage profile; see Storage Layout.
Validating a Storage Configuration¶
Creating a Warehouse or updating its storage fails if any storage or configuration check fails. The validation endpoints run the same checks without saving anything and report each one; see Storage Validation.