Describe the bug
With S3_ENDPOINT pointing at a non-AWS S3-compatible store, direct browser uploads fail. This covers knowledge base documents, workspace files, profile pictures and attachments. There are two independent causes.
- Upload metadata is lost.
getS3PresignedUploadUrl (apps/sim/lib/uploads/providers/s3/client.ts) lets the AWS presigner hoist x-amz-meta-* into the query string. AWS S3 stores query-string metadata. Other S3 implementations don't, including OVHcloud Object Storage. The object lands without metadata, and headProviderObject (apps/sim/lib/uploads/upload-session/provider.ts) throws Upload object <key> is missing required provider metadata, so /documents/uploads/<id>/complete returns 500.
- CSP blocks the PUT.
connect-src (apps/sim/lib/core/security/csp.ts) only allows *.amazonaws.com and a few other fixed hosts. It never includes the S3_ENDPOINT host, so the browser refuses the presigned PUT before it is sent.
To Reproduce
- Self-host with
STORAGE_PROVIDER=s3, S3_ENDPOINT=https://s3.de.io.cloud.ovh.net, AWS_REGION=de, bucket CORS allowing PUT from the app origin. Reproduced on v0.8.41 and current main.
- Upload a document to a knowledge base.
- The browser console shows a CSP
connect-src violation for https://<bucket>.s3.de.io.cloud.ovh.net/....
- With the CSP worked around, the PUT succeeds (200), but
complete fails with "missing required provider metadata". HeadObject on the uploaded key returns Metadata: {}.
Expected behavior
Uploads work against any S3-compatible endpoint configured via S3_ENDPOINT.
Additional context
I tested against the real OVH bucket with @aws-sdk/s3-request-presigner:
| Presign mode |
PUT |
HeadObject().Metadata |
| default (metadata hoisted into query) |
200 |
{} |
unhoistableHeaders: new Set(['x-amz-meta-…']) and the same headers sent by the client |
200 |
{"uploadid":"abc123"} |
A possible fix:
- When
S3_ENDPOINT is set, presign with the metadata keys as unhoistableHeaders and return them in headers so the client sends them. They are then signed, which AWS accepts too. Bucket CORS must allow these headers (AllowedHeaders: *).
- Add the origin of the
S3_ENDPOINT bucket host (virtual-hosted or path-style) to connect-src in both build-time and runtime CSP.
Happy to open a PR.
Describe the bug
With
S3_ENDPOINTpointing at a non-AWS S3-compatible store, direct browser uploads fail. This covers knowledge base documents, workspace files, profile pictures and attachments. There are two independent causes.getS3PresignedUploadUrl(apps/sim/lib/uploads/providers/s3/client.ts) lets the AWS presigner hoistx-amz-meta-*into the query string. AWS S3 stores query-string metadata. Other S3 implementations don't, including OVHcloud Object Storage. The object lands without metadata, andheadProviderObject(apps/sim/lib/uploads/upload-session/provider.ts) throwsUpload object <key> is missing required provider metadata, so/documents/uploads/<id>/completereturns 500.connect-src(apps/sim/lib/core/security/csp.ts) only allows*.amazonaws.comand a few other fixed hosts. It never includes theS3_ENDPOINThost, so the browser refuses the presigned PUT before it is sent.To Reproduce
STORAGE_PROVIDER=s3,S3_ENDPOINT=https://s3.de.io.cloud.ovh.net,AWS_REGION=de, bucket CORS allowing PUT from the app origin. Reproduced on v0.8.41 and currentmain.connect-srcviolation forhttps://<bucket>.s3.de.io.cloud.ovh.net/....completefails with "missing required provider metadata".HeadObjecton the uploaded key returnsMetadata: {}.Expected behavior
Uploads work against any S3-compatible endpoint configured via
S3_ENDPOINT.Additional context
I tested against the real OVH bucket with
@aws-sdk/s3-request-presigner:HeadObject().Metadata{}unhoistableHeaders: new Set(['x-amz-meta-…'])and the same headers sent by the client{"uploadid":"abc123"}A possible fix:
S3_ENDPOINTis set, presign with the metadata keys asunhoistableHeadersand return them inheadersso the client sends them. They are then signed, which AWS accepts too. Bucket CORS must allow these headers (AllowedHeaders: *).S3_ENDPOINTbucket host (virtual-hosted or path-style) toconnect-srcin both build-time and runtime CSP.Happy to open a PR.