Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): support custom endpoint and addressing style for S3 #6924

Merged
merged 1 commit into from
Jan 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/afraid-otters-crash.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@logto/core": patch
---

support custom endpoint and addressing style for S3
3 changes: 2 additions & 1 deletion packages/core/src/utils/storage/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,15 @@
return storage.uploadFile;
}

const { endpoint, bucket, accessKeyId, accessSecretKey, region } = config;
const { endpoint, bucket, accessKeyId, forcePathStyle, accessSecretKey, region } = config;

Check warning on line 22 in packages/core/src/utils/storage/index.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/utils/storage/index.ts#L22

Added line #L22 was not covered by tests

const storage = buildS3Storage({
endpoint,
bucket,
accessKeyId,
secretAccessKey: accessSecretKey,
region,
forcePathStyle,

Check warning on line 30 in packages/core/src/utils/storage/index.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/utils/storage/index.ts#L30

Added line #L30 was not covered by tests
});

return storage.uploadFile;
Expand Down
25 changes: 25 additions & 0 deletions packages/core/src/utils/storage/s3-storage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
secretAccessKey: string;
region?: string;
endpoint?: string;
forcePathStyle?: boolean;
};

export const buildS3Storage = ({
Expand All @@ -24,6 +25,7 @@
secretAccessKey,
region,
endpoint,
forcePathStyle,

Check warning on line 28 in packages/core/src/utils/storage/s3-storage.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/utils/storage/s3-storage.ts#L28

Added line #L28 was not covered by tests
}: BuildS3StorageParameters) => {
if (!region && !endpoint) {
throw new Error('Either region or endpoint must be provided');
Expand All @@ -35,6 +37,7 @@
const client = new S3Client({
region: finalRegion,
endpoint,
forcePathStyle,

Check warning on line 40 in packages/core/src/utils/storage/s3-storage.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/utils/storage/s3-storage.ts#L40

Added line #L40 was not covered by tests
credentials: {
accessKeyId,
secretAccessKey,
Expand All @@ -56,6 +59,28 @@
return { url: `${publicUrl}/${objectKey}` };
}

if (endpoint) {
// Custom endpoint URL construction
if (forcePathStyle) {
// Path-style URL: https://endpoint/bucket/key
return {
url: `${endpoint}/${bucket}/${objectKey}`,
};
}
// Virtual-hosted style URL: https://bucket.endpoint/key
return {
url: `${endpoint.replace(/^(https?:\/\/)/, `$1${bucket}.`)}/${objectKey}`,
};
}

// AWS S3 standard URL construction
if (forcePathStyle) {
// Path-style URL: https://s3.region.amazonaws.com/bucket/key
return {
url: `https://s3.${finalRegion}.amazonaws.com/${bucket}/${objectKey}`,
};
}
// Virtual-hosted style URL: https://bucket.s3.region.amazonaws.com/key

Check warning on line 83 in packages/core/src/utils/storage/s3-storage.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/utils/storage/s3-storage.ts#L62-L83

Added lines #L62 - L83 were not covered by tests
return {
url: `https://${bucket}.s3.${finalRegion}.amazonaws.com/${objectKey}`,
};
Expand Down
1 change: 1 addition & 0 deletions packages/schemas/src/types/system.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ export const storageProviderDataGuard = z.discriminatedUnion('provider', [
endpoint: z.string().optional(),
region: z.string().optional(),
bucket: z.string(),
forcePathStyle: z.boolean().optional(),
accessKeyId: z.string(),
accessSecretKey: z.string(),
...basicConfig,
Expand Down
Loading