Repository navigation
Add DynamoDB module - #1613
Add DynamoDB module#1613stschulte wants to merge 11 commits into
Conversation
cristianrgreco
left a comment
There was a problem hiding this comment.
Thanks @stschulte! DynamoDB is on our list of missing modules (#1575).
- Keep
-inMemoryin the command whatever options are set. TodaywithSharedDB()on its own switches to an on-disk database. - Drop
withInMemory()(already the image default) andwithDbPath()(can't be combined with in-memory). - Set the wait strategy in the constructor.
- Each test should fail if its option never reaches the container. The in-memory and telemetry tests pass either way.
- The branch conflicts with
main. Please rebase and regenerate the lockfile withnpm install -w @testcontainers/dynamodb.
Nits: please retitle to Add DynamoDB module to match the other module PRs.
|
|
||
| // If we pass arguments we also have to pass | ||
| // -jar DynamoDBLocal.jar | ||
| return ["-jar", "DynamoDBLocal.jar", ...command]; |
There was a problem hiding this comment.
The image's default command is -jar DynamoDBLocal.jar -inMemory. Replacing it drops -inMemory, so withSharedDB() or withDisableTelemetry() on their own silently switch to an on-disk database (the startup banner shows InMemory: false). Always pass -inMemory.
| * Fore more information please read | ||
| * https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.UsageNotes.html#DynamoDBLocal.CommandLineOptions | ||
| */ | ||
| public withInMemory(): this { |
There was a problem hiding this comment.
In-memory is already the image's default, so this option changes nothing on its own, and its test passes without it. Please drop it.
| * | ||
| * @param dbPath - The path to the database file inside the container | ||
| */ | ||
| public withDbPath(dbPath: string): this { |
There was a problem hiding this comment.
-dbPath can't be combined with -inMemory: withInMemory().withDbPath(".") exits at startup with dbPath and inMemory cannot be set at the same time. The file also lives in the container's filesystem, so it only survives a restart of the same container. Neither the Go nor the .NET module has this option; please drop it.
There was a problem hiding this comment.
I now dropped withInMemory but I kept withDbPath. I agree the file is gone when the container stops though you could probably combine it with .withBindMounts to prevent this.
I don't see a lot of usecases for it myself but I am not sure how your userbase typically uses the library. I use it in integration tests and I don't have usescases where i want to restart the container but keep the data. Other containers seem to expect such a scenario though (e.g. valkey or redis both allow persistent options)
There was a problem hiding this comment.
Redis's withPersistence() bind-mounts a host directory, so the data outlives the container. Here the file stays inside the container and only survives restart(), and you don't need it yourself, so please drop it. We can add host persistence if someone asks for it.
|
|
||
| constructor(image: string) { | ||
| super(image); | ||
| this.withExposedPorts(DYNAMODB_PORT).withStartupTimeout(120_000); |
There was a problem hiding this comment.
Please set the wait strategy here so users can override it. GET / returns 400 once DynamoDB Local is serving, which is what the .NET module waits on: .withWaitStrategy(Wait.forHttp("/", DYNAMODB_PORT).forStatusCode(400)).
| * const client = new DynamoDBClient(container.clientConfig()); | ||
| * ``` | ||
| */ | ||
| public clientConfig(): DynamoDBLocalClientConfig { |
There was a problem hiding this comment.
Please drop clientConfig() and its type. LocalStack and the Go and .NET DynamoDB modules only expose the endpoint, and this ties the module's public API to the AWS SDK's config shape. The return type isn't exported from index.ts either. The connectEndpoint example already shows the full client setup.
| await teardown(); | ||
| }); | ||
|
|
||
| it("separates data by client by default", async () => { |
There was a problem hiding this comment.
This tests DynamoDB Local's own behaviour. The sharedDB test already fails if -sharedDb doesn't reach the container, so this one can go.
There was a problem hiding this comment.
Not sure what you mean. I think I can either validate the withXYZ methods by checking they resulted in the expected command (similar to https://github.com/testcontainers/testcontainers-node/blob/main/packages/modules/redis/src/redis-container.test.ts#L185-L186) or by oberserving the behaviour of the container this flag is supposed to influence. I assumed the latter is preferred and then I would argue we only know withSharedDB caused the desired effect if we know it behaves differently when we do not call withSharedDB. Or am I misunderstanding?
There was a problem hiding this comment.
Fair point, it shows the sharedDB test can fail. Happy to keep it.
| * @param tableName - The desired table name | ||
| * @returns - Three functions: teardown, addUser, getUser | ||
| */ | ||
| async function createDummyTable( |
There was a problem hiding this comment.
This can be much smaller. DynamoDB Local returns tables as ACTIVE straight away, so the waiters aren't needed, and teardown isn't either because the container is removed after each test. The JSDoc can go too.
There was a problem hiding this comment.
Is this documented somewhere? According to https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.UsageNotes.html it says:
DynamoDB (downloadable version) can perform some tasks almost instantaneously
so I prefer to include the wait unless we know for sure we don't suddenly create a race condition
There was a problem hiding this comment.
Fair, the waiter checks the table's status rather than sleeping, so keep it.
| waitUntilTableExists, | ||
| waitUntilTableNotExists, | ||
| } from "@aws-sdk/client-dynamodb"; | ||
| import { describe, it } from "vitest"; |
There was a problem hiding this comment.
Nit: describe and it are globals, like expect. The other modules don't import them.
|
|
||
| ### Connect to local endpoint | ||
|
|
||
| In order to use a DynamoDB Client we have to specify |
There was a problem hiding this comment.
The other module pages go straight from the IMAGE line to the examples. Please drop this paragraph, and the "Connect via client config" section along with clientConfig().
| <!--codeinclude--> | ||
| [](../../packages/modules/dynamodb/src/dynamodb-container.test.ts) inside_block:connectClientConfig | ||
| <!--/codeinclude--> | ||
|
|
There was a problem hiding this comment.
Nit: whitespace-only line.
|
I think I adressed your comments despite dropping |
cristianrgreco
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround @stschulte! No need to squash, as we squash-merge, but please merge or rebase main in, because the branch still conflicts.
Remaining:
- Drop
withDbPath(), and turn telemetry off by default instead ofwithDisableTelemetry()(replies inline). - Restore the startup timeout.
- The JSDoc blocks are still there.
Nits inline.
|
|
||
| constructor(image: string) { | ||
| super(image); | ||
| this.withExposedPorts(DYNAMODB_PORT).withWaitStrategy(Wait.forHttp("/", DYNAMODB_PORT).forStatusCode(400)); |
There was a problem hiding this comment.
Please keep .withStartupTimeout(120_000) from the first version. Most modules set it here too.
| return new StartedDynamoDBContainer(started); | ||
| } | ||
|
|
||
| private createCommand(): string[] | null { |
There was a problem hiding this comment.
Once withDbPath() is gone, this can collapse into start() and always set the command:
this.withCommand(["-jar", "DynamoDBLocal.jar", "-inMemory", ...(this.sharedDB ? ["-sharedDb"] : [])]);With no options that's the image's own default, so the null path, skipInMemory and the comment can go.
| return this.startedTestContainer.getMappedPort(DYNAMODB_PORT); | ||
| } | ||
|
|
||
| /*+ |
There was a problem hiding this comment.
The JSDoc blocks are still here, and this one still points at the removed clientConfig. Please drop them.
| * client configuration suitable for most usecases | ||
| */ | ||
| public getEndpoint(): string { | ||
| const url = `http://${this.getHost()}:${this.getPort().toString()}`; |
There was a problem hiding this comment.
Nit: return `http://${this.getHost()}:${this.getPort()}`; is enough.
There was a problem hiding this comment.
I normally use @typescript-eslint/restrict-template-expressions where this would report an error but I see it is not used here and a matter of taste, so I removed the toString() in 633d41c
| }); | ||
| }); | ||
|
|
||
| it("stores data in memory when no dbPath set", async () => { |
There was a problem hiding this comment.
Starting this one with .withSharedDB() would cover the command the module builds. With no options the image's default command runs, so this test still passes if -inMemory is dropped again.
There was a problem hiding this comment.
To be honest, I am not sure what you mean by this comment. We now no longer allow to influence the storage behaviour. Testing wether or not data survives a container reboot now seems like an implementation detail of the container we no longer care about, so I simply dropped the test in 633d41c. If this is not what you meant, let me know.
|
|
||
| it("stores data", async () => { | ||
| await using container = await new DynamoDBContainer(IMAGE).start(); | ||
| const client = new DynamoDBClient({ |
There was a problem hiding this comment.
Nit: the same client config is repeated in almost every test. A small createClient() helper would do for the tests the docs don't include.
Adds a new `DynamoDBContainer` to help running a local AWS DynamoDB container as described [in the docs][1] [1]: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html#docker
The project coding guidelines wants helper functions go beyond the caller, so we now move `createCommand` to the bottom.
The default arguments of `dynamodb-local` already include `-inMemory` so we do not need a helper to enable it. Disabling should be possible when specifying a database path.
We cannot really observe a change in behaviour to validate telemetry is disabled but we can at least validate we are passing the correct command to our container.
Vitest is currently configured with
```typescript
export default defineConfig({
test: {
globals: true,
}
}
```
So we do not have to import `describe`, `it`, `expect`, etc. Drop those
imports.
We now adopt the same wait strategy that is used in other languages that support DynamoDB
Most of the documentation seems to assume the application inside the behaviour is well understood so we now also drop additional explanation around the usage of the DynamoDB container.
In order to easily create a table with dummy data inside our tests we introduced a small helper function. We now cut functionality we don't really require like changing the table name or returning a function to destroy the table. Since we throw away the whole container there is not reason to delete a single table.
There a a lot of ways to construct a URL, so use what seems to be best practice inside this repository.
c320419 to
5848adc
Compare
|
I incroporated your feedback in 633d41c:
I rebased against main. |
After PR comments we now drop dbPath since there is no current usecase for it and we disable telemetry by default instead of providing it as an option.
5848adc to
633d41c
Compare
Adds a new
DynamoDBContainerto help running a local AWS DynamoDB container as described in the docs