Skip to content

README.md

Latest commit

 

History

History
137 lines (99 loc) · 7.81 KB

README.md

File metadata and controls

137 lines (99 loc) · 7.81 KB
Raw

Topcoder Forms API v6

A NestJS/TypeScript service for named website forms, immutable form versions, typed PostgreSQL submissions, and reporting. Payload CMS manages editorial definitions; the API validates and owns the published schema and submission data.

There are no JSON/JSONB database columns. Each answer references a field in one specific form version and uses a text, integer, exact decimal, boolean, date, or relational choice value. Publishing creates a SQL view with a named column for every field:

SELECT submitted_at, email, full_name, interests, receive_updates
FROM forms.event_interest_v1
ORDER BY submitted_at;

All tables and views live in the forms schema; dev uses the shared topcoder-services database. This uses normalized typed answer tables and per-version reporting views. It does not create a new physical table for every form. The data model and tradeoffs explain the distinction.

Stack

  • Node 26.12.1, pinned in .nvmrc and the Docker build.
  • NestJS 11.1.28, matching the other v6 services and the supported peer range of Nest Throttler.
  • TypeScript 5.9.3, strict checking and decorator metadata.
  • Prisma 7.10.0, generated TypeScript client and @prisma/adapter-pg.
  • PostgreSQL 17, pnpm 11.21.0.

Run locally

Run all commands from this project directory. Select Node before any package-manager command, including in a new terminal.

cd forms-api-v6
nvm install
nvm use
npm install --global pnpm@11.21.0
pnpm install --frozen-lockfile
cp .env.example .env
docker compose up --build -d

API: http://localhost:3006/v6/forms. OpenAPI UI: http://localhost:3006/v6/forms/api-docs. Readiness: http://localhost:3006/v6/forms/health/ready.

Compose runs migrations as a separate job before starting the non-root API container. Its credentials and localhost bindings are for development. FORMS_HTTP_PORT and FORMS_DATABASE_PORT change host ports.

Create/publish an example form, submit a sample, and print its report:

nvm use
pnpm demo

The demo signs a short-lived local token internally and does not print it. It requires local HS256 configuration and refuses NODE_ENV=production. Each demo run intentionally creates one new sample submission; repeat form registration/publication is idempotent.

For local source development with only PostgreSQL in Docker:

docker compose up -d postgres
nvm use
pnpm migrate:deploy
pnpm build
pnpm start:dev

Stop the Compose API first if it already occupies port 3006. pnpm start:dev uses TypeScript decorator metadata and watches imported source files. pnpm start:prod runs the compiled build.

Workflow

  1. Register a stable key such as event_interest.
  2. Save revision 1 with its named fields and types.
  3. Publish it. The API creates forms.event_interest_v1 transactionally.
  4. Add a Payload form block referencing event_interest to a content page.
  5. The website fetches the current API schema and submits the exact displayed revision with a UUID Idempotency-Key.
  6. Optionally include "kafka": true alongside the submitted answers to publish a form.submitted event through Bus API; see the event contract and retry behavior.
  7. Read private paginated JSON/CSV reports or query the SQL view with a reporting database role.

Published definitions are immutable. Changes use a new sequential revision; publication retires the previous one. An already-open older page receives a 409 and must reload. Historical reports remain available. Exact retries of accepted submissions return the original receipt, including after retirement.

Anonymous and signed-in member access are configurable per form. Member identity comes from a verified Topcoder JWT, never a submitted memberId. Machine tokens manage forms and read reports through separate scopes; they cannot submit visitor forms.

Integrations and documentation

The portable integrations live inside this service. Register them in the host Payload configuration and website renderer as documented. The dev CMS and website integrations are deployed at https://www.topcoder-dev.com/forms-test.

Verify

nvm use
pnpm lint
pnpm build
pnpm typecheck
pnpm test

Integration tests require an explicitly selected disposable database, migrated with this project's migrations. They never silently skip database checks or reset a database. They create uniquely named fixtures and leave them for inspection.

nvm use
DATABASE_URL=postgresql://forms:forms_local@127.0.0.1:5546/forms?schema=forms pnpm migrate:deploy
TEST_DATABASE_URL=postgresql://forms:forms_local@127.0.0.1:5546/forms?schema=forms pnpm test:integration

Coverage includes actual PostgreSQL writes, all field types, SQL views, CSV output, concurrent idempotency, immutable versions, JWT access, required-answer/ownership constraints, Payload synchronization over HTTP, and browser rendering/retry behavior. CI executes the same checks with PostgreSQL 17.

Forms portal endpoints, inclusive UTC date filtering, and full CSV exports are specified in docs/api.md. The dev and production /lets-talk migration, Kafka contract, seed command, and rollout order are in docs/lets-talk.md.

Shared authentication and bus client (v6 refresh)

This branch uses tc-core-library-js 4.0.0-rc.1. Use its named createJwtVerifier / createJwtAuthenticator, createM2MClient, and createBusApiClient exports for shared authentication and bus API publishing. Service-specific roles, scopes, sessions and ownership checks remain in this service. The former bus wrapper dependency and local JWT cryptography must not be reintroduced.

RSA verification requires an exact configured issuer and audience (JWT_AUDIENCE for client tokens, AUTH0_AUDIENCE for machine/API tokens, or this service's explicit issuer-policy settings). HMAC secrets must be at least 32 bytes; no default secret is supplied. Expiry is enforced. Machine identity comes from the shared verifier, not the presence of scopes. Supply real development credentials through the service's existing environment configuration before deployment.

Bus publishers use createBusApiClient(config).postEvent(event) with the existing event envelope and optional key. Configure BUSAPI_URL for v6 and the existing OAuth credentials/token provider. The client uses bounded HTTPS requests and does not retry POSTs automatically. Application retry/outbox behavior remains local.

See the core migration guide for the API mapping, compatibility requirements, and rollout scope.

Security maintenance

The runtime and one-shot migration images use Alpine 3.24's patched Node 26 package and run as UID 10001. The runtime contains only production dependencies and compiled code. Migration tasks invoke the installed Prisma CLI directly with Node; npm and pnpm are not shipped in either final image. The RDS CA bundle and NODE_EXTRA_CA_CERTS remain configured in both images.

The lockfile pins patched dependencies and an immutable shared core revision. pnpm test:dependency-compat verifies the legacy matcher adapter. Integration fixtures use real RS256 machine tokens with an isolated JWKS endpoint, so the shared verifier's issuer, signature, audience, and machine-identity restrictions remain enabled.