diff --git a/docs/content/automation/triage_engine/about.md b/docs/content/automation/triage_engine/about.md index 242baa0aa6..d35081116c 100644 --- a/docs/content/automation/triage_engine/about.md +++ b/docs/content/automation/triage_engine/about.md @@ -121,7 +121,8 @@ Runs and deliveries are both kept for 180 days by default, then pruned. The prod ## Where to go next * [Building Rules](../building_rules/) covers the editor, triggers, scope, conditions and templates. -* [Node Reference](../node_reference/) documents all 41 nodes. +* [Webhook Receivers](../webhook_receivers/) covers inbound webhooks and two-way sync with Downstream Connectors. +* [Node Reference](../node_reference/) documents all 47 nodes. * [Runs](../runs/) covers execution, traces, cascading and limits. * [Deliveries](../deliveries/) covers channels, statuses, retries and replay. * [Converting from Rules Engine](../converting_from_rules_engine/) covers moving existing rules across. diff --git a/docs/content/automation/triage_engine/building_rules.md b/docs/content/automation/triage_engine/building_rules.md index 76e7b7881b..8dc9b7fcaf 100644 --- a/docs/content/automation/triage_engine/building_rules.md +++ b/docs/content/automation/triage_engine/building_rules.md @@ -147,6 +147,22 @@ A path that does not resolve produces no value rather than an error. An Asset rule reads its items the same way, through `product.*`, `product_type.*` and `ctx.*` paths. `ctx.changed_fields` carries the names of the fields an update changed, and the insert menu only offers paths the rule's items actually carry. +### Referring to webhook data + +A rule started by **On an Inbound Webhook** reads the delivery through `webhook.*` paths. Once **Find Findings by a Value** has found a Finding, the item also carries every `finding.*` path above, and the `webhook` block stays with it. + +``` +webhook.payload.issue.key a value in the item's object +webhook.root.webhookEvent a value in the whole payload +webhook.fields.state a named field the trigger read +webhook.headers.x-event-type a header the receiver keeps +webhook.receiver.label the receiver +ctx.receipt_id the receipt the run came from +ctx.ticket_link_id the ticket a lookup matched +``` + +The receiver's **Sample Payload** tab lists every path in its sample, and the insert menu offers them in the rule editor. See [Webhook Receivers](../webhook_receivers/). + ### Conditioning on an exception With [Risk Acceptances 2.0](/triage_findings/findings_workflows/pro__risk_acceptance/) enabled, diff --git a/docs/content/automation/triage_engine/configuration.de.md b/docs/content/automation/triage_engine/configuration.de.md index 26baffad99..41b93e2187 100644 --- a/docs/content/automation/triage_engine/configuration.de.md +++ b/docs/content/automation/triage_engine/configuration.de.md @@ -1,7 +1,7 @@ --- title: Konfiguration description: Einstellungen auf Deployment-Ebene für Triage Engine -weight: 7 +weight: 8 audience: pro aliases: - /de/automation/rules_engine_v2/configuration/ diff --git a/docs/content/automation/triage_engine/configuration.es.md b/docs/content/automation/triage_engine/configuration.es.md index dd70dcb5c1..176b0aaad8 100644 --- a/docs/content/automation/triage_engine/configuration.es.md +++ b/docs/content/automation/triage_engine/configuration.es.md @@ -1,7 +1,7 @@ --- title: Configuración description: Ajustes a nivel de despliegue para Triage Engine -weight: 7 +weight: 8 audience: pro aliases: - /es/automation/rules_engine_v2/configuration/ diff --git a/docs/content/automation/triage_engine/configuration.fr.md b/docs/content/automation/triage_engine/configuration.fr.md index 162dc13a90..c392c28f3f 100644 --- a/docs/content/automation/triage_engine/configuration.fr.md +++ b/docs/content/automation/triage_engine/configuration.fr.md @@ -1,7 +1,7 @@ --- title: Configuration description: Paramètres au niveau du déploiement pour Triage Engine -weight: 7 +weight: 8 audience: pro aliases: - /fr/automation/rules_engine_v2/configuration/ diff --git a/docs/content/automation/triage_engine/configuration.it.md b/docs/content/automation/triage_engine/configuration.it.md index a157c74ac1..d8fe9e6def 100644 --- a/docs/content/automation/triage_engine/configuration.it.md +++ b/docs/content/automation/triage_engine/configuration.it.md @@ -1,7 +1,7 @@ --- title: Configurazione description: Impostazioni a livello di deployment per Triage Engine -weight: 7 +weight: 8 audience: pro aliases: - /it/automation/rules_engine_v2/configuration/ diff --git a/docs/content/automation/triage_engine/configuration.ja.md b/docs/content/automation/triage_engine/configuration.ja.md index c99911f858..22729f77d1 100644 --- a/docs/content/automation/triage_engine/configuration.ja.md +++ b/docs/content/automation/triage_engine/configuration.ja.md @@ -1,7 +1,7 @@ --- title: 設定 description: Triage Engine のデプロイメントレベルの設定 -weight: 7 +weight: 8 audience: pro aliases: - /ja/automation/rules_engine_v2/configuration/ diff --git a/docs/content/automation/triage_engine/configuration.md b/docs/content/automation/triage_engine/configuration.md index 2b0a8584b7..e449fa3e17 100644 --- a/docs/content/automation/triage_engine/configuration.md +++ b/docs/content/automation/triage_engine/configuration.md @@ -1,7 +1,7 @@ --- title: "Configuration" description: "Deployment level settings for Triage Engine" -weight: 7 +weight: 8 audience: pro aliases: - /automation/rules_engine_v2/configuration/ @@ -131,6 +131,98 @@ A node with **One Message per Item** turned on, and **Generate a Report** with O Past this ceiling the node records a **visible skip** saying how many items it did not send about. It does not fail the run, and it does not silently stop. +## Webhook receivers + +These settings bound what a [webhook receiver](../webhook_receivers/) accepts. + +### `DD_RULES_V2_WEBHOOK_MAX_BODY_BYTES` + +**Default: 1048576 (1 MiB).** + +The largest body a receiver accepts. A larger delivery is refused and recorded as a rejected receipt. The Docker Compose bundles and the Helm chart (`webhookGateway.maxBodyBytes`) feed this one value to DefectDojo, to the gateway and to nginx's limit on receiver URLs, so change it in one place: set it in the deployment's environment (or the chart value), not on one container. + +### `DD_RULES_V2_WEBHOOK_DEDUPE_WINDOW_SECONDS` + +**Default: 86400 (one day). `0` turns it off.** + +Without the gateway, how long DefectDojo remembers a delivery so a sender's retry is recorded once. With a dedupe header on the receiver, a repeat is the same header value with the same body. Without one, two identical bodies within the window count once. With the gateway, the gateway recognizes a sender's retries (by the dedupe header together with a hash of the body) and DefectDojo records each gateway event once, so this setting is unused. See [Retries of the same event](../webhook_receivers/#retries-of-the-same-event). + +### `DD_RULES_V2_WEBHOOK_RATE_LIMIT` + +**Default: 600. `0` turns it off.** + +Without the gateway, the most deliveries one receiver accepts per minute. Past it, a delivery is answered `429` with a `Retry-After` header, and refused deliveries count too. Deliveries from the gateway skip it, because the gateway limits what it accepts itself (see [Rate limits](#rate-limits)). + +### `DD_RULES_V2_RECEIPT_RETENTION_DAYS` + +**Default: 180.** + +How many days a receipt is kept. `0` keeps receipts forever. + +### The webhook gateway + +The gateway runs as its own `webhook-gateway` service. nginx sends receiver URLs to it, and it delivers to DefectDojo over nginx's internal listener, signing each delivery so DefectDojo accepts deliveries only from it. It stores every delivery in DefectDojo's own database, in a schema of its own. + +DefectDojo itself defaults to serving receiver URLs directly (`DD_WEBHOOK_GATEWAY_MODE=direct`). The gateway is turned on explicitly where it is deployed: the Docker Compose bundles set `DD_WEBHOOK_GATEWAY_MODE=whook` and `WEBHOOK_GATEWAY_ENABLED=true`, and the Helm chart does the same when `webhookGateway.enabled` is on. The ECS task definitions run without it, in direct mode. Upgrading an existing installation to a release with the gateway is covered in [Adding the Webhook Gateway on Upgrade](/releases/pro/webhook-gateway/). + +To stop inbound webhook traffic without redeploying, turn off the **Inbound Webhooks** feature flag. See [Turning inbound webhooks off](../webhook_receivers/#turning-inbound-webhooks-off). + +| Setting | Default | Notes | +|---------|---------|-------| +| `DD_WEBHOOK_GATEWAY_MODE` | `direct` | `whook` puts the gateway in front of every receiver. `direct` has DefectDojo answer receiver URLs itself, with no durability during an outage. The Docker Compose bundles set `whook`. | +| `WEBHOOK_GATEWAY_ENABLED` | `true` in the Docker Compose bundles | On the nginx and gateway containers: whether nginx routes receiver URLs to the gateway. Off, the gateway idles. Set it together with `DD_WEBHOOK_GATEWAY_MODE`: `true` with `whook`, `false` with `direct`. | +| `DD_WEBHOOK_GATEWAY_URL` | `http://webhook-gateway:8080` | The gateway's admin address. Never routed by nginx. | +| `DD_WEBHOOK_GATEWAY_DELIVER_BASE_URL` | `https://nginx:7443` | Where the gateway delivers. It must be reachable from the gateway and must not be public. | +| `DD_WEBHOOK_GATEWAY_MAX_ATTEMPTS` | `12` | Delivery attempts before the gateway gives up on an event and keeps it as a dead letter. The wait starts at 2 seconds and triples each time, up to an hour, so twelve attempts cover about four and a half hours. | +| `DD_WEBHOOK_GATEWAY_SCHEMA` | `whook` | The schema inside DefectDojo's database that holds the gateway's tables. DefectDojo's initializer and the gateway both read it, so set it once for the whole deployment. Empty skips creating it, for a deployment that creates it itself. | +| `DD_WEBHOOK_GATEWAY_DB_ROLE` | `defectdojo_webhook_gateway` | The gateway's own database login role. Empty (not unset) has the gateway use DefectDojo's database credentials instead. | +| `DD_WEBHOOK_GATEWAY_ADMIN_TOKEN` | derived | The token DefectDojo uses to configure the gateway. | +| `DD_WEBHOOK_GATEWAY_SECRET_KEY` | derived | The key the gateway encrypts stored receiver tokens with. | +| `DD_WEBHOOK_GATEWAY_DELIVERY_SECRET` | derived | The key the gateway signs its deliveries to DefectDojo with. | +| `DD_WEBHOOK_GATEWAY_DB_PASSWORD` | derived | The password of the gateway's database role. | + +Every ten minutes, whenever a worker starts, and whenever the gateway refuses DefectDojo's admin token, DefectDojo reconciles the gateway with its receivers, so a gateway that lost its configuration recovers on its own. `manage.py reconcile_webhook_gateway` does the same on demand, and `--replay-dead-letters` also sends every enabled receiver's dead letters back to the gateway's delivery queue. + +#### Gateway secrets + +Each installation derives its own gateway secrets from `DD_SECRET_KEY`, one per purpose (the admin token, the secret key, the delivery secret and the database role's password), so there is nothing to generate or ship. A variable from the table above that is set and not empty is used instead of the derived value. In the Docker Compose bundles, DefectDojo's `init` container writes the resolved values to a volume only it and the gateway mount (`webhook_gateway_secrets`), so the gateway never receives `DD_SECRET_KEY` itself. The Helm chart derives them from `dojo.secretKey` and renders them into its Secret; see the chart's installation guide for an existing Secret. + +Because the gateway secrets follow `DD_SECRET_KEY`, they are only as private as it is. DefectDojo warns at startup (system check `pro.W002`) while the gateway is in use and `DD_SECRET_KEY` is still a value shipped in DefectDojo's deployment files. + +#### Changing the secret key or the gateway secrets + +Changing `DD_SECRET_KEY`, or any explicit gateway secret, changes what DefectDojo and the gateway have to agree on. While they disagree, the gateway answers senders with a server error when it cannot decrypt a receiver's stored secret, and DefectDojo answers the gateway's deliveries with a server error when their signature does not verify. Both are retried, so nothing is dropped, but keep the gap short: + +1. Restart DefectDojo (the web containers and every Celery worker) and the gateway together. In the Docker Compose bundles, let `init` run first: it writes the new secrets and sets the gateway role's new password. +2. DefectDojo re-registers every receiver with the gateway when a worker starts, so the gateway reseals each receiver's token with the new key. To do it at once, run `manage.py reconcile_webhook_gateway`. + +The same reconcile also runs every ten minutes and after the gateway refuses DefectDojo's admin token, so an installation that restarted out of step catches up on its own. + +#### Database role and schema + +DefectDojo's initializer creates the gateway's login role (`DD_WEBHOOK_GATEWAY_DB_ROLE`) and its schema (`DD_WEBHOOK_GATEWAY_SCHEMA`), owned by that role, inside DefectDojo's database. The role can use its own schema and nothing else of DefectDojo's. + +- Creating the role needs `CREATEROLE` on DefectDojo's database user. Without it, the gateway uses DefectDojo's credentials, confined to its schema by `search_path` only, and the initializer logs the statements a database administrator can run to give it a role of its own. +- Creating the schema needs `CREATE` on DefectDojo's database. Without it, the initializer logs the exact `CREATE SCHEMA` (or `GRANT`) statement to run, the gateway refuses to start and prints the same statement, and the receivers list shows the gateway as **Not Started**. + +The statements, with your own database, user and password: + +```sql +CREATE ROLE defectdojo_webhook_gateway LOGIN PASSWORD ''; +GRANT CONNECT ON DATABASE TO defectdojo_webhook_gateway; +CREATE SCHEMA IF NOT EXISTS whook AUTHORIZATION defectdojo_webhook_gateway; +``` + +Then set `DD_WEBHOOK_GATEWAY_DB_PASSWORD` to that password. Without a dedicated role, `CREATE SCHEMA IF NOT EXISTS whook AUTHORIZATION ;` is enough. + +#### Connection budget + +The gateway holds at most `WHOOK_DB_MAX_CONNS` connections (default 5; `webhookGateway.database.maxConnections` in the Helm chart) on DefectDojo's database server, on top of DefectDojo's own. Count them against the server's `max_connections`, or a managed database's connection limit. + +#### Rate limits + +nginx allows each sender address `DD_WEBHOOK_RECEIVER_RATE` receiver requests (default `100r/s`, burst `DD_WEBHOOK_RECEIVER_BURST`, default 1000), and the gateway accepts `WHOOK_INGEST_RATE` deliveries per second per receiver (default 100, burst `WHOOK_INGEST_BURST`, default 1000). Both answer `429` above the limit, which senders retry. + ## Related settings Some Triage Engine nodes use system-wide integration configuration rather than their own: diff --git a/docs/content/automation/triage_engine/configuration.pt-br.md b/docs/content/automation/triage_engine/configuration.pt-br.md index ceaa3e4006..b7d9950195 100644 --- a/docs/content/automation/triage_engine/configuration.pt-br.md +++ b/docs/content/automation/triage_engine/configuration.pt-br.md @@ -1,7 +1,7 @@ --- title: Configuração description: Configurações em nível de implantação para o Triage Engine -weight: 7 +weight: 8 audience: pro aliases: - /pt-br/automation/rules_engine_v2/configuration/ diff --git a/docs/content/automation/triage_engine/configuration.zh-hans.md b/docs/content/automation/triage_engine/configuration.zh-hans.md index 5303a6336b..8c5eff6418 100644 --- a/docs/content/automation/triage_engine/configuration.zh-hans.md +++ b/docs/content/automation/triage_engine/configuration.zh-hans.md @@ -1,7 +1,7 @@ --- title: 配置 description: Triage Engine 的部署层面设置 -weight: 7 +weight: 8 audience: pro aliases: - /zh-hans/automation/rules_engine_v2/configuration/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.de.md b/docs/content/automation/triage_engine/converting_from_rules_engine.de.md index f7a9cd16de..823198ad2f 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.de.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.de.md @@ -1,7 +1,7 @@ --- title: Migration von der Rules Engine description: Bestehende Rules-Engine-Regeln in Rules-Engine-2.0-Graphen überführen -weight: 6 +weight: 7 audience: pro aliases: - /de/automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.es.md b/docs/content/automation/triage_engine/converting_from_rules_engine.es.md index e3ee1dd9a0..6d32ce633a 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.es.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.es.md @@ -1,7 +1,7 @@ --- title: Migración desde Rules Engine description: Migrar reglas existentes de Rules Engine a grafos de Triage Engine -weight: 6 +weight: 7 audience: pro aliases: - /es/automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.fr.md b/docs/content/automation/triage_engine/converting_from_rules_engine.fr.md index ea95cd8ef6..4484e39dcf 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.fr.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.fr.md @@ -2,7 +2,7 @@ title: Conversion depuis Rules Engine description: Faire migrer les règles Rules Engine existantes vers des graphes Rules Engine 2.0 -weight: 6 +weight: 7 audience: pro aliases: - /fr/automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.it.md b/docs/content/automation/triage_engine/converting_from_rules_engine.it.md index d1f0d44076..171bfe215d 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.it.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.it.md @@ -2,7 +2,7 @@ title: Conversione da Rules Engine description: Spostare le regole esistenti di Rules Engine nei grafi di Rules Engine 2.0 -weight: 6 +weight: 7 audience: pro aliases: - /it/automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.ja.md b/docs/content/automation/triage_engine/converting_from_rules_engine.ja.md index c623b5088a..4415dd8643 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.ja.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.ja.md @@ -1,7 +1,7 @@ --- title: Rules Engine からの移行 description: 既存の Rules Engine のルールを Triage Engine のグラフへ移行する -weight: 6 +weight: 7 audience: pro aliases: - /ja/automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.md b/docs/content/automation/triage_engine/converting_from_rules_engine.md index f8f0971fdd..4bf5e81f0e 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.md @@ -1,7 +1,7 @@ --- title: "Converting from Rules Engine" description: "Move existing Rules Engine rules across to Triage Engine graphs" -weight: 6 +weight: 7 audience: pro aliases: - /automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.pt-br.md b/docs/content/automation/triage_engine/converting_from_rules_engine.pt-br.md index dfd7acd5be..4888df326f 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.pt-br.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.pt-br.md @@ -1,7 +1,7 @@ --- title: Migrando do Rules Engine description: Migre regras existentes do Rules Engine para grafos do Triage Engine -weight: 6 +weight: 7 audience: pro aliases: - /pt-br/automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/converting_from_rules_engine.zh-hans.md b/docs/content/automation/triage_engine/converting_from_rules_engine.zh-hans.md index 67e732a64c..05b1ffba16 100644 --- a/docs/content/automation/triage_engine/converting_from_rules_engine.zh-hans.md +++ b/docs/content/automation/triage_engine/converting_from_rules_engine.zh-hans.md @@ -1,7 +1,7 @@ --- title: 从 Rules Engine 转换 description: 将现有的 Rules Engine 规则迁移为 Triage Engine 的图 -weight: 6 +weight: 7 audience: pro aliases: - /zh-hans/automation/rules_engine_v2/converting_from_rules_engine/ diff --git a/docs/content/automation/triage_engine/deliveries.de.md b/docs/content/automation/triage_engine/deliveries.de.md index e3f7c479ec..463c278da9 100644 --- a/docs/content/automation/triage_engine/deliveries.de.md +++ b/docs/content/automation/triage_engine/deliveries.de.md @@ -2,7 +2,7 @@ title: Zustellungen description: Das Protokoll aller ausgehenden Sendungen von Regeln sowie der Funktionsweise von Wiederholungsversuchen und erneutem Senden -weight: 5 +weight: 6 audience: pro aliases: - /de/automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/deliveries.es.md b/docs/content/automation/triage_engine/deliveries.es.md index 6654999c40..394a3a2849 100644 --- a/docs/content/automation/triage_engine/deliveries.es.md +++ b/docs/content/automation/triage_engine/deliveries.es.md @@ -2,7 +2,7 @@ title: Entregas description: El registro de todo lo que las reglas envían hacia afuera, y cómo funcionan los reintentos y la repetición -weight: 5 +weight: 6 audience: pro aliases: - /es/automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/deliveries.fr.md b/docs/content/automation/triage_engine/deliveries.fr.md index 93ccfc3a49..8c5d95e639 100644 --- a/docs/content/automation/triage_engine/deliveries.fr.md +++ b/docs/content/automation/triage_engine/deliveries.fr.md @@ -2,7 +2,7 @@ title: Livraisons description: Le registre de tout ce que les règles envoient vers l'extérieur, ainsi que le fonctionnement des nouvelles tentatives et de la relecture -weight: 5 +weight: 6 audience: pro aliases: - /fr/automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/deliveries.it.md b/docs/content/automation/triage_engine/deliveries.it.md index 83a06457c9..fb5dc7b463 100644 --- a/docs/content/automation/triage_engine/deliveries.it.md +++ b/docs/content/automation/triage_engine/deliveries.it.md @@ -2,7 +2,7 @@ title: Consegne description: Il registro di tutto ciò che le regole inviano verso l'esterno, e come funzionano i nuovi tentativi e i reinvii -weight: 5 +weight: 6 audience: pro aliases: - /it/automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/deliveries.ja.md b/docs/content/automation/triage_engine/deliveries.ja.md index f97f71bd43..c1934806ca 100644 --- a/docs/content/automation/triage_engine/deliveries.ja.md +++ b/docs/content/automation/triage_engine/deliveries.ja.md @@ -1,7 +1,7 @@ --- title: 配信 description: ルールが外部へ送信するすべてを記録する台帳と、再試行および再送の仕組み -weight: 5 +weight: 6 audience: pro aliases: - /ja/automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/deliveries.md b/docs/content/automation/triage_engine/deliveries.md index 38c116b3da..f5606efea9 100644 --- a/docs/content/automation/triage_engine/deliveries.md +++ b/docs/content/automation/triage_engine/deliveries.md @@ -1,7 +1,7 @@ --- title: "Deliveries" description: "The ledger of everything rules send outward, and how retries and replay work" -weight: 5 +weight: 6 audience: pro aliases: - /automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/deliveries.pt-br.md b/docs/content/automation/triage_engine/deliveries.pt-br.md index 239a4eb7c5..defa1be381 100644 --- a/docs/content/automation/triage_engine/deliveries.pt-br.md +++ b/docs/content/automation/triage_engine/deliveries.pt-br.md @@ -2,7 +2,7 @@ title: Entregas description: O registro de tudo que as regras enviam para fora, e como funcionam as tentativas e a reprodução -weight: 5 +weight: 6 audience: pro aliases: - /pt-br/automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/deliveries.zh-hans.md b/docs/content/automation/triage_engine/deliveries.zh-hans.md index 715abadaa1..6ca1b9c5cb 100644 --- a/docs/content/automation/triage_engine/deliveries.zh-hans.md +++ b/docs/content/automation/triage_engine/deliveries.zh-hans.md @@ -1,7 +1,7 @@ --- title: 投递记录 description: 记录规则所有对外发送内容的台账，以及重试和重放机制的工作原理 -weight: 5 +weight: 6 audience: pro aliases: - /zh-hans/automation/rules_engine_v2/deliveries/ diff --git a/docs/content/automation/triage_engine/node_reference.de.md b/docs/content/automation/triage_engine/node_reference.de.md index d3edb79c6a..e9f8026742 100644 --- a/docs/content/automation/triage_engine/node_reference.de.md +++ b/docs/content/automation/triage_engine/node_reference.de.md @@ -1,7 +1,7 @@ --- title: Knotenreferenz description: Jeder Knoten, den Triage Engine mitbringt, und was er jeweils tut -weight: 3 +weight: 4 audience: pro aliases: - /de/automation/rules_engine_v2/node_reference/ diff --git a/docs/content/automation/triage_engine/node_reference.es.md b/docs/content/automation/triage_engine/node_reference.es.md index 9fee4d412f..d0218fb785 100644 --- a/docs/content/automation/triage_engine/node_reference.es.md +++ b/docs/content/automation/triage_engine/node_reference.es.md @@ -2,7 +2,7 @@ title: Referencia de nodos description: Todos los nodos con los que se distribuye Triage Engine, y qué hace cada uno -weight: 3 +weight: 4 audience: pro aliases: - /es/automation/rules_engine_v2/node_reference/ diff --git a/docs/content/automation/triage_engine/node_reference.fr.md b/docs/content/automation/triage_engine/node_reference.fr.md index 63aee99703..c189a83b14 100644 --- a/docs/content/automation/triage_engine/node_reference.fr.md +++ b/docs/content/automation/triage_engine/node_reference.fr.md @@ -1,7 +1,7 @@ --- title: Référence des nœuds description: Tous les nœuds fournis avec Triage Engine, et ce que fait chacun d'eux -weight: 3 +weight: 4 audience: pro aliases: - /fr/automation/rules_engine_v2/node_reference/ diff --git a/docs/content/automation/triage_engine/node_reference.it.md b/docs/content/automation/triage_engine/node_reference.it.md index 46704d6b16..67275aacf3 100644 --- a/docs/content/automation/triage_engine/node_reference.it.md +++ b/docs/content/automation/triage_engine/node_reference.it.md @@ -1,7 +1,7 @@ --- title: Riferimento dei nodi description: Tutti i nodi inclusi in Triage Engine, e cosa fa ciascuno -weight: 3 +weight: 4 audience: pro aliases: - /it/automation/rules_engine_v2/node_reference/ diff --git a/docs/content/automation/triage_engine/node_reference.ja.md b/docs/content/automation/triage_engine/node_reference.ja.md index 02667b3a5c..bdf86ccf2d 100644 --- a/docs/content/automation/triage_engine/node_reference.ja.md +++ b/docs/content/automation/triage_engine/node_reference.ja.md @@ -1,7 +1,7 @@ --- title: ノードリファレンス description: Triage Engineに搭載されているすべてのノードと、それぞれの機能 -weight: 3 +weight: 4 audience: pro aliases: - /ja/automation/rules_engine_v2/node_reference/ diff --git a/docs/content/automation/triage_engine/node_reference.md b/docs/content/automation/triage_engine/node_reference.md index 42d3a14bfa..af50c5c7f7 100644 --- a/docs/content/automation/triage_engine/node_reference.md +++ b/docs/content/automation/triage_engine/node_reference.md @@ -1,7 +1,7 @@ --- title: "Node Reference" description: "Every node Triage Engine ships with, and what each one does" -weight: 3 +weight: 4 audience: pro aliases: - /automation/rules_engine_v2/node_reference/ @@ -189,13 +189,40 @@ Two egress settings are built for these items. **Generate a Report** has a **Fin The shipped template **Report when a group of scans has landed** wires all of this: the group trigger, a report on `complete`, and an email naming the missing scans on `incomplete`. +### On an Inbound Webhook + +`trigger.webhook` + +Runs when a [webhook receiver](../webhook_receivers/) records a delivery. The payload never travels in the event that wakes the rule: the trigger reads it from the receipt. + +| Setting | Default | Notes | +|---------|---------|-------| +| **Webhook Receiver** | required | The receiver whose deliveries wake this rule. Only receivers the rule owner can see are offered. | +| **Items From** | empty | A dot path to the object, or list of objects, that become items, for example `issues`. Empty makes the whole payload one item. A list becomes one item per element. | +| **Fields** | empty | Named values read from each item into `webhook.fields.`. Each row takes a path, a template or a fixed value, an optional transform, an optional type (`string`, `int`, `float`, `bool`, `datetime`, `string_list`, `severity`), a default, and whether it is required. A later row can read an earlier one. | +| **Drop Items Missing a Required Field** | off | Skip an item whose required field has no value, instead of passing it on. | + +Each item carries the payload under `webhook`: + +``` +webhook.payload.* the item's object, for example webhook.payload.issue.key +webhook.root.* the whole payload, when Items From picked something inside it +webhook.fields.* the named, typed fields +webhook.headers.* the request headers the receiver keeps +webhook.receiver.* id, label, slug and kind of the receiver +ctx.receipt_id the receipt this run came from +ctx.item_index the item's position when Items From is a list +``` + +A webhook item has no Finding yet, so a **Findings** node does nothing to it until **Find Findings by a Value** has found one. A value that does not fit its type becomes empty and is counted in the node's trace; it never fails the run. + ## Logic ### If / Filter `filter.if` -Routes each item down the **true** or the **false** branch, by conditions. This is the only node with two outputs, and it is how a graph branches. +Routes each item down the **true** or the **false** branch, by conditions. It is how a graph branches. | Setting | Default | Notes | |---------|---------|-------| @@ -226,6 +253,36 @@ Keeps the first item per key and drops later ones carrying the same key. Scoped A common use is `finding.component_name`, to notify once per affected component instead of once per Finding. +### Find Findings by a Value + +`lookup.finding` + +Looks up the Findings a value names, such as a ticket key from a webhook, and passes them on. It has two outputs: **found** carries one item per Finding, and **not found** carries the items that named nothing, so a rule can alert on "this ticket is not linked to any Finding". + +| Setting | Default | Notes | +|---------|---------|-------| +| **Match By** | Downstream Connector Ticket | What the value identifies: a Downstream Connector ticket, a classic Jira issue (key or id), a Finding id, `unique_id_from_tool`, a hash code, or a tag. | +| **Value** | required | The value to look up, for example `{{webhook.fields.issue_key}}`. | +| **Fallback Value** | empty | Looked up instead when Value renders empty or finds nothing, for example `{{webhook.fields.issue_id}}`. | +| **Connection** | empty | For a Downstream Connector ticket: only tickets this connection created. Set it whenever two connections could share ticket keys, such as two Jira sites. The rule owner must be able to see the connection. | +| **Require a Connection** | off | For a Downstream Connector ticket: fail the run instead of matching every connection's tickets when no connection is set. | +| **Connector** | any | For a Downstream Connector ticket: only tickets of this connector type. | +| **Jira Site** | empty | For a classic Jira issue: the Jira base URL the event came from, for example `{{webhook.fields.site}}`. Only issues of the classic Jira instance configured for that site match, so the instance's URL must be the site's base URL. | +| **Include Finding Group Members** | on | A ticket or classic Jira issue for a Finding Group finds every Finding in the group. | +| **Limit** | `1000` | The most Findings one run may find. Items past it go to **not found**. | + +The lookup runs with the rule owner's visibility: a Finding the owner cannot see is indistinguishable from one that does not exist. A ticket is matched by its key first and by the ticket system's numeric id only when the key finds nothing, and a classic Jira issue by its numeric id first. A classic Jira issue that is an engagement epic is reported as not found, as the classic webhook ignores epics too. With **Jira Site** set and several classic Jira instances, an event whose site matches none of them is reported as not found. Found items keep their `webhook` block and gain: + +``` +ctx.lookup_via finding, or finding_group when found through a group ticket +ctx.lookup_value the value that matched +ctx.ticket_link_id the Downstream Connector ticket that matched +ctx.issue_tracker_mapping_id its issue tracker mapping +ctx.jira_issue_id the classic Jira issue that matched +ctx.jira_instance_id its classic Jira instance +ctx.lookup_reason on not found items: empty_value, no_match, engagement_epic, other_jira_site or limit_reached +``` + ## Findings These nodes change Findings. Every change is attributed back to the rule, run and node that made it, and shows up on the Finding's provenance timeline. @@ -283,6 +340,45 @@ Adds a note to the Finding. |---------|-------| | **Note** | The note text. Supports placeholders. | +### Apply the Ticket's Status + +`finding.apply_status_mapping` + +Closes, reopens, false-positives or risk accepts each Finding to match its linked ticket. The ticket's state and close reason are read through four lists: **closed states**, **open states**, **false positive reasons** and **accepted risk reasons**. Each list comes from the ticket's own issue tracker mapping (**Coming Back From** on the connector's status mapping) where it is set. For a classic Jira issue, the resolution lists come from its own classic Jira instance. Otherwise this node's lists apply. For Jira, the two state lists hold status category keys (`new`, `indeterminate`, `done`) and the reason lists hold resolution names. + +| Setting | Default | Notes | +|---------|---------|-------| +| **Ticket State** | required | The ticket's state, for example `{{webhook.fields.state}}`. | +| **Close Reason** | empty | The ticket's close reason or resolution. | +| **Ticket Last Updated** | empty | The ticket's own last-modified time, for example `{{webhook.fields.updated}}`. An event older than the last one applied to the ticket is ignored. | +| **Ticket Key** | empty | Names the ticket in a risk acceptance created from it. | +| **Accepted By** | empty | Who a risk acceptance created from the ticket names as accepting it, for example the assignee. | +| **Use the Connector's Status Mapping** | on | Read the lists from the ticket's mapping where it sets them. | +| **Closed States**, **Open States**, **False Positive Reasons**, **Accepted Risk Reasons** | empty | The lists to use when the mapping does not say. Comma separated, matched regardless of case. | +| **Close Findings** | on | Apply closures. | +| **Reopen Findings** | on | Reopen a closed Finding whose ticket is open again. | +| **Also Reopen Finding Groups** | off | A ticket cannot say which member of a group should reopen, so this is off by default. | +| **Note** | empty | Added to each Finding that changed. `{{ctx.ticket_change}}` says what changed in words, such as "Closed as a false positive" or "Reopened", and `{{ctx.ticket_status}}` is its code. | + +A close reason only ever classifies a closed state: false positive first, then accepted risk, then plain mitigation. A reopened ticket that still carries its old resolution reopens. Findings already in the target state are left alone. + +An accepted risk works the way the classic Jira webhook does: it creates a full risk acceptance where the Finding's product allows one, a simple risk acceptance where only that is allowed, and otherwise closes the Finding as mitigated. Reopening or false-positiving a risk-accepted Finding removes its risk acceptance first. A status change is only made on a Finding the rule owner may edit (and, for a risk acceptance, may accept risk on). + +### Add a Ticket Comment as a Note + +`finding.add_ticket_comment` + +Adds a comment made on the linked ticket as a note on the Finding, once. A comment DefectDojo posted to the ticket itself is recognized by its comment id, or by its text while the id is still on its way, and skipped. A comment on a Finding Group's ticket is added to every Finding in the group. + +| Setting | Notes | +|---------|-------| +| **Comment ID** | The ticket system's id for the comment, for example `{{webhook.fields.comment_id}}`. | +| **Comment Text** | The comment as the ticket system sent it. | +| **Comment Author** | The author's identities, comma separated. For a classic Jira issue, a comment whose author is the classic Jira instance's user is DefectDojo's own and is skipped. | +| **Note** | The note to add, for example `({{webhook.fields.commenter}}): {{webhook.fields.comment_body}}`. | + +Notes this node adds are never pushed back to the ticket. + ### Set Owners `finding.set_owners` @@ -328,8 +424,8 @@ so a person decides. They stay active and counted the whole time. With that feat review state to use, so they are simply left alone — never accepted, which is the point of the limit. A rule preview creates nothing, as with every other action. -Two behaviours worth knowing: a severity the rule cannot recognise counts as *over* the limit (if it -cannot be ranked it cannot be called safe), while a *limit* that cannot be recognised is ignored +Two behaviors worth knowing: a severity the rule cannot recognize counts as *over* the limit (if it +cannot be ranked it cannot be called safe), while a *limit* that cannot be recognized is ignored rather than blocking everything, because a rule that silently stops working is harder to notice than one that keeps going. @@ -389,7 +485,7 @@ Writes one of this instance's [Custom Fields](/asset_modelling/pro__custom_field The value is checked against the field's current definition when the rule is saved and again on every run, so a rule can never write a value the field's data type refuses. A text template that renders empty for a Finding leaves that Finding untouched (removal is the Clear node's job), setting a multi-select replaces the whole stored list, and Findings already holding the value are left alone. -Three behaviours worth knowing: +Three behaviors worth knowing: * **Scope is the boundary.** Like every Findings node, the write applies to every Finding the trigger produced under the rule owner's visibility. * **A custom field write is not a Finding save.** Nothing that follows a Finding save runs: no SLA recomputation, no deduplication, no re-prioritization. A custom field edited by hand on a Finding's page does not wake **On Finding Event** rules either. A write made by a rule does cascade: other rules see it as an `updated` event, and later nodes in the same run read the new value. @@ -478,7 +574,7 @@ Writes one of this instance's [Custom Fields](/asset_modelling/pro__custom_field | **Field** | Which custom field to write. One entry per Asset custom field defined on the instance. | | **Value** | Typed to the field. | -Two behaviours of its own: Assets the rule owner may not edit are counted on the node trace as `skipped_unauthorized` rather than touched, and the write lands on the custom field value rather than the Asset row itself, so nothing that follows an Asset edit runs. A custom field edited by hand does not wake **On Asset Event** rules; a write made by a rule still cascades as an `updated` event. +Two behaviors of its own: Assets the rule owner may not edit are counted on the node trace as `skipped_unauthorized` rather than touched, and the write lands on the custom field value rather than the Asset row itself, so nothing that follows an Asset edit runs. A custom field edited by hand does not wake **On Asset Event** rules; a write made by a rule still cascades as an `updated` event. ### Clear a Custom Field diff --git a/docs/content/automation/triage_engine/node_reference.pt-br.md b/docs/content/automation/triage_engine/node_reference.pt-br.md index 0d7e462926..2441ae8256 100644 --- a/docs/content/automation/triage_engine/node_reference.pt-br.md +++ b/docs/content/automation/triage_engine/node_reference.pt-br.md @@ -1,7 +1,7 @@ --- title: Referência de Nós description: Todos os nós com que o Triage Engine vem, e o que cada um faz -weight: 3 +weight: 4 audience: pro aliases: - /pt-br/automation/rules_engine_v2/node_reference/ diff --git a/docs/content/automation/triage_engine/node_reference.zh-hans.md b/docs/content/automation/triage_engine/node_reference.zh-hans.md index 43d9216913..80b990ef45 100644 --- a/docs/content/automation/triage_engine/node_reference.zh-hans.md +++ b/docs/content/automation/triage_engine/node_reference.zh-hans.md @@ -1,7 +1,7 @@ --- title: 节点参考 description: Triage Engine 内置的每一个节点及其作用 -weight: 3 +weight: 4 audience: pro aliases: - /zh-hans/automation/rules_engine_v2/node_reference/ diff --git a/docs/content/automation/triage_engine/runs.de.md b/docs/content/automation/triage_engine/runs.de.md index 720c15ab09..c855379693 100644 --- a/docs/content/automation/triage_engine/runs.de.md +++ b/docs/content/automation/triage_engine/runs.de.md @@ -2,7 +2,7 @@ title: Läufe description: Wie eine Regel ausgeführt wird, was ein Lauf aufzeichnet und wie die Kaskadierung begrenzt wird -weight: 4 +weight: 5 audience: pro aliases: - /de/automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/runs.es.md b/docs/content/automation/triage_engine/runs.es.md index 81fd82b3f9..84316fbf46 100644 --- a/docs/content/automation/triage_engine/runs.es.md +++ b/docs/content/automation/triage_engine/runs.es.md @@ -2,7 +2,7 @@ title: Ejecuciones description: Cómo se ejecuta una regla, qué registra una ejecución y cómo se limita el encadenamiento -weight: 4 +weight: 5 audience: pro aliases: - /es/automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/runs.fr.md b/docs/content/automation/triage_engine/runs.fr.md index 94fb73d250..d5b9411891 100644 --- a/docs/content/automation/triage_engine/runs.fr.md +++ b/docs/content/automation/triage_engine/runs.fr.md @@ -2,7 +2,7 @@ title: Exécutions description: Comment une règle s'exécute, ce qu'une exécution enregistre, et comment l'enchaînement est limité -weight: 4 +weight: 5 audience: pro aliases: - /fr/automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/runs.it.md b/docs/content/automation/triage_engine/runs.it.md index 3f8585d018..6a69c3f5df 100644 --- a/docs/content/automation/triage_engine/runs.it.md +++ b/docs/content/automation/triage_engine/runs.it.md @@ -2,7 +2,7 @@ title: Esecuzioni description: Come viene eseguita una regola, cosa registra un'esecuzione e come viene limitata la propagazione a cascata -weight: 4 +weight: 5 audience: pro aliases: - /it/automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/runs.ja.md b/docs/content/automation/triage_engine/runs.ja.md index b87a89d38e..0cce29b4be 100644 --- a/docs/content/automation/triage_engine/runs.ja.md +++ b/docs/content/automation/triage_engine/runs.ja.md @@ -1,7 +1,7 @@ --- title: 実行 description: ルールがどのように実行されるか、実行が何を記録するか、カスケードがどのように制限されるか -weight: 4 +weight: 5 audience: pro aliases: - /ja/automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/runs.md b/docs/content/automation/triage_engine/runs.md index a78eac51c2..ede9879694 100644 --- a/docs/content/automation/triage_engine/runs.md +++ b/docs/content/automation/triage_engine/runs.md @@ -1,7 +1,7 @@ --- title: "Runs" description: "How a rule executes, what a run records, and how cascading is bounded" -weight: 4 +weight: 5 audience: pro aliases: - /automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/runs.pt-br.md b/docs/content/automation/triage_engine/runs.pt-br.md index dda663e5ba..d9fdafee87 100644 --- a/docs/content/automation/triage_engine/runs.pt-br.md +++ b/docs/content/automation/triage_engine/runs.pt-br.md @@ -2,7 +2,7 @@ title: Execuções description: Como uma regra é executada, o que uma execução registra e como o encadeamento é limitado -weight: 4 +weight: 5 audience: pro aliases: - /pt-br/automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/runs.zh-hans.md b/docs/content/automation/triage_engine/runs.zh-hans.md index 47c2f00f66..10e6e37e12 100644 --- a/docs/content/automation/triage_engine/runs.zh-hans.md +++ b/docs/content/automation/triage_engine/runs.zh-hans.md @@ -1,7 +1,7 @@ --- title: 运行 description: 规则如何执行、一次运行会记录哪些内容，以及级联如何被限制 -weight: 4 +weight: 5 audience: pro aliases: - /zh-hans/automation/rules_engine_v2/runs/ diff --git a/docs/content/automation/triage_engine/webhook_receivers.md b/docs/content/automation/triage_engine/webhook_receivers.md new file mode 100644 index 0000000000..d0cc2f3bc9 --- /dev/null +++ b/docs/content/automation/triage_engine/webhook_receivers.md @@ -0,0 +1,152 @@ +--- +title: "Webhook Receivers" +description: "Let other tools talk back to DefectDojo: receive webhooks, find the Findings they are about, and act on them" +weight: 3 +audience: pro +--- +Note: Triage Engine is a DefectDojo Pro-only feature. + +A **Webhook Receiver** gives the Triage Engine an inbound URL. Any tool that can send a webhook (Jira, a ticketing system, a CI pipeline, an internal service) posts to it, and the rules that listen to the receiver decide what happens next: close a Finding, reopen it, add a note, raise an alert. + +Receivers live under **Triage Engine > Webhook Receivers**. There are two kinds: + +- **Supported webhooks**, such as Jira. You enter a secret and pick a few behaviors, and DefectDojo builds and maintains the rule for you. +- **Custom webhooks**, for anything else. You paste an example payload and build the rule yourself with the same nodes a supported webhook uses. + +## How a delivery is handled + +1. The sender posts to the receiver's URL, `https:///api/webhooks/in///`. +2. Where the **webhook gateway** is deployed, it checks the token, stores the delivery and answers the sender immediately. If DefectDojo is restarting or busy, the gateway delivers it once DefectDojo is back, retrying for about four and a half hours. Without the gateway, DefectDojo answers the sender itself. +3. DefectDojo checks the delivery (its token or the gateway's signature and, where the sender signs, the sender's signature), records it as a **receipt**, and hands it to every enabled rule that listens to this receiver. +4. Each rule runs as usual. **Runs** shows what each one did. + +Nothing in the payload travels through the engine except as data. A payload cannot run code, reach another receiver's data, or touch a Finding the rule's owner cannot see. + +## Turning on two-way sync for a Downstream Connector + +For a connector that supports it (Jira today), the fastest route is the connection itself: + +1. Open **Connect > Downstream**, then the Jira connection. +2. Choose **Turn On Two-way Sync**. This creates a Jira receiver already bound to this connection. +3. Enter the webhook secret you will give Jira, review the behaviors, and save. The receiver and its rule start **disabled**. +4. Follow the **Setup** tab: in Jira, open **System > WebHooks**, create a webhook with the receiver's URL and secret, and subscribe it to **Issue updated** and **Comment created**. Limit its JQL to the projects your connector pushes to. A Jira Data Center version that offers no webhook secret cannot sign, so switch the receiver to **URL Token Only** for it. +5. Enable the receiver. From then on, the **Receipts** tab shows each delivery and the **Rules** tab links to the rule that acts on them. + +A connection has at most one two-way sync receiver. When someone else already turned it on, the connection's **Two-way Sync** card says **Managed by another user** instead of offering to create a second one. + +Binding the receiver to its connection matters when you have more than one Jira site: ticket keys such as `SEC-101` are only unique within one site, so a bound receiver only ever matches the tickets its own connection created. If the connection is deleted, DefectDojo switches its receiver off and says why on the receiver; choose a new connection and turn it back on. + +### What Jira changes do + +| In Jira | On the linked Finding | +|---------|-----------------------| +| The issue moves to a **Done** status category | The Finding closes. Its resolution decides how: mitigated by default, a false positive or an accepted risk when the resolution is in that list. | +| The issue leaves the Done category | A closed Finding reopens. Findings in a Finding Group stay closed unless you turn on **Also Reopen Finding Groups**, because Jira cannot say which member should reopen. | +| Somebody comments | The comment is added to the Finding as a note, once. Comments DefectDojo posted itself are recognized and skipped. | + +Closure follows the issue's **status category**, never the status name or the resolution alone. Workflows that leave a resolution on a reopened issue, or set a default resolution on new ones, therefore behave correctly. A change is only made on a Finding the receiver's owner may edit, and an event older than the last one applied to the issue is ignored, so a late retry cannot undo a newer change. + +The lists live on the connector's **status mapping**, under **Coming Back From Jira**, so each Jira project can say what its own statuses and resolutions mean: + +- **Closing Status Categories** and **Reopening Status Categories** take Jira's status category keys: `new` (To Do), `indeterminate` (In Progress) and `done`. By default `done` closes, and `new` and `indeterminate` reopen. +- **False Positive Resolutions** and **Accepted Risk Resolutions** take Jira resolution names. + +A list left empty uses the receiver's default. For an issue linked through the classic Jira integration, the resolutions come from the classic Jira instance configured for the issue's own site, so give each classic instance the base URL of its Jira site (for example `https://your-organization.atlassian.net`). With exactly one classic instance, its resolution mappings are also the receiver's defaults for Downstream Connector tickets. With several, no instance's resolutions are imposed on another site's tickets: set them on each mapping. + +### Sending notes to Jira + +On the same connector, **Push Notes as Comments** on an issue tracker mapping posts new notes on a Finding to its Jira issue as comments, made by the connection's account. Nobody on your team needs Jira write access for it. + +- Every public note attached to a linked Finding is posted, whichever way it was added: the API (`POST /api/v2/findings/{id}/notes/`), the classic UI, the Pro UI, a bulk edit, or closing a Finding with a note. +- **Private notes are never posted.** Mark a note private to keep an internal discussion out of Jira. +- The comment reads `(Author name): note text`, so the Jira audience can see who wrote it. The note is sent in full, up to Jira's comment length limit (a longer one is cut and ends with `[truncated]`). +- Markdown in the note is converted to Jira formatting. Anything that would ping a Jira user, embed an image or open a macro is escaped and shows as plain text. +- A note on a grouped Finding goes to the Finding Group's issue. A bulk action that adds the same note to many members of one group posts it there once: the same text is not posted to the same group issue twice within ten minutes. +- Notes created by Triage Engine rules, including the notes two-way sync adds from Jira comments, are never sent back. +- Editing or deleting a note does not change the Jira comment. + +Posting comments needs a go-integrators version that provides it. When DefectDojo is upgraded before go-integrators, or go-integrators is rolled back, each mapping records one integration error saying so (rather than one per note), and posting is tried again an hour later. Upgrade go-integrators, or turn off **Push Notes as Comments** on the mapping. + +## Building a rule for a custom webhook + +1. **New Webhook Receiver**, then **Custom Webhook**. Give it a label and choose how the sender authenticates. +2. Save. The **Setup** tab shows the URL to give the sender. +3. On **Sample Payload**, paste an example from the sender's documentation, or send one for real. **Parse** lists every path in it, such as `webhook.payload.issue.key`, with an example value and a copy button. +4. **Create a Rule** opens the editor with an **On an Inbound Webhook** trigger for this receiver. Add **Find Findings by a Value** to turn a payload value into Findings, then any Findings or Egress nodes. +5. **Preview** runs the rule against the sample, or against a payload you paste into **Test With a Payload**, and changes nothing. A webhook rule has no manual **Run**: it runs when its receiver records a delivery. + +See the [Node Reference](../node_reference/) for **On an Inbound Webhook**, **Find Findings by a Value**, **Apply the Ticket's Status** and **Add a Ticket Comment as a Note**. + +## Authentication + +Every receiver URL carries a random 256-bit token. A delivery whose token does not match is answered `404`, exactly like a URL that does not exist, so nobody can learn which receivers exist by guessing. With the webhook gateway in front, the gateway checks the token and stores nothing for a mismatch. On top of the token, a receiver can require: + +| Mode | The sender proves itself by | +|------|-----------------------------| +| **URL Token Only** | The token alone. For senders that cannot sign, such as Jira Data Center versions without a webhook secret. | +| **Shared Secret Header** | Sending a fixed secret in a header you name. | +| **HMAC-SHA256 Signature** | Signing the body with a shared secret, in a header you name (Jira Cloud uses `X-Hub-Signature` with the prefix `sha256=`). | +| **HTTP Basic** | A username and password. | +| **Vendor Scheme** | The supported webhook's own verification, where it has one. | + +**Rotate Token** issues a new URL; the old one stops working at once. Deliveries the gateway already captured under the old token are still delivered. Secrets are encrypted at rest and never shown again after you save them. + +Deleting a receiver retires its URL right away: from then on it answers like any unknown URL. A deleted receiver's URL name is never given to a new receiver, so a new receiver with the same label never sees the old one's deliveries. + +## Receipts + +A receipt is written for every delivery to a known receiver, including the ones that were refused, so "the sender says it sent it and nothing happened" is always answerable. + +| Status | Meaning | +|--------|---------| +| **Received** | Recorded, and about to be handed to the rules. | +| **Dispatched** | Accepted, and at least one enabled rule was woken. | +| **No Listeners** | Accepted, but no enabled rule listens to this receiver. | +| **Dispatch Failed** | Recorded, but DefectDojo's task queue refused it (for example during a broker outage). The sender is told to retry, and DefectDojo also tries again every 15 minutes, a limited number of times. | +| **Rejected** | Refused. The reason says why: authentication failed, the body was too large, not JSON, or an unsupported content type. Only the size and a digest of a refused body are kept. | +| **Replayed** | Sent into the rules again by a person, with **Replay**. | + +**Replay** sends an accepted receipt's payload into the rules again, as a new receipt. It is refused while the receiver is off, and a rejected receipt cannot be replayed. + +A receipt keeps the body and the headers you choose to keep. The receiver token, `Authorization`, cookies and the receiver's own secret header are never stored. Receipts are deleted after 180 days by default; the receipts page says when each one goes. + +### Retries of the same event + +Senders retry, and a retry should not act twice. How a repeat is recognized depends on how receiver URLs are served: + +- **With the gateway**, the gateway recognizes a sender's retry by the receiver's **dedupe header** (for Jira, `X-Atlassian-Webhook-Identifier`) together with a hash of the body, and stores it once. Without a dedupe header on the receiver, a sender's retry is processed again. DefectDojo records each gateway event once, so the gateway's own retries and replays never act twice. +- **Without the gateway**, DefectDojo remembers deliveries for a day (`DD_RULES_V2_WEBHOOK_DEDUPE_WINDOW_SECONDS`). With a dedupe header on the receiver, a repeat is the same header value with the same body, and a delivery without the header is never treated as a repeat. Without a dedupe header, two identical bodies within the window count once. Set the window to `0` for a sender whose legitimate repeats are byte-identical. + +## The webhook gateway + +The gateway is what makes a delivery durable before DefectDojo has seen it. Many senders, Jira Data Center among them, do not resend a webhook that failed, and Jira Cloud retries for about half an hour at most. With the gateway in front, a DefectDojo restart or a busy moment costs nothing. + +The receiver's **Gateway** tab shows what the gateway holds for it: recent deliveries, where each one is in its retries, and any the gateway gave up on (**Dead Letters**), with **Replay** for those. Turning a receiver off pauses delivery of new events; events that arrive while it is off can be replayed from this tab once it is on again. + +The gateway retries a delivery DefectDojo could not take for about four and a half hours, then keeps it as a dead letter. DefectDojo replays dead letters by itself, a bounded number per receiver, once DefectDojo or the gateway has recovered from an outage, when the **Inbound Webhooks** flag is turned back on, and when a worker starts. **Replay** on the **Gateway** tab sends one again by hand. + +The gateway runs in the Docker Compose bundles, and in the Helm chart when `webhookGateway.enabled` is on. Elsewhere, including the ECS task definitions, DefectDojo answers receiver URLs itself, and a delivery during an outage is lost unless the sender retries. Operators configure it with the settings in [Configuration](../configuration/#webhook-receivers). The receivers list shows the gateway's state at the top: **Healthy**, **Unreachable**, **Not Started** when its database schema is missing (an administrator has to create it, see [Configuration](../configuration/#database-role-and-schema)), or **Turned Off**. + +### Turning inbound webhooks off + +The **Inbound Webhooks** feature flag, under **Settings > Feature Flags**, is on by default. Turn it off to stop inbound webhook traffic at once, for example while you investigate a misbehaving sender: + +- DefectDojo refuses every delivery with `503` and records nothing. Without the gateway the sender gets that answer, with a `Retry-After` header. With the gateway, the gateway keeps capturing deliveries and retrying them. +- DefectDojo stops talking to the webhook gateway. Receivers saved in the meantime wait to register. +- The **Receipts** and **Gateway** tabs show a warning that inbound webhooks are off, and the receivers list shows the gateway as **Turned Off**. + +Nothing already captured is lost. The gateway keeps each delivery and retries it for about four and a half hours, and senders that retry will try again. When you turn the flag back on, DefectDojo registers any waiting receivers and replays the dead letters that piled up meanwhile. + +Turning the Triage Engine off refuses deliveries the same way, whatever this flag says. + +## Permissions + +Webhook receivers use the Triage Engine permissions. Viewing needs **Rule View**, creating needs **Rule Add**, changing, rotating, replaying or syncing needs **Rule Edit**, and deleting needs **Rule Delete**. + +- A receiver belongs to the person who created it. Others do not see it, except superusers, who see every receiver. +- A rule can only listen to a receiver its owner can see. +- Labels are unique among one person's receivers, so two people can each have a receiver called "Jira". +- Binding any receiver, custom or supported, to a Downstream Connector connection needs permission to view that connection. +- A connection has at most one two-way sync receiver. + +A rule a receiver generates is **managed**: the editor shows it read-only, because the receiver rebuilds it whenever its settings change. **Detach and Edit** turns it into an ordinary rule that starts from the working graph. The receiver then leaves the detached rule alone, even when its settings change, until you choose **Regenerate** on the receiver. diff --git a/docs/content/connectors/downstream/PRO__jira_guide.md b/docs/content/connectors/downstream/PRO__jira_guide.md index 0528825772..7a6d8eaebb 100644 --- a/docs/content/connectors/downstream/PRO__jira_guide.md +++ b/docs/content/connectors/downstream/PRO__jira_guide.md @@ -43,6 +43,7 @@ The one exception is **Engagement epics**. The Downstream Connector has no conce * **Severity mappings** and **status mappings** (your open and close transition keys) are carried across. * Each **Jira Project** configuration becomes an issue tracker mapping, keeping its project key and issue type, and stays assigned to the same Asset or Engagement. * **Push All Issues** is preserved: projects that had it enabled keep pushing automatically. +* **Push Notes** becomes **Push Notes as Comments** on the mapping. Private notes are still never posted. * **Custom fields**, **close/reopen transition fields**, **component**, **default assignee**, and **labels** are converted to field mappings. Where you used *Add Vulnerability Id as a Jira label*, that becomes a label mapping too. * A **custom issue template** directory becomes a ticket template. The stock templates are not copied, because the connector already ships equivalents. @@ -50,9 +51,9 @@ The one exception is **Engagement epics**. The Downstream Connector has no conce These are reported as warnings on the migration run — they do not stop it. Look for the *"things the connector cannot carry over"* list in the results. -* **Jira → DefectDojo reverse sync.** This is the important one. The Downstream Connector does not sync changes *back* from Jira, so resolution mappings that apply Risk Acceptance or False Positive from a Jira resolution are not migrated. **If you rely on reverse sync, leave the classic Jira instance configured** — the migration does not remove it. +* **Jira → DefectDojo reverse sync is not switched on for you.** The connector syncs back from Jira through [two-way sync](/connectors/toolreference/jira/#two-way-sync), which needs a new webhook in Jira pointing at a Triage Engine receiver, so the migration cannot turn it on. Each migrated connection carries a warning saying so. Your classic resolution mappings are carried over to the connector's status mapping, under **Coming Back From Jira**, so two-way sync treats a resolution the way classic Jira did. **Until you turn two-way sync on, leave the classic Jira webhook in place** if you rely on it: the migration does not remove it. * **Engagement Epic Mapping** — the connector has no epic concept. -* **Push Notes**, **SLA notification comments**, and **risk acceptance expiration comments** — the connector does not post these to Jira. +* **SLA notification comments** and **risk acceptance expiration comments**: the connector does not post these to Jira. * Custom fields named `summary`, `description`, `project`, `issuetype` or `status` — these are reserved by the connector, and a field mapping using one is skipped. * Custom field values longer than 512 characters — skipped rather than truncated. * A Jira Project attached to neither an Asset nor an Engagement produces no assignment. diff --git a/docs/content/connectors/downstream/troubleshooting_jira.md b/docs/content/connectors/downstream/troubleshooting_jira.md index 40c4657e2c..15ff2073de 100644 --- a/docs/content/connectors/downstream/troubleshooting_jira.md +++ b/docs/content/connectors/downstream/troubleshooting_jira.md @@ -123,6 +123,49 @@ This error message can appear when attempting to add a created Jira configuratio * If comments aren't appearing, check **loop prevention**: DefectDojo skips a comment when its author matches the Jira account DefectDojo uses to post comments. Run the Automation rule as a different Jira user if you want those comments ingested. * Use Automation's payload preview to confirm the smart values resolve as expected — their names can vary between Jira instances. +## Two-way sync: changes from Jira are not reaching Findings + +This section is about [two-way sync](/connectors/toolreference/jira/#two-way-sync) for the Jira Downstream Connector, where Jira posts to a Triage Engine [webhook receiver](/automation/triage_engine/webhook_receivers/). The receiver keeps a **receipt** for every delivery it gets, so start from its **Receipts** tab (**Triage Engine > Webhook Receivers**, then the receiver). + +**There is no receipt at all.** The delivery never reached the receiver: + +* Check that the receiver is enabled, and that the **Inbound Webhooks** feature flag and the Triage Engine are on. While either is off, DefectDojo refuses every delivery with `503` and records nothing. +* Compare the URL in Jira's webhook with the one on the receiver's **Setup** tab, character for character. A URL with a wrong or old token (for example after **Rotate Token**) is answered `404`, exactly like a URL that does not exist, and nothing is recorded. With the webhook gateway in front, that `404` comes from the gateway. +* Jira Cloud only delivers to a URL with a certificate from a globally trusted authority. +* A sender that exceeds the rate limit on receiver URLs is answered `429` and retries later. + +**The receipt is Rejected.** The reason says why: + +* **Authentication Failed**: the signature did not match. The secret in Jira's webhook must be the secret saved on the receiver. A Jira Data Center version that offers no webhook secret cannot sign: switch the receiver to **URL Token Only**. +* **Body Too Large**: raise `DD_RULES_V2_WEBHOOK_MAX_BODY_BYTES` for the whole deployment (see [Configuration](/automation/triage_engine/configuration/#dd_rules_v2_webhook_max_body_bytes)). +* **Unsupported Content Type** or **Invalid JSON**: the sender must post JSON with `Content-Type: application/json`, which Jira's own webhooks do. + +**The receipt says No Listeners.** No enabled rule listens to the receiver. Open its **Rules** tab and enable the rule. If the rule was detached, it is an ordinary rule now: enable it in the rule editor, or choose **Regenerate** on the receiver. + +**The receipt says Dispatch Failed.** DefectDojo recorded the delivery but its task queue refused it, usually during a broker outage. DefectDojo tries again every 15 minutes, a limited number of times, and the sender is told to retry. **Replay** on the receipt sends it into the rules again. + +**The receipt says Dispatched, but the Finding did not change.** Open the rule's run in **Runs** and look at its trace: + +* **Find Findings by a Value** sent the item to **not found**. `no_match` means the issue is not linked to a Finding through this receiver's connection. `other_jira_site` means the issue is linked through the classic Jira integration, but no classic Jira instance's URL matches the Jira site the event came from: set the instance's URL to the site's base URL. +* The Finding is one the receiver's owner cannot edit. Changes are made as the owner. +* The issue's status category is in neither **Closing Status Categories** nor **Reopening Status Categories** on the connector's status mapping, or a grouped Finding would have to reopen while **Also Reopen Finding Groups** is off. +* The event is older than one already applied to the issue, so it was ignored. +* For comments: **Add Jira Comments as Notes** is off, the comment is one DefectDojo posted itself, or its author is in **Also Ignore Comments From**. + +**Checking the webhook gateway.** Where the gateway is deployed, the top of the receivers list shows its state: + +* **Unreachable**: DefectDojo cannot reach the gateway. Check that the `webhook-gateway` container is running and healthy. +* **Not Started**: the gateway's database schema is missing, because DefectDojo's database user may not create it. A database administrator has to create it; the statements are in [Configuration](/automation/triage_engine/configuration/#database-role-and-schema). +* **Turned Off**: the **Inbound Webhooks** feature flag is off. + +A receiver listed as not registered with the gateway yet can be registered from its **Gateway** tab with **Sync Now**, or for every receiver at once with `manage.py reconcile_webhook_gateway`. DefectDojo also does this every ten minutes and whenever a worker starts. + +The receiver's **Gateway** tab shows every delivery the gateway captured for it and where each one is in its retries. A delivery DefectDojo could not take is retried for about four and a half hours, then listed under **Dead Letters**. DefectDojo replays dead letters by itself once it or the gateway recovers, and **Replay** sends one again by hand. Deliveries that arrived while the receiver was off are never delivered on their own: replay them from this tab once it is on again. + +If every delivery keeps failing and DefectDojo's log says a gateway delivery "carried a signature that does not verify", DefectDojo and the gateway are running with different secrets, usually after `DD_SECRET_KEY` changed and only one side restarted. Restart DefectDojo, its workers and the gateway together (see [Changing the secret key or the gateway secrets](/automation/triage_engine/configuration/#changing-the-secret-key-or-the-gateway-secrets)). The deliveries are retried meanwhile, so nothing is lost if this is fixed within the retry window. + +**Notes are not reaching Jira.** Check that **Push Notes as Comments** is on for the issue tracker mapping, that the note is not private, and that the Finding already has a Jira issue. A mapping that cannot post records an integration error saying why, for example a `403` from Jira naming a missing scope, or a go-integrators version that cannot post comments yet. + ## Jira Epics aren't being created `"Field 'customfield_xyz' cannot be set. It is not on the appropriate screen, or unknown."` diff --git a/docs/content/connectors/toolreference/jira.md b/docs/content/connectors/toolreference/jira.md index c3e2dac49c..141ec589d3 100644 --- a/docs/content/connectors/toolreference/jira.md +++ b/docs/content/connectors/toolreference/jira.md @@ -4,7 +4,7 @@ description: "How to set up the Jira Downstream Connector for DefectDojo" weight: 82 audience: pro --- -The Jira integration pushes DefectDojo Findings and Finding Groups to a Jira project as issues, keeps each issue's status in sync with the Finding, and links the Finding back to the created issue. Both Jira **Cloud** and **Data Center / Server** are supported. Jira Service Management is not supported. +The Jira integration pushes DefectDojo Findings and Finding Groups to a Jira project as issues, keeps each issue's status in sync with the Finding, and links the Finding back to the created issue. With [two-way sync](#two-way-sync), Jira status changes and comments come back to the Finding too. Both Jira **Cloud** and **Data Center / Server** are supported. Jira Service Management is not supported. ### Choosing an authentication method @@ -96,6 +96,19 @@ Statuses vary per project workflow, so these defaults are meant to be edited to - **False Positive Mapping**: `Done` - **Risk Accepted Mapping**: `Done` +#### Coming back from Jira + +When [two-way sync](#two-way-sync) is on, the status mapping also says what a Jira change means for the Finding. The status lists take Jira's status **category** keys, never status names, and the resolution lists are matched regardless of case: + +- **Closing Status Categories**: categories that close the Finding, from `new` (To Do), `indeterminate` (In Progress) and `done`. Default `done`. +- **Reopening Status Categories**: categories that reopen a closed Finding. Default `new` and `indeterminate`. +- **False Positive Resolutions**: resolutions that close the Finding as a false positive. +- **Accepted Risk Resolutions**: resolutions that close the Finding as an accepted risk. + +A resolution in neither list closes the Finding as mitigated. Leave a list empty to use the receiver's default. The resolution defaults come from your classic Jira instance when you have exactly one (the migration from classic Jira also copies them onto each migrated mapping). With several classic instances, no defaults are imposed: set the resolutions on each mapping. An issue linked through the classic Jira integration always reads the resolutions of the classic instance configured for its own site, so each classic instance's URL must be its Jira site's base URL. + +The form warns when an outbound status would come back as the opposite change, for example when **Active Mapping** names a status in the `done` category, which two-way sync would read as a closure. + ### Custom Fields (optional) You can map additional Jira fields — for example a required `resolution` on close, or `labels` — in the mapping's **Custom Fields** step. Each custom-field mapping has four parts: @@ -109,9 +122,39 @@ You can map additional Jira fields — for example a required `resolution` on cl By default Jira issues use DefectDojo's built-in title and body. To customize them, attach a **Ticket Template** to the mapping in its **Ticket Template** step. A template defines four independently-optional pieces — the **Finding** summary and description, and the **Finding Group** summary and description. Any piece left blank falls back to the built-in default, so you can override just the title, just the body, or all four. Use **Test render** in the template editor to preview the rendered output against sample data — catching mistakes such as unknown placeholders or values that exceed a field's length limit — before saving. If a template is later deleted, the mappings that used it revert to the built-in defaults automatically. +### Push notes as comments (optional) + +Turn on **Push Notes as Comments** on the mapping to post new notes on a linked Finding to its Jira issue as comments, made by the connection's account. + +- Every public note attached to the Finding is posted, however it was added: the API (`POST /api/v2/findings/{id}/notes/`), the classic UI, the Pro UI, a bulk edit, or closing the Finding with a note. +- Private notes are never posted. +- The comment reads `(Author name): note text`. The note is sent in full, up to Jira's comment length limit (a longer one is cut and ends with `[truncated]`). +- Markdown is converted to Jira formatting. Mentions (`[~accountid:...]`), images and macros are escaped and show as plain text, so a note cannot ping a Jira user. +- A note on a grouped Finding goes to the Finding Group's issue. The same text is posted to one group issue once within ten minutes, so a bulk action on many members of a group adds one comment, not one per member. +- A Finding with no issue yet gets no comment: a note never creates an issue. +- Notes added by Triage Engine rules, including the ones two-way sync adds from Jira comments, are never sent back. +- Editing or deleting a note does not change the comment. + +With classic scopes, posting comments needs no scope beyond `write:jira-work`. With granular scopes, Atlassian's reference for the Add comment operation lists `write:comment:jira` together with the read scopes `read:comment:jira`, `read:comment.property:jira`, `read:group:jira`, `read:project:jira`, `read:project-role:jira`, `read:user:jira` and `read:avatar:jira`. Atlassian changes these lists from time to time, so check its [REST API reference](https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issue-comments/#api-rest-api-2-issue-issueidorkey-comment-post) if a comment fails with a `403` naming a scope. + +Posting comments also needs a go-integrators version that provides it. Until one is deployed, each mapping records one integration error saying so, and posting is tried again an hour later. + +### Two-way sync + +The connector pushes to Jira. To have Jira talk back, open the connection and choose **Turn On Two-way Sync**. This creates a Triage Engine [webhook receiver](/automation/triage_engine/webhook_receivers/) bound to the connection, and the rule that acts on its deliveries: + +- An issue moving into a closed status category closes the linked Finding, as a false positive or an accepted risk when its resolution is in that list. +- An issue moving back to an open category reopens the Finding. +- A comment on the issue is added to the Finding as a note, once. Comments DefectDojo posted itself are recognized and skipped. + +The receiver's **Setup** tab gives the URL and the steps to create the webhook in Jira. Because the receiver is bound to one connection, two Jira sites with overlapping project keys never touch each other's Findings. A connection has one two-way sync receiver; when somebody else turned it on, the connection shows **Managed by another user**. + +If Jira changes do not arrive, see [two-way sync troubleshooting](/connectors/downstream/troubleshooting_jira/#two-way-sync-changes-from-jira-are-not-reaching-findings). + ### How it works - **Create / Update / Delete:** creating pushes a new issue and records the link on the Finding; updating edits the existing issue; deleting a Finding force-closes its issue (nothing is deleted in Jira). Pushes can be manual ("Push to Integrator") or automatic per the Issue Tracker Assignment. +- **Comment:** with **Push Notes as Comments** on, a new note on a Finding is added to its issue as a comment. - **Status reconciliation:** after creating (and on every update) DefectDojo reads the issue's current status and, if it differs from the mapped target, finds a single workflow transition that reaches it and applies it. If no such transition exists, the mapping records an error rather than failing silently. Any transition-scoped custom fields are sent with that transition. - **Ticket link:** the link surfaced on the Finding is `https://your-site.atlassian.net/browse/{ISSUE-KEY}` — always your public site URL, never the internal gateway. - **Token lifecycle (OAuth):** DefectDojo owns the whole flow — it performs the authorization-code exchange, stores the access and refresh tokens, and refreshes on demand before a push, persisting the new refresh token each time (Atlassian rotates it on every refresh). diff --git a/docs/content/releases/pro/changelog.md b/docs/content/releases/pro/changelog.md index 213146b239..923c2fe9d0 100644 --- a/docs/content/releases/pro/changelog.md +++ b/docs/content/releases/pro/changelog.md @@ -18,6 +18,14 @@ For Open Source release notes, please see the [Releases page on GitHub](https:// ## September 2026: v3.3 +### September 28, 2026: v3.3.300 + +New features: +* **(Triage Engine)** Webhook receivers let Jira and other tools send changes back to DefectDojo, and the Jira Downstream Connector gains two-way sync and Push Notes as Comments. + +Upgrade notes: +* **(Deployment)** Self-hosted Docker Compose deployments, and Helm deployments with `webhookGateway.enabled`, run a new webhook gateway service in front of webhook receivers, with its own schema and database role inside DefectDojo's database. Please see [additional instructions](/releases/pro/webhook-gateway) for more details. + ### September 22, 2026: v3.3.200 New features: diff --git a/docs/content/releases/pro/webhook-gateway.md b/docs/content/releases/pro/webhook-gateway.md new file mode 100644 index 0000000000..9769d091f3 --- /dev/null +++ b/docs/content/releases/pro/webhook-gateway.md @@ -0,0 +1,131 @@ +--- +title: "Adding the Webhook Gateway on Upgrade" +toc_hide: true +weight: -20260928 +description: "What an existing self-hosted DefectDojo Pro installation gets with the webhook gateway, what a database administrator may need to run, how to verify it, and how to roll it back." +audience: pro +--- + +DefectDojo Pro can put a **webhook gateway** in front of Triage Engine [webhook receivers](/automation/triage_engine/webhook_receivers/), the URLs Jira and other tools post to for [two-way sync](/connectors/toolreference/jira/#two-way-sync). The gateway stores each webhook and answers the sender before DefectDojo is involved, then delivers it with retries, so a DefectDojo restart or short outage loses nothing. + +This page is for operators upgrading an existing self-hosted installation to the first release that ships the gateway. Most installations have nothing to do: the upgrade creates everything the gateway needs. Read on if your database user has restricted privileges, if you count database connections, or if you want to know what to check and how to roll back. + +## What changes + +### Docker Compose + +The Docker Compose bundles turn the gateway on (`DD_WEBHOOK_GATEWAY_MODE=whook` and `WEBHOOK_GATEWAY_ENABLED=true`). An upgrade with `dojo-compose-cli` brings: + +- **A new container and image**, `webhook-gateway`, from the same registry and version as the rest of DefectDojo Pro. The standard image is published for `linux/amd64` and `linux/arm64`; FIPS deployments use the `-fips` variant, published for `linux/amd64`. Air-gapped installations need this image in the transferred bundle too. +- **Two new nginx files** in the deployment directory, `nginx/webhook-gateway.conf` and `nginx/webhook-direct.conf`, bind-mounted into the nginx container. `dojo-compose-cli deploy download` fetches them with the rest of the deployment files. +- **A new named volume**, `webhook_gateway_secrets`. DefectDojo's `init` container writes the gateway's secrets and database URL there, and only `init` and the gateway mount it. +- **A schema and a database role** inside DefectDojo's existing database (see below). No new database is needed. + +### Kubernetes + +The Helm chart runs the gateway when `webhookGateway.enabled` is on. Check the value's default in the chart version you install: a chart may ship it off until the DefectDojo Pro release that publishes the gateway image. With it off, DefectDojo serves receiver URLs itself (`DD_WEBHOOK_GATEWAY_MODE=direct`). With it on, the gateway pod's `webhook-gateway-db-setup` init container prepares the same schema and role described below and prints the statements to run when it cannot. An installation that uses `dojo.existingSecret` keeps the gateway off until you add its keys to your Secret and set `webhookGateway.existingSecretHasKeys`, because the chart cannot derive them without seeing `dojo.secretKey`. + +### ECS + +The ECS task definitions do not include the gateway. They set `DD_WEBHOOK_GATEWAY_MODE=direct`, so DefectDojo answers receiver URLs itself and nothing below applies. + +## The database schema and role + +The gateway keeps its tables in a schema of its own, `whook` by default (`DD_WEBHOOK_GATEWAY_SCHEMA`), inside DefectDojo's database. It logs in with a role of its own, `defectdojo_webhook_gateway` by default (`DD_WEBHOOK_GATEWAY_DB_ROLE`), which owns that schema and can use nothing else of DefectDojo's. + +On startup, DefectDojo creates both when they are missing, using DefectDojo's own database user: + +- Creating the role needs `CREATEROLE`. The admin user of most managed databases has it, and so does DefectDojo's user in a new installation of the Docker Compose bundle with a bundled database. A bundled database created before this release does not grant it: run `ALTER USER CREATEROLE;` once as the `postgres` user, then restart DefectDojo. Without it, the gateway connects with DefectDojo's credentials, confined to its schema by `search_path` only, and `init` logs the statements that would give it a role of its own. +- Creating the schema needs `CREATE` on DefectDojo's database, which the database's owner always has. Without it, `init` logs the exact statement to run, the gateway refuses to start and prints the same statement, and the receivers list shows the gateway as **Not Started**. + +When DefectDojo's user may do neither, have a database administrator run the following once, with your database name and a password of your choosing, connected to DefectDojo's database: + +```sql +CREATE ROLE defectdojo_webhook_gateway LOGIN PASSWORD ''; +GRANT CONNECT ON DATABASE TO defectdojo_webhook_gateway; +CREATE SCHEMA IF NOT EXISTS whook AUTHORIZATION defectdojo_webhook_gateway; +``` + +Then set `DD_WEBHOOK_GATEWAY_DB_PASSWORD` to that password in the deployment's environment, so DefectDojo hands the gateway the same one. To keep the gateway on DefectDojo's own credentials instead, set `DD_WEBHOOK_GATEWAY_DB_ROLE` to an empty value and create only the schema: + +```sql +CREATE SCHEMA IF NOT EXISTS whook AUTHORIZATION ; +``` + +## Secrets + +The gateway needs an admin token, a key that encrypts stored receiver tokens, a key it signs its deliveries to DefectDojo with, and its database role's password. Each installation derives its own from `DD_SECRET_KEY`, so there is nothing to generate before the upgrade, and the gateway container never receives `DD_SECRET_KEY` itself. The Helm chart derives them from `dojo.secretKey`. To manage one yourself, set `DD_WEBHOOK_GATEWAY_ADMIN_TOKEN`, `DD_WEBHOOK_GATEWAY_SECRET_KEY`, `DD_WEBHOOK_GATEWAY_DELIVERY_SECRET` or `DD_WEBHOOK_GATEWAY_DB_PASSWORD`; a value that is set wins over the derived one. + +Because the derived secrets follow `DD_SECRET_KEY`, an installation still running on the `DD_SECRET_KEY` from DefectDojo's deployment files gets a startup warning (system check `pro.W002`). Give it its own key. When you change `DD_SECRET_KEY` later, restart DefectDojo, its workers and the gateway together; see [Changing the secret key or the gateway secrets](/automation/triage_engine/configuration/#changing-the-secret-key-or-the-gateway-secrets). + +## Connections and rate limits + +- The gateway holds at most 5 connections to DefectDojo's database server by default (`WHOOK_DB_MAX_CONNS`; `webhookGateway.database.maxConnections` in the Helm chart), on top of DefectDojo's own. Count them against the server's `max_connections`, or a managed database's connection limit. During a Kubernetes rollout two gateway pods briefly run at once. +- nginx allows each sender address 100 receiver requests per second by default (`DD_WEBHOOK_RECEIVER_RATE`, burst `DD_WEBHOOK_RECEIVER_BURST`, 1000), and the gateway accepts 100 deliveries per second per receiver (`WHOOK_INGEST_RATE`, burst `WHOOK_INGEST_BURST`, 1000). Both answer `429` above the limit, which senders retry. +- The largest webhook body is `DD_RULES_V2_WEBHOOK_MAX_BODY_BYTES` (1 MiB by default), one value that nginx, the gateway and DefectDojo all read. + +## Verifying the upgrade + +1. **The initializer prepared the database.** In the `init` container's log, look for: + + ``` + Prepared the webhook gateway's database role and schema + Wrote the webhook gateway's secrets + ``` + + `Prepared the webhook gateway's schema (it uses DefectDojo's database credentials)` means the role could not be created and the gateway uses DefectDojo's user. `Could not prepare the webhook gateway's database; see the warning above` means the schema could not be created either: run the statements from the warning. + + ```bash + docker compose logs init | grep -i "webhook gateway" + ``` + +2. **The gateway is running and healthy.** Its health check fails while it cannot reach its database. + + ```bash + docker compose ps webhook-gateway + ``` + +3. **The gateway created its tables.** Connected to DefectDojo's database with `psql`: + + ``` + \dt whook.* + ``` + + lists the gateway's tables once it has started. `\du defectdojo_webhook_gateway` shows its role. + +4. **DefectDojo can reach it.** In DefectDojo, open **Triage Engine > Webhook Receivers**. The top of the list shows the gateway as **Healthy**. **Unreachable** means DefectDojo cannot reach the gateway container, **Not Started** means its schema is missing, and **Turned Off** means the **Inbound Webhooks** feature flag is off. + +On Kubernetes, the gateway pod's setup log shows the same preparation: + +```bash +kubectl logs -n deploy/-webhook-gateway -c webhook-gateway-db-setup +``` + +## Running without the gateway + +To have DefectDojo answer receiver URLs itself, set `WEBHOOK_GATEWAY_ENABLED=false` and `DD_WEBHOOK_GATEWAY_MODE=direct` together (on Kubernetes, `webhookGateway.enabled: false`). Receivers keep working, but a delivery that arrives while DefectDojo is down is lost unless the sender retries it. + +## Rolling back + +A release without the gateway does not need anything the gateway added, but three things outlive a rollback and should be removed by hand: + +- **The gateway container.** An older bundle has no `webhook-gateway` service, so a plain `docker compose up` leaves the running one behind, still holding database connections. Remove it with: + + ```bash + docker compose up -d --remove-orphans + ``` + +- **The secrets volume.** Once the gateway container is gone: + + ```bash + docker volume rm _webhook_gateway_secrets + ``` + +- **The schema and the role.** The schema holds captured webhook bodies and headers for up to 30 days (90 for failed deliveries). Drop it once nothing needs to be replayed from it, and drop the role if DefectDojo created it: + + ```sql + DROP SCHEMA whook CASCADE; + DROP ROLE defectdojo_webhook_gateway; + ``` + +On a release without webhook receivers, their URLs answer `404`, and senders retry for their own retry window.