diff --git a/docs/languages/go.md b/docs/languages/go/index.md similarity index 97% rename from docs/languages/go.md rename to docs/languages/go/index.md index c873f2c9..0a037d58 100644 --- a/docs/languages/go.md +++ b/docs/languages/go/index.md @@ -406,3 +406,8 @@ func Handle(w http.ResponseWriter, r *http.Request) { } ``` +## Examples + +* [The director pattern](/languages/patterns/director/) +* [The fan-out pattern](/languages/patterns/fan-out/) +* [The singleton pattern](/languages/patterns/singleton/) diff --git a/docs/languages/node.md b/docs/languages/node.md index 2d30fcb0..5ba4fb47 100644 --- a/docs/languages/node.md +++ b/docs/languages/node.md @@ -353,3 +353,8 @@ functions: - The `NODE_OPTIONS` environment variable needs to have the value `--require @opentelemetry/auto-instrumentations-node/register` to register and initialize the auto instrumentation module. To see the full range of configuration options, see [Module Configuration](https://opentelemetry.io/docs/zero-code/js/configuration/). + +## Examples + +* [The director pattern](/languages/patterns/director/) +* [The fan-out pattern](/languages/patterns/fan-out/) diff --git a/docs/languages/overview.md b/docs/languages/overview.md index c6550317..27f15fd6 100644 --- a/docs/languages/overview.md +++ b/docs/languages/overview.md @@ -12,7 +12,7 @@ There are many community templates, of varying levels of support and maintenance There are a number of official templates maintained and recommended by OpenFaaS Ltd, the following are currently documented: -* [Go](./go.md) +* [Go](./go/index.md) * [Node](./node.md) * [Python](./python/index.md) * [Dockerfile](./dockerfile.md) @@ -34,3 +34,9 @@ You can also [create your own custom templates](./custom.md), or fork an existin You can bring along your own [pre-existing Dockerfiles and container images](/languages/dockerfile), so long as they conform to the [OpenFaaS workloads contract](/reference/workloads). You may need to add a health or readiness endpoint to make sure that no requests are lost during scaling up and down of your function. +### Function patterns + +After choosing a language, see the +[common function design patterns](/languages/patterns/) for ways to coordinate +functions into workflows, process work in parallel, or run a function with a +fixed replica count. diff --git a/docs/languages/patterns/director.md b/docs/languages/patterns/director.md new file mode 100644 index 00000000..b24be59b --- /dev/null +++ b/docs/languages/patterns/director.md @@ -0,0 +1,552 @@ +The [Director pattern](/languages/patterns/#director-pattern) can be used to +implement workflow functions that coordinate other functions through +sequencing, branching, or parallel execution. + +This page builds a deliberately small example: two functions each return a +JSON object, and a director combines them into one response. + +```text + [ Client ] + │ + ▼ +┌──────────────────────┐ +│ director │ +└──────────┬───────────┘ + ├── 1. invoke ──► [ function-a ] ──► {"a": 1} ──┐ + ├── 2. invoke ──► [ function-b ] ──► {"b": 2} ──┤ + │ │ + ├────────────── merge results ◄─────────────────┘ + │ + ▼ +[ Response: {"a": 1, "b": 2} ] +``` + +This simple workflow highlights several properties of the Director pattern: + +* **Single endpoint:** the caller only invokes `director` and receives one + response. +* **Sequencing:** the director invokes `function-a`, followed by `function-b`. +* **Independent functions:** each function is deployed and scaled separately, + and could be written in a different language. +* **Composition and error handling:** the director validates and merges both + results, or returns an error when something fails. + +## Create the functions + +Choose a language and scaffold all three functions in one `stack.yaml` file: + +=== "Go" + + ```bash + faas-cli template store pull golang-middleware + faas-cli new --lang golang-middleware director \ + --prefix ttl.sh/openfaas-examples + faas-cli new --lang golang-middleware function-a \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + faas-cli new --lang golang-middleware function-b \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + ``` + +=== "Python" + + ```bash + faas-cli template store pull python3-http + faas-cli new --lang python3-http director \ + --prefix ttl.sh/openfaas-examples + faas-cli new --lang python3-http function-a \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + faas-cli new --lang python3-http function-b \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + ``` + +=== "Node.js" + + ```bash + faas-cli template store pull node24 + faas-cli new --lang node24 director \ + --prefix ttl.sh/openfaas-examples + faas-cli new --lang node24 function-a \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + faas-cli new --lang node24 function-b \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + ``` + +The example uses the public [ttl.sh] registry. Replace the +prefix with your own registry for production use. + +The full source code and `stack.yaml` files are available on GitHub for +[Go](https://github.com/openfaas/function-patterns/tree/master/go/director), +[Python](https://github.com/openfaas/function-patterns/tree/master/python/director), +and [Node.js](https://github.com/openfaas/function-patterns/tree/master/node/director). + +## Implement the two functions + +Create two simple functions that each return a JSON payload. Since neither +function needs input, the completed workflow is invoked with a `GET` request. + +=== "Go" + + `function-a/handler.go`: + + ```go + package function + + import ( + "context" + "encoding/json" + "net/http" + ) + + func Handle(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(map[string]int{"a": 1}) + } + ``` + + `function-b/handler.go`: + + ```go + package function + + import ( + "encoding/json" + "net/http" + ) + + func Handle(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(map[string]int{"b": 2}) + } + ``` + +=== "Python" + + `function-a/handler.py`: + + ```python + def handle(event, context): + return {"statusCode": 200, "body": {"a": 1}} + ``` + + `function-b/handler.py`: + + ```python + def handle(event, context): + return {"statusCode": 200, "body": {"b": 2}} + ``` + +=== "Node.js" + + `function-a/handler.js`: + + ```javascript + 'use strict' + + module.exports = async (event, context) => context + .status(200) + .headers({ 'Content-Type': 'application/json' }) + .succeed({ a: 1 }) + ``` + + `function-b/handler.js`: + + ```javascript + 'use strict' + + module.exports = async (event, context) => context + .status(200) + .headers({ 'Content-Type': 'application/json' }) + .succeed({ b: 2 }) + ``` + +## Implement the director + +The director invokes `function-a` and `function-b` in sequence. It decodes each +response as a JSON object and copies its fields into the combined response. + +=== "Go" + + `director/handler.go`: + + ```go + package function + + import ( + "context" + "encoding/json" + "fmt" + "net/http" + "os" + "strings" + "time" + ) + + const defaultStageTimeout = 5 * time.Second + + func Handle(w http.ResponseWriter, r *http.Request) { + gateway := os.Getenv("gateway_url") + if gateway == "" { + gateway = "http://gateway.openfaas:8080" + } + gateway = strings.TrimRight(gateway, "/") + + timeout := stageTimeout() + + // Invoke function-a and decode its JSON response. + ctxA, cancelA := context.WithTimeout(r.Context(), timeout) + defer cancelA() + reqA, err := http.NewRequestWithContext( + ctxA, + http.MethodGet, + gateway+"/function/function-a", + nil, + ) + if err != nil { + http.Error(w, fmt.Sprintf("function-a: %s", err), http.StatusBadGateway) + return + } + + resA, err := http.DefaultClient.Do(reqA) + if err != nil { + http.Error(w, fmt.Sprintf("function-a: %s", err), http.StatusBadGateway) + return + } + defer resA.Body.Close() + if resA.StatusCode != http.StatusOK { + status := resA.StatusCode + http.Error(w, fmt.Sprintf("function-a: returned %s", resA.Status), status) + return + } + + combined := map[string]any{} + if err := json.NewDecoder(resA.Body).Decode(&combined); err != nil { + http.Error( + w, + fmt.Sprintf("function-a: returned invalid JSON: %s", err), + http.StatusBadGateway, + ) + return + } + + // Invoke function-b after function-a has completed. + ctxB, cancelB := context.WithTimeout(r.Context(), timeout) + defer cancelB() + reqB, err := http.NewRequestWithContext( + ctxB, + http.MethodGet, + gateway+"/function/function-b", + nil, + ) + if err != nil { + http.Error(w, fmt.Sprintf("function-b: %s", err), http.StatusBadGateway) + return + } + + resB, err := http.DefaultClient.Do(reqB) + if err != nil { + http.Error(w, fmt.Sprintf("function-b: %s", err), http.StatusBadGateway) + return + } + defer resB.Body.Close() + if resB.StatusCode != http.StatusOK { + status := resB.StatusCode + http.Error(w, fmt.Sprintf("function-b: returned %s", resB.Status), status) + return + } + + functionB := map[string]any{} + if err := json.NewDecoder(resB.Body).Decode(&functionB); err != nil { + http.Error( + w, + fmt.Sprintf("function-b: returned invalid JSON: %s", err), + http.StatusBadGateway, + ) + return + } + + // Merge function-b into function-a and return the combined object. + for key, value := range functionB { + combined[key] = value + } + + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(combined) + } + + func stageTimeout() time.Duration { + if value := os.Getenv("stage_timeout"); value != "" { + if timeout, err := time.ParseDuration(value); err == nil && timeout > 0 { + return timeout + } + } + return defaultStageTimeout + } + ``` + +=== "Python" + + `director/handler.py`: + + ```python + import os + + import requests + + + GATEWAY_URL = os.getenv( + "gateway_url", "http://gateway.openfaas:8080" + ).rstrip("/") + STAGE_TIMEOUT = float(os.getenv("stage_timeout", "5")) + + + def handle(event, context): + # Invoke function-a and decode its JSON response. + try: + response_a = requests.get( + f"{GATEWAY_URL}/function/function-a", + timeout=STAGE_TIMEOUT, + ) + except requests.RequestException as err: + return error(502, f"function-a: {err}") + + if response_a.status_code != 200: + return error(response_a.status_code, f"function-a: {response_a.text}") + + try: + combined = response_a.json() + except ValueError as err: + return error(502, f"function-a returned invalid JSON: {err}") + + if not isinstance(combined, dict): + return error(502, "function-a did not return a JSON object") + + # Invoke function-b after function-a has completed. + try: + response_b = requests.get( + f"{GATEWAY_URL}/function/function-b", + timeout=STAGE_TIMEOUT, + ) + except requests.RequestException as err: + return error(502, f"function-b: {err}") + + if response_b.status_code != 200: + return error(response_b.status_code, f"function-b: {response_b.text}") + + try: + function_b = response_b.json() + except ValueError as err: + return error(502, f"function-b returned invalid JSON: {err}") + + if not isinstance(function_b, dict): + return error(502, "function-b did not return a JSON object") + + # Merge function-b into function-a and return the combined object. + combined.update(function_b) + + return {"statusCode": 200, "body": combined} + + + def error(status_code, message): + return {"statusCode": status_code, "body": message.strip()} + ``` + + Add `requests` to `director/requirements.txt`. + +=== "Node.js" + + `director/handler.js`: + + ```javascript + 'use strict' + + const gatewayURL = process.env.gateway_url || + 'http://gateway.openfaas:8080' + const stageTimeout = Number(process.env.stage_timeout || '5') * 1000 + + module.exports = async (event, context) => { + const gateway = gatewayURL.replace(/\/$/, '') + + // Invoke function-a and decode its JSON response. + let responseA + try { + responseA = await fetch(`${gateway}/function/function-a`, { + method: 'GET', + signal: AbortSignal.timeout(stageTimeout) + }) + } catch (error) { + return fail(context, 502, `function-a: ${error.message}`) + } + + if (responseA.status !== 200) { + return fail(context, responseA.status, `function-a: ${await responseA.text()}`) + } + + let combined + try { + combined = await responseA.json() + } catch (error) { + return fail(context, 502, `function-a returned invalid JSON: ${error.message}`) + } + + if (!combined || Array.isArray(combined) || typeof combined !== 'object') { + return fail(context, 502, 'function-a did not return a JSON object') + } + + // Invoke function-b after function-a has completed. + let responseB + try { + responseB = await fetch(`${gateway}/function/function-b`, { + method: 'GET', + signal: AbortSignal.timeout(stageTimeout) + }) + } catch (error) { + return fail(context, 502, `function-b: ${error.message}`) + } + + if (responseB.status !== 200) { + return fail(context, responseB.status, `function-b: ${await responseB.text()}`) + } + + let functionB + try { + functionB = await responseB.json() + } catch (error) { + return fail(context, 502, `function-b returned invalid JSON: ${error.message}`) + } + + if (!functionB || Array.isArray(functionB) || typeof functionB !== 'object') { + return fail(context, 502, 'function-b did not return a JSON object') + } + + // Merge function-b into function-a and return the combined object. + Object.assign(combined, functionB) + + return context + .status(200) + .headers({ 'Content-Type': 'application/json' }) + .succeed(combined) + } + + function fail (context, status, message) { + return context + .status(status) + .headers({ 'Content-Type': 'text/plain' }) + .succeed(message.trim()) + } + ``` + +The director calls each function through the gateway, so the functions can be +written in different languages and scaled independently. A transport error or +invalid JSON response returns `502 Bad Gateway`. A non-200 response is +attributed to the function that returned it and its status is passed through to +the caller. + +Configure the director's timeouts in `stack.yaml`: + +=== "Go" + + ```yaml + functions: + director: + lang: golang-middleware + handler: ./director + image: ttl.sh/openfaas-examples/director:latest + environment: + stage_timeout: 5s + exec_timeout: 15s + read_timeout: 16s + write_timeout: 16s + ``` + +=== "Python" + + ```yaml + functions: + director: + lang: python3-http + handler: ./director + image: ttl.sh/openfaas-examples/director:latest + environment: + stage_timeout: "5" + exec_timeout: 15s + read_timeout: 16s + write_timeout: 16s + ``` + +=== "Node.js" + + ```yaml + functions: + director: + lang: node24 + handler: ./director + image: ttl.sh/openfaas-examples/director:latest + environment: + stage_timeout: "5" + exec_timeout: 15s + read_timeout: 16s + write_timeout: 16s + ``` + +## Configure timeouts + +A director stays active while it waits for the functions it invokes, so each +call should have a timeout. + +This example uses the custom `stage_timeout` +environment variable, the other variables below are OpenFaaS watchdog settings. + +| Setting | Scope | +|---------|-------| +| `stage_timeout` | Custom HTTP client timeout for each downstream call | +| `exec_timeout` | Maximum duration of the complete director invocation | +| `read_timeout`, `write_timeout` | Watchdog timeouts, set slightly longer than `exec_timeout` | + +This director calls the functions sequentially, so its expected duration is +the sum of both calls plus a small amount of overhead. Configure the director's +timeout for that complete path, not for a single function. + +The gateway's `upstream_timeout` must be at least as long as the director's +`exec_timeout`. Adjust the example values for your own functions and see +[extended timeouts](/tutorials/expanded-timeouts/) for the complete +configuration. + +## Deploy and invoke + +Build, push, and deploy all three functions: + +```bash +faas-cli up --tag=sha +``` + +Invoke the director: + +```bash +curl -s http://127.0.0.1:8080/function/director | jq +``` + +The director returns the union of the two responses: + +```json +{ + "a": 1, + "b": 2 +} +``` + +## Workflow considerations + +* Invoke dependent functions in sequence. Independent functions can run in + parallel to reduce latency, but the director must still wait for every result + it needs before continuing. +* The director owns the error policy. Depending on the workflow, it can stop, + retry, return a partial result, or save progress for a later invocation. +* If retrying the director could repeat side effects in a function that already + completed, make those operations idempotent. +* The functions invoked by a director are deployed independently. Each can use + a different language, scale separately, and be updated without moving the + workflow logic out of the director. +* For a long-running workflow, the director can be invoke asynchronously through + `/async-function/director` with an `X-Callback-Url`. The director continues to + wait for its functions, while the client receives the final result through + the callback. See [asynchronous functions](/reference/async/#how-it-works). diff --git a/docs/languages/patterns/fan-in.md b/docs/languages/patterns/fan-in.md new file mode 100644 index 00000000..dcaba1e7 --- /dev/null +++ b/docs/languages/patterns/fan-in.md @@ -0,0 +1,672 @@ +The Fan-in pattern complements the +[Fan-out pattern](/languages/patterns/#fan-out-pattern) by bringing the +independent results back together. The workflow can then continue after every +item in the batch has completed. + +A Fan-in implementation associates each result with its batch, records +progress as results arrive, and triggers the next step once all expected work +is complete. It should also store the individual results when the next step needs +to combine them. + +This example extends the [Fan-out example](/languages/patterns/fan-out/) by +sending the result of each `batch-worker` invocation to a `fan-in` function. +The function uses a PostgreSQL counter to track completion and invokes the +next function when the counter reaches zero. + +```text +1. Initialize the batch + +[ Client ] ── size 3 ──► [ fan-in ] ──► [ PostgreSQL: remaining 3 ] + ▲ │ + └────── batch ID ─────────┘ + +2. Fan the callbacks back in + +[ Queue-worker ] + ├── "one" ───► [ batch-worker ] ──┐ + ├── "two" ───► [ batch-worker ] ──┼── callbacks ──► [ fan-in ] + └── "three" ─► [ batch-worker ] ──┘ │ + ▼ + [ PostgreSQL: 3 → 2 → 1 → 0 ] + │ + ▼ + [ printer ] +``` + +* A counter is created with the number of items before any + work is submitted. +* Every worker result is sent to the same callback function + with the batch ID. +* PostgreSQL updates the counter atomically, so callbacks + can arrive concurrently and reach different `fan-in` replicas. +* Only the callback that changes the counter to zero invokes + the next function for the completed batch. + +The example only tracks completion. A workflow that needs to combine results +would also store each callback body under the batch ID, then have the final +function read and combine those results. + +## Create the function + +The `fan-out` and `batch-worker` functions come from the +[Fan-out example](/languages/patterns/fan-out/). Scaffold the additional `fan-in` function: + +=== "Go" + + ```bash + mkdir fan-in && cd fan-in + faas-cli template store pull golang-middleware + faas-cli new --lang golang-middleware fan-in \ + --prefix ttl.sh/openfaas-examples + ``` + +=== "Python" + + ```bash + mkdir fan-in && cd fan-in + faas-cli template store pull python3-http + faas-cli new --lang python3-http fan-in \ + --prefix ttl.sh/openfaas-examples + ``` + +=== "Node.js" + + ```bash + mkdir fan-in && cd fan-in + faas-cli template store pull node24 + faas-cli new --lang node24 fan-in \ + --prefix ttl.sh/openfaas-examples + ``` + +The example uses the public [ttl.sh](https://ttl.sh) registry. Replace the +prefix with your own registry for production use. + +The full source code and `stack.yaml` files are available on GitHub for +[Go](https://github.com/openfaas/function-patterns/tree/master/go/fan-in), +[Python](https://github.com/openfaas/function-patterns/tree/master/python/fan-in), +and [Node.js](https://github.com/openfaas/function-patterns/tree/master/node/fan-in). + +## Configure PostgreSQL + +Create a table with one row for each batch. Completed rows remain at zero so a +late callback cannot invoke the final function again. + +`fan-in/schema.sql`: + +```sql +CREATE TABLE IF NOT EXISTS batch_counters ( + batch_id text PRIMARY KEY, + remaining integer NOT NULL CHECK (remaining >= 0) +); +``` + +Apply the schema to a PostgreSQL database that the function can reach: + +```bash +psql "$POSTGRES_CONNECTION" -f fan-in/schema.sql +``` + +Store the same connection string in `postgres-connection.txt`, then create an +OpenFaaS secret from it: + +```bash +faas-cli secret create postgres-connection \ + --from-file=postgres-connection.txt +``` + +The function reads the connection string from +`/var/openfaas/secrets/postgres-connection`. + +## Implement fan-in + +The function exposes two paths: + +* `POST /batch` accepts a size, creates the counter, and returns a generated + batch ID. +* `POST /callback?batch_id=` atomically decrements the counter. The + callback that changes it to zero posts a completion message to `notify_url`. + +The SQL statement used by `/callback` both updates and reads the counter. The +`decremented` value distinguishes the callback that completed the batch from a +later callback that reads the existing zero. + +=== "Go" + + `fan-in/handler.go`: + + ```go + package function + + import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "net/http" + "os" + "strings" + "time" + + "github.com/jackc/pgx/v5" + ) + + const ( + connectionSecret = "/var/openfaas/secrets/postgres-connection" + notifyTimeout = 5 * time.Second + ) + + const decrementQuery = ` + WITH updated AS ( + UPDATE batch_counters + SET remaining = remaining - 1 + WHERE batch_id = $1 + AND remaining > 0 + RETURNING remaining + ) + SELECT remaining, true + FROM updated + UNION ALL + SELECT remaining, false + FROM batch_counters + WHERE batch_id = $1 + AND NOT EXISTS (SELECT 1 FROM updated) + LIMIT 1` + + type postgresCounter struct { + conn *pgx.Conn + } + + func Handle(w http.ResponseWriter, r *http.Request) { + defer r.Body.Close() + + connectionString, err := postgresConnection() + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + + conn, err := pgx.Connect(r.Context(), connectionString) + if err != nil { + http.Error(w, "connect to PostgreSQL", http.StatusBadGateway) + return + } + defer conn.Close(r.Context()) + + counter := postgresCounter{conn: conn} + switch r.URL.Path { + case "/batch": + createBatch(w, r, counter) + case "/callback": + completeItem(w, r, counter) + default: + http.NotFound(w, r) + } + } + + func createBatch(w http.ResponseWriter, r *http.Request, counter postgresCounter) { + var input struct { + Size int `json:"size"` + } + if err := json.NewDecoder(r.Body).Decode(&input); err != nil || + input.Size < 1 { + http.Error(w, "expected a positive size", http.StatusBadRequest) + return + } + + var batchID string + err := counter.conn.QueryRow(r.Context(), ` + INSERT INTO batch_counters (batch_id, remaining) + VALUES (gen_random_uuid()::text, $1) + RETURNING batch_id`, input.Size).Scan(&batchID) + if err != nil { + http.Error(w, "create batch", http.StatusBadGateway) + return + } + + writeJSON(w, http.StatusCreated, map[string]any{ + "batch_id": batchID, + "remaining": input.Size, + }) + } + + func completeItem(w http.ResponseWriter, r *http.Request, counter postgresCounter) { + batchID := strings.TrimSpace(r.URL.Query().Get("batch_id")) + if batchID == "" { + http.Error(w, "batch_id is required", http.StatusBadRequest) + return + } + + var remaining int + var decremented bool + err := counter.conn.QueryRow(r.Context(), decrementQuery, batchID). + Scan(&remaining, &decremented) + if errors.Is(err, pgx.ErrNoRows) { + http.Error(w, "batch not found", http.StatusNotFound) + return + } + if err != nil { + http.Error(w, "update batch", http.StatusBadGateway) + return + } + + complete := remaining == 0 + if complete && decremented { + if err := notify(r.Context(), batchID); err != nil { + http.Error(w, err.Error(), http.StatusBadGateway) + return + } + } + + writeJSON(w, http.StatusOK, map[string]any{ + "batch_id": batchID, + "remaining": remaining, + "complete": complete, + }) + } + + func postgresConnection() (string, error) { + value, err := os.ReadFile(connectionSecret) + if err != nil { + return "", err + } + return strings.TrimSpace(string(value)), nil + } + + func notify(parent context.Context, batchID string) error { + body, _ := json.Marshal(map[string]string{ + "batch_id": batchID, + "status": "complete", + }) + + ctx, cancel := context.WithTimeout(parent, notifyTimeout) + defer cancel() + + req, err := http.NewRequestWithContext( + ctx, http.MethodPost, os.Getenv("notify_url"), bytes.NewReader(body), + ) + if err != nil { + return err + } + req.Header.Set("Content-Type", "application/json") + + res, err := http.DefaultClient.Do(req) + if err != nil { + return err + } + defer res.Body.Close() + + if res.StatusCode < 200 || res.StatusCode >= 300 { + return fmt.Errorf("notification endpoint returned %s", res.Status) + } + return nil + } + + func writeJSON(w http.ResponseWriter, status int, value any) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + json.NewEncoder(w).Encode(value) + } + ``` + + Add `github.com/jackc/pgx/v5` to `fan-in/go.mod`. + +=== "Python" + + `fan-in/handler.py`: + + ```python + import json + import os + + import psycopg + import requests + + + CONNECTION_SECRET = "/var/openfaas/secrets/postgres-connection" + NOTIFY_TIMEOUT = 5 + + DECREMENT_QUERY = """ + WITH updated AS ( + UPDATE batch_counters + SET remaining = remaining - 1 + WHERE batch_id = %s + AND remaining > 0 + RETURNING remaining + ) + SELECT remaining, true + FROM updated + UNION ALL + SELECT remaining, false + FROM batch_counters + WHERE batch_id = %s + AND NOT EXISTS (SELECT 1 FROM updated) + LIMIT 1 + """ + + + def handle(event, context): + try: + with psycopg.connect(postgres_connection(), autocommit=True) as connection: + if event.path == "/batch": + return create_batch(event, connection) + if event.path == "/callback": + return complete_item(event, connection) + return error(404, "not found") + except (KeyError, OSError, psycopg.Error, requests.RequestException) as err: + return error(502, str(err)) + + + def create_batch(event, connection): + try: + size = int(json.loads(event.body)["size"]) + except (KeyError, TypeError, ValueError): + return error(400, "expected a positive size") + + if size < 1: + return error(400, "expected a positive size") + + row = connection.execute( + """ + INSERT INTO batch_counters (batch_id, remaining) + VALUES (gen_random_uuid()::text, %s) + RETURNING batch_id + """, + (size,), + ).fetchone() + + return { + "statusCode": 201, + "body": {"batch_id": row[0], "remaining": size}, + } + + + def complete_item(event, connection): + batch_id = event.query.get("batch_id", "").strip() + if not batch_id: + return error(400, "batch_id is required") + + row = connection.execute( + DECREMENT_QUERY, + (batch_id, batch_id), + ).fetchone() + if row is None: + return error(404, "batch not found") + + remaining, decremented = row + complete = remaining == 0 + if complete and decremented: + response = requests.post( + os.environ["notify_url"], + json={"batch_id": batch_id, "status": "complete"}, + timeout=NOTIFY_TIMEOUT, + ) + response.raise_for_status() + + return { + "statusCode": 200, + "body": { + "batch_id": batch_id, + "remaining": remaining, + "complete": complete, + }, + } + + + def postgres_connection(): + with open(CONNECTION_SECRET, encoding="utf-8") as secret: + return secret.read().strip() + + + def error(status_code, message): + return {"statusCode": status_code, "body": message} + ``` + + Add `psycopg[binary]` and `requests` to `fan-in/requirements.txt`. + +=== "Node.js" + + `fan-in/handler.js`: + + ```javascript + 'use strict' + + const fs = require('node:fs') + const { Client } = require('pg') + + const connectionSecret = '/var/openfaas/secrets/postgres-connection' + const notifyTimeout = 5000 + + const decrementQuery = ` + WITH updated AS ( + UPDATE batch_counters + SET remaining = remaining - 1 + WHERE batch_id = $1 + AND remaining > 0 + RETURNING remaining + ) + SELECT remaining, true AS decremented + FROM updated + UNION ALL + SELECT remaining, false AS decremented + FROM batch_counters + WHERE batch_id = $1 + AND NOT EXISTS (SELECT 1 FROM updated) + LIMIT 1` + + module.exports = async (event, context) => { + if (event.path !== '/batch' && event.path !== '/callback') { + return respond(context, 404, 'not found') + } + + const client = new Client({ connectionString: postgresConnection() }) + try { + await client.connect() + if (event.path === '/batch') { + return await createBatch(event, context, client) + } + return await completeItem(event, context, client) + } catch (error) { + return respond(context, 502, error.message) + } finally { + await client.end() + } + } + + async function createBatch (event, context, client) { + let size + try { + size = Number(JSON.parse(requestBody(event.body)).size) + } catch (error) { + return respond(context, 400, 'expected a positive size') + } + + if (!Number.isInteger(size) || size < 1) { + return respond(context, 400, 'expected a positive size') + } + + const result = await client.query( + `INSERT INTO batch_counters (batch_id, remaining) + VALUES (gen_random_uuid()::text, $1) + RETURNING batch_id`, + [size] + ) + + return respond(context, 201, { + batch_id: result.rows[0].batch_id, + remaining: size + }) + } + + async function completeItem (event, context, client) { + const batchID = String((event.query || {}).batch_id || '').trim() + if (!batchID) { + return respond(context, 400, 'batch_id is required') + } + + const result = await client.query(decrementQuery, [batchID]) + if (result.rows.length === 0) { + return respond(context, 404, 'batch not found') + } + + const remaining = Number(result.rows[0].remaining) + const complete = remaining === 0 + if (complete && result.rows[0].decremented) { + const response = await fetch(process.env.notify_url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ batch_id: batchID, status: 'complete' }), + signal: AbortSignal.timeout(notifyTimeout) + }) + if (!response.ok) { + throw new Error(`notification endpoint returned ${response.status}`) + } + } + + return respond(context, 200, { + batch_id: batchID, + remaining, + complete + }) + } + + function postgresConnection () { + return fs.readFileSync(connectionSecret, 'utf8').trim() + } + + function requestBody (body) { + if (Buffer.isBuffer(body)) { + return body.toString() + } + return String(body || '') + } + + function respond (context, status, body) { + const contentType = typeof body === 'object' + ? 'application/json' + : 'text/plain; charset=utf-8' + return context + .status(status) + .headers({ 'Content-Type': contentType }) + .succeed(body) + } + ``` + + Add `pg` to `fan-in/package.json`. + +## Configure the function + +=== "Go" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + fan-in: + lang: golang-middleware + handler: ./fan-in + image: ttl.sh/openfaas-examples/fan-in:latest + environment: + notify_url: http://gateway.openfaas:8080/async-function/printer + secrets: + - postgres-connection + ``` + +=== "Python" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + fan-in: + lang: python3-http + handler: ./fan-in + image: ttl.sh/openfaas-examples/python-fan-in:latest + environment: + notify_url: http://gateway.openfaas:8080/async-function/printer + secrets: + - postgres-connection + ``` + +=== "Node.js" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + fan-in: + lang: node24 + handler: ./fan-in + image: ttl.sh/openfaas-examples/node-fan-in:latest + environment: + notify_url: http://gateway.openfaas:8080/async-function/printer + secrets: + - postgres-connection + ``` + +The `notify_url` setting is use to configure the function or HTTP endpoint +invoked after the complete batch finishes. + +## Deploy and try Fan-in + +Deploy the `printer` function and follow its logs: + +```bash +faas-cli store deploy printer +faas-cli logs printer -t +``` + +Deploy `fan-in` after completing the +[Fan-out example](/languages/patterns/fan-out/): + +```bash +faas-cli up --tag=sha +``` + +Create a counter for the three-item batch and save its generated ID: + +```bash +batch_id=$(curl -s http://127.0.0.1:8080/function/fan-in/batch \ + -H "Content-Type: application/json" \ + -d '{"size":3}' | jq -r .batch_id) +``` + +Submit the same batch from the Fan-out example, using `fan-in` as the callback: + +```bash +curl -s http://127.0.0.1:8080/function/fan-out \ + -H "Content-Type: application/json" \ + -H "X-Callback-Url: http://gateway.openfaas:8080/function/fan-in/callback?batch_id=${batch_id}" \ + -d '["one", "two", "three"]' +``` + +Each callback decrements the counter. After the third callback, the `printer` +logs receive one completion message: + +```json +{ + "batch_id": "4876cf09-a90e-40ba-9ac7-56fc87dd536b", + "status": "complete" +} +``` + +## Fan-in considerations + +* Progress must be stored in shared storage and updated atomically because + callbacks may be concurrent and may reach different `fan-in` replicas. +* To combine results, persist each callback body with its batch ID and have the + final function read them after the counter reaches zero. See the + [original Fan-out and Fan-in example](https://www.openfaas.com/blog/fan-out-and-back-in-using-functions/) + for a larger workflow that stores and combines individual results. diff --git a/docs/languages/patterns/fan-out.md b/docs/languages/patterns/fan-out.md new file mode 100644 index 00000000..12393f3b --- /dev/null +++ b/docs/languages/patterns/fan-out.md @@ -0,0 +1,543 @@ +The [Fan-out pattern](/languages/patterns/#fan-out-pattern) can be used to split +a batch into independent items and submit each item to a worker function for +asynchronous processing. + +This page builds a deliberately small example: `fan-out` accepts a JSON array +of strings, queues one invocation of `batch-worker` for each item, and returns +the number of submitted items. + +```text + [ Client ] + │ batch with three items + ▼ + [ fan-out ] ──► [ Async queue ] ──► [ Queue-worker ] + │ ├── "one" ───► [ batch-worker ] + │ ├── "two" ───► [ batch-worker ] + │ └── "three" ─► [ batch-worker ] + ▼ +[ Response: 202 {"submitted": 3} ] +``` + +This simple batch highlights several properties of the Fan-out pattern: + +* **Batch splitting:** `fan-out` creates one asynchronous invocation for each + item in the input batch. +* **Asynchronous response:** the caller receives a response after the items + have been accepted by the queue, without waiting for processing to finish. +* **Independent processing:** items can be processed concurrently and may + complete in a different order from the input. +* **Independent scaling:** the queue provides back pressure while OpenFaaS can + scale the `batch-worker` function to handle the load. + +## Create the functions + +Choose a language and scaffold both functions in one `stack.yaml` file: + +=== "Go" + + ```bash + faas-cli template store pull golang-middleware + faas-cli new --lang golang-middleware fan-out \ + --prefix ttl.sh/openfaas-examples + faas-cli new --lang golang-middleware batch-worker \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + ``` + +=== "Python" + + ```bash + faas-cli template store pull python3-http + faas-cli new --lang python3-http fan-out \ + --prefix ttl.sh/openfaas-examples + faas-cli new --lang python3-http batch-worker \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + ``` + +=== "Node.js" + + ```bash + faas-cli template store pull node24 + faas-cli new --lang node24 fan-out \ + --prefix ttl.sh/openfaas-examples + faas-cli new --lang node24 batch-worker \ + --append stack.yaml --prefix ttl.sh/openfaas-examples + ``` + +The example uses the public [ttl.sh](https://ttl.sh) registry. Replace the +prefix with your own registry for production use. + +The full source code and `stack.yaml` files are available on GitHub for +[Go](https://github.com/openfaas/function-patterns/tree/master/go/fan-out), +[Python](https://github.com/openfaas/function-patterns/tree/master/python/fan-out), +and [Node.js](https://github.com/openfaas/function-patterns/tree/master/node/fan-out). + +## Implement the worker + +The worker receives one string from the batch and returns a JSON result. A real +worker could transform a record, generate a report, or process a file stored in +object storage. + +=== "Go" + + `batch-worker/handler.go`: + + ```go + package function + + import ( + "encoding/json" + "io" + "net/http" + ) + + func Handle(w http.ResponseWriter, r *http.Request) { + defer r.Body.Close() + + item, err := io.ReadAll(r.Body) + if err != nil { + http.Error(w, "unable to read item", http.StatusBadRequest) + return + } + + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(map[string]interface{}{ + "item": string(item), + "processed": true, + }) + } + ``` + +=== "Python" + + `batch-worker/handler.py`: + + ```python + def handle(event, context): + item = ( + event.body.decode() + if isinstance(event.body, bytes) + else str(event.body) + ) + + return { + "statusCode": 200, + "body": {"item": item, "processed": True}, + } + ``` + +=== "Node.js" + + `batch-worker/handler.js`: + + ```javascript + 'use strict' + + module.exports = async (event, context) => { + const item = Buffer.isBuffer(event.body) + ? event.body.toString() + : String(event.body || '') + + return context + .status(200) + .headers({ 'Content-Type': 'application/json' }) + .succeed({ item, processed: true }) + } + ``` + +## Implement fan-out + +The `fan-out` function decodes the batch and posts each item to the gateway's +`/async-function/batch-worker` route. Each `202 Accepted` response confirms +that an item was queued, not that the worker has completed it. + +=== "Go" + + `fan-out/handler.go`: + + ```go + package function + + import ( + "bytes" + "context" + "encoding/json" + "fmt" + "net/http" + "os" + "strings" + "time" + ) + + const ( + workerFunction = "batch-worker" + submitTimeout = 5 * time.Second + ) + + func Handle(w http.ResponseWriter, r *http.Request) { + defer r.Body.Close() + + var batch []string + if err := json.NewDecoder(r.Body).Decode(&batch); err != nil { + http.Error(w, "expected a JSON array of strings", http.StatusBadRequest) + return + } + if len(batch) == 0 { + http.Error(w, "batch must contain at least one item", http.StatusBadRequest) + return + } + + gateway := os.Getenv("gateway_url") + if gateway == "" { + gateway = "http://gateway.openfaas:8080" + } + endpoint := strings.TrimRight(gateway, "/") + + "/async-function/" + workerFunction + callback := r.Header.Get("X-Callback-Url") + + // Submit one asynchronous invocation for each item in the batch. + for i, item := range batch { + if err := submit(r.Context(), endpoint, item, callback); err != nil { + http.Error(w, fmt.Sprintf("item %d: %s", i+1, err), http.StatusBadGateway) + return + } + } + + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusAccepted) + json.NewEncoder(w).Encode(map[string]int{"submitted": len(batch)}) + } + + func submit(parent context.Context, endpoint, item, callback string) error { + ctx, cancel := context.WithTimeout(parent, submitTimeout) + defer cancel() + + req, err := http.NewRequestWithContext( + ctx, + http.MethodPost, + endpoint, + bytes.NewBufferString(item), + ) + if err != nil { + return err + } + req.Header.Set("Content-Type", "text/plain") + if callback != "" { + req.Header.Set("X-Callback-Url", callback) + } + + res, err := http.DefaultClient.Do(req) + if err != nil { + return err + } + defer res.Body.Close() + + if res.StatusCode != http.StatusAccepted { + return fmt.Errorf("queue returned %s", res.Status) + } + + return nil + } + ``` + +=== "Python" + + `fan-out/handler.py`: + + ```python + import json + import os + + import requests + + + WORKER_FUNCTION = "batch-worker" + SUBMIT_TIMEOUT = 5 + + + def handle(event, context): + try: + batch = json.loads(event.body) + except (TypeError, ValueError): + return error(400, "expected a JSON array of strings") + + if ( + not isinstance(batch, list) + or not batch + or not all(isinstance(item, str) for item in batch) + ): + return error(400, "expected a non-empty JSON array of strings") + + gateway = os.getenv("gateway_url", "http://gateway.openfaas:8080") + endpoint = f"{gateway.rstrip('/')}/async-function/{WORKER_FUNCTION}" + callback = event.headers.get("X-Callback-Url", "") + + # Submit one asynchronous invocation for each item in the batch. + for index, item in enumerate(batch): + headers = {"Content-Type": "text/plain"} + if callback: + headers["X-Callback-Url"] = callback + + try: + response = requests.post( + endpoint, + data=item.encode(), + headers=headers, + timeout=SUBMIT_TIMEOUT, + ) + except requests.RequestException as err: + return error(502, f"item {index + 1}: {err}") + + if response.status_code != 202: + return error( + 502, + f"item {index + 1}: queue returned {response.status_code}", + ) + + return {"statusCode": 202, "body": {"submitted": len(batch)}} + + + def error(status_code, message): + return {"statusCode": status_code, "body": message} + ``` + + Add `requests` to `fan-out/requirements.txt`. + +=== "Node.js" + + `fan-out/handler.js`: + + ```javascript + 'use strict' + + const workerFunction = 'batch-worker' + const submitTimeout = 5000 + + module.exports = async (event, context) => { + let batch + try { + batch = JSON.parse(requestBody(event.body)) + } catch (error) { + return fail(context, 400, 'expected a JSON array of strings') + } + + if (!Array.isArray(batch) || batch.length === 0 || + !batch.every(item => typeof item === 'string')) { + return fail(context, 400, 'expected a non-empty JSON array of strings') + } + + const gateway = process.env.gateway_url || + 'http://gateway.openfaas:8080' + const endpoint = `${gateway.replace(/\/$/, '')}/async-function/${workerFunction}` + const callback = String( + (event.headers || {})['x-callback-url'] || '' + ) + + // Submit one asynchronous invocation for each item in the batch. + for (const [index, item] of batch.entries()) { + const headers = { 'Content-Type': 'text/plain' } + if (callback) { + headers['X-Callback-Url'] = callback + } + + let response + try { + response = await fetch(endpoint, { + method: 'POST', + headers, + body: item, + signal: AbortSignal.timeout(submitTimeout) + }) + } catch (error) { + return fail(context, 502, `item ${index + 1}: ${error.message}`) + } + + if (response.status !== 202) { + return fail( + context, + 502, + `item ${index + 1}: queue returned ${response.status}` + ) + } + } + + return context + .status(202) + .headers({ 'Content-Type': 'application/json' }) + .succeed({ submitted: batch.length }) + } + + function requestBody (body) { + if (Buffer.isBuffer(body)) { + return body.toString() + } + return String(body || '') + } + + function fail (context, status, message) { + return context + .status(status) + .headers({ 'Content-Type': 'text/plain' }) + .succeed(message) + } + ``` + +The five-second submission timeout belongs to the `fan-out` implementation in +this example. It limits how long the function waits for the gateway to accept +each item; it does not limit how long `batch-worker` may run after the item has +been queued. + +## Configure the functions + +=== "Go" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + fan-out: + lang: golang-middleware + handler: ./fan-out + image: ttl.sh/openfaas-examples/fan-out:latest + + batch-worker: + lang: golang-middleware + handler: ./batch-worker + image: ttl.sh/openfaas-examples/batch-worker:latest + ``` + +=== "Python" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + fan-out: + lang: python3-http + handler: ./fan-out + image: ttl.sh/openfaas-examples/python-fan-out:latest + build_args: + TEST_ENABLED: "true" + + batch-worker: + lang: python3-http + handler: ./batch-worker + image: ttl.sh/openfaas-examples/python-batch-worker:latest + build_args: + TEST_ENABLED: "true" + ``` + +=== "Node.js" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + fan-out: + lang: node24 + handler: ./fan-out + image: ttl.sh/openfaas-examples/node-fan-out:latest + + batch-worker: + lang: node24 + handler: ./batch-worker + image: ttl.sh/openfaas-examples/node-batch-worker:latest + ``` + +## Deploy and submit a batch + +Build and deploy both functions: + +```bash +faas-cli up --tag=sha +``` + +Submit a batch: + +```bash +curl -i http://127.0.0.1:8080/function/fan-out \ + -H "Content-Type: application/json" \ + -d '["one", "two", "three"]' +``` + +The function confirms that all three items have been accepted by the queue: + +```text +HTTP/1.1 202 Accepted +Content-Type: application/json + +{"submitted":3} +``` + +The queue-worker then invokes `batch-worker` once for each item. Its response +for the first item is: + +```json +{ + "item": "one", + "processed": true +} +``` + +By default, responses from asynchronous invocations are discarded. To receive +each result, set `X-Callback-Url` on the request to `fan-out`. The function +forwards the header to each queued invocation, and the queue-worker sends each +result to that endpoint. Callbacks are independent and may arrive out of order. + +### Receive callback results + +Deploy the `printer` function from the store and follow its logs: + +```bash +faas-cli store deploy printer +faas-cli logs printer -t +``` + +In another terminal, submit the batch with a callback URL: + +```bash +curl -s http://127.0.0.1:8080/function/fan-out \ + -H "Content-Type: application/json" \ + -H "X-Callback-Url: http://gateway.openfaas:8080/function/printer" \ + -d '["one", "two", "three"]' +``` + +The `printer` logs receive one callback per item, with a body similar to: + +```json +{ + "item": "one", + "processed": true +} +``` + +When one action should run only after the complete batch has finished, use the +[fan-in pattern](/languages/patterns/fan-in/) to collect and combine the individual +results. + +## Fan-out considerations + +* Batch items must be independent. If one item needs another item's result, + coordinate them in sequence instead. +* The gateway returns an `X-Call-Id` for each accepted asynchronous + invocation. Retain these IDs when individual items need to be tracked or + cancelled with `DELETE /async-function/`. See + [cancel asynchronous invocations](/reference/async/#cancel-async-invocations) + for more information. +* The queue-worker controls how many items it + [processes concurrently](/reference/async/#parallelism), and OpenFaaS can + [autoscale](/architecture/autoscaling/) `batch-worker` when the load increases. +* The queue-worker handles + [retries for failed invocations](/openfaas-pro/retries/), allowing individual + batch items to recover from temporary errors. diff --git a/docs/languages/patterns/index.md b/docs/languages/patterns/index.md new file mode 100644 index 00000000..617ab259 --- /dev/null +++ b/docs/languages/patterns/index.md @@ -0,0 +1,237 @@ +OpenFaaS functions can be combined into workflows, run in the background or in +parallel, or use a fixed replica count. This page introduces common patterns +that work with any language template. + +| Pattern | Summary | Useful when | Example | +|---------|-----------|-------------|---------| +| [Director](#director-pattern) | One function coordinates several other functions | A workflow needs sequencing, branching, parallel function calls, or a combined response | [View example](/languages/patterns/director/) | +| [Async invocation](#async-invocation-pattern) | The gateway queues an invocation for async processing | Work should continue in the background, or queued processing is needed for back pressure | — | +| [Fan-out](#fan-out-pattern) | One invocation is divided into many independent asynchronous invocations | A batch needs concurrent processing that can scale across multiple function replicas | [View example](/languages/patterns/fan-out/) | +| [Singleton](#singleton-pattern) | A function is kept at one replica | State is tied to a connection, or the function must be limited to a single replica | [View example](/languages/patterns/singleton/) | + +The patterns can also be combined. For example, a director can fan out a batch +to another function, or an asynchronous invocation can start a director without +keeping the original caller connected. + +## Director pattern + +A director is an OpenFaaS function that coordinates a workflow. It invokes +other functions through the gateway, passes results between them, decides what +runs next, and returns or stores the final result. + +A director can invoke functions in sequence, choose the next function from an +earlier result, or run independent functions in parallel. It owns the workflow +and presents a single endpoint to the caller. + +```text + [ Client ] + │ + ▼ HTTP POST / invoke +┌──────────────────────┐ +│ Director Function │ ◄── Owns the workflow +└──────────┬───────────┘ + │ + ├── 1. Invoke ──► [ Function A ] + │ │ + │◄──── response ─────────┘ + │ + ├── 2. Branch on result + │ ├── if X ──► [ Function B ] + │ └── if Y ──► [ Function C ] + │ + ├── 3. Invoke independent functions in parallel + │ ├──► [ Function D ] ──┐ + │ └──► [ Function E ] ──┤ + │ ▼ + │ [ Combine results ] + │ + ▼ +[ Return response / persist result ] +``` + +The functions invoked by the director are deployed independently, so each can +use a different language, scale separately, and be updated without moving the +workflow logic out of the director + +**Useful when:** + +* Exposing a single endpoint for an operation that involves several functions +* Hiding the coordination between functions from the caller +* Coordinating functions that are developed, deployed, or scaled independently +* A workflow needs sequencing, branching, or independent functions to run + in parallel. + +**Design considerations:** + +* The director remains active while it waits for downstream functions to + finish. Its [timeouts](/tutorials/expanded-timeouts/) should be configured to + cover the longest expected path through the complete workflow, rather than a + single downstream function call. +* The director decides how to handle errors from the functions it invokes. It + can stop, retry, return a partial result, or save its progress. +* For a long-running workflow, the director can be + [invoked asynchronously](#async-invocation-pattern), so the caller does not + need to wait. The director can send the final result to a callback URL when + the workflow completes. + +**Examples:** + +* [Coordinate a workflow with a director](/languages/patterns/director/) + +## Async invocation pattern + +Any OpenFaaS function can be invoked asynchronously by changing the gateway +route from `/function/` to `/async-function/`. The caller sends an +HTTP POST and immediately receives `202 Accepted`. The request is added to a +queue and processed later by the queue-worker. + +This decouples the HTTP request from the function response, so the caller does +not need to wait while long-running work, background jobs, or batches are +processed. + +An optional callback can receive the result when processing +completes. + +```text + [ Client ] ──► [ Gateway ] ──► [ Queue ] + ▲ │ │ + └──── 202 ─────┘ ▼ + [ Queue-worker ] ──► [ Function ] + │ + └── callback ──► [ Result endpoint ] +``` + +**Useful when:** + +* Work takes longer than the caller should wait. +* Queued processing provides back pressure when requests arrive faster than + available capacity. +* Failed asynchronous invocations need to be + [retried automatically](/openfaas-pro/retries/) with a back-off. + +See [asynchronous functions](/reference/async/) for more information on +invocations, callbacks, retries and queue configuration. + +## Fan-out pattern + +Fan-out combines asynchronous invocation with independently scalable +functions. One function divides a batch into independent items and invokes a +worker function asynchronously for each item. It returns after the invocations +are queued rather than waiting for them to finish. + +The queue-worker drains the queued items as capacity becomes available, while +OpenFaaS can [autoscale the target function](/architecture/autoscaling/) across +multiple replicas. + +The queue-worker automatically +[retries failed invocations](/openfaas-pro/retries/), allowing batch items to +complete even when an invocation fails. + +```text + [ Input batch ] + │ + ▼ +┌────────────────────────┐ +│ Fan-out Function │ +└────────────┬───────────┘ + │ + ▼ +┌────────────────────────┐ +│ Queue-worker │ +└────────────┬───────────┘ + ├── item 1 / async ──► [ Function ] ──┐ + ├── item 2 / async ──► [ Function ] ──┤ + └── item N / async ──► [ Function ] ──┘ + │ callback + ▼ + [ Result collector ] +``` + +**Useful when:** + +* A batch contains records that can be processed independently. +* Processing should happen concurrently and can complete out of order. +* Processing capacity needs to scale across multiple worker function replicas, + independently of the function that submits the batch. + +**Design considerations:** + +* Items must be independent. If one item needs another item's result, process + them in sequence instead. +* Callbacks can deliver each result independently and may arrive in a different + order from the input. + +**Fan-in:** + +When the next step depends on every item completing, the individual +results can be fanned back in. One way to implement this is to: + +* Set a counter to the number of items in the batch. +* Store each callback result in shared storage and atomically decrement the + counter. +* When the counter reaches zero, invoke a final function to combine the stored + results, send a notification, or start the next step in the workflow. + +When results do not need to be combined, each callback can be handled +independently. See [fan-in](/languages/patterns/fan-in/) for the implementation. + +**Examples:** + +* [Process a batch with fan-out](/languages/patterns/fan-out/) +* [Combine individual results with fan-out and fan-in](https://www.openfaas.com/blog/fan-out-and-back-in-using-functions/). + +## Singleton pattern + +A singleton function is configured with a fixed replica count of one. It can be +useful when a workload keeps connection-local state, wraps software that cannot +run concurrently, or must limit access to an external resource. + +```text + [ Requests ] + │ + ▼ HTTP / invoke +┌──────────────────────────┐ +│ Singleton Function │ +│ │ +│ [ Replica 1 ] │ +└──────────────────────────┘ +``` + +Set the minimum and maximum replica labels to the same value to disable +horizontal scaling for the function: + +```yaml +functions: + stateful-function: + labels: + com.openfaas.scale.min: 1 + com.openfaas.scale.max: 1 +``` + +**Useful when:** + +* Reusing an expensive process-local resource, such as a database client, + downloaded dataset, or machine-learning model. +* Keeping stateful connections or subscribers in the same process. +* Running software that permits only one active instance. +* Maintaining a single session with an upstream service that limits active + clients. + +**Design considerations:** + +* A fixed replica count keeps one replica running during normal operation, but + does not make the function or its local filesystem durable. The replica can + be replaced during a restart, rescheduling, or deployment, so state that + needs to survive these events should still be stored externally. +* The function cannot add replicas to increase its capacity. +* If the goal is to process one request at a time, set the `max_inflight: 1` + environment variable on the function. Fixing the replica count alone does + not prevent concurrent requests within the replica. + +For more information on configuring function scaling, see the +[autoscaling documentation](/architecture/autoscaling/). + +**Examples:** + +* [Run a function as a singleton](/languages/patterns/singleton/) +* [Apply singleton scaling to WebSocket connections](https://www.openfaas.com/blog/serverless-websockets/#scaling-websockets). diff --git a/docs/languages/patterns/singleton.md b/docs/languages/patterns/singleton.md new file mode 100644 index 00000000..3053b916 --- /dev/null +++ b/docs/languages/patterns/singleton.md @@ -0,0 +1,309 @@ +The [Singleton pattern](/languages/patterns/#singleton-pattern) configures a +function with a desired replica count of one. With one replica, every request +or connection is routed to the same function process, allowing them to share +process-local state and reuse the same resources. + +This page uses a deliberately small example: a function performs simulated +expensive setup once when its process starts, then reuses the resulting +resource for every request. + +```text + ┌──────────────────────────┐ +[ Client A ] ──┐ │ Singleton Function │ +[ Client B ] ──┼── connections ────►│ │ +[ Client C ] ──┘ │ [ Replica 1 ] │ + └──────────────────────────┘ +``` + +This simple function highlights several properties of the Singleton pattern: + +* **One replica:** setting both replica limits to one disables horizontal + scaling, so OpenFaaS does not add replicas as load increases. +* **Reuse:** requests are routed to the same replica, so they can use the same + process-local resource. + +The example uses a two-second delay to simulate expensive setup. In a real +function, the same point in the lifecycle could be used to: + +* Create a database connection pool or an SDK client. +* Download and load a large dataset or machine-learning model. +* Open a persistent connection whose state remains in the function process. +* Establish a session with an upstream service that has costly setup or permits + only one active client. + +## Create the function + +Choose a language and scaffold the function: + +=== "Go" + + ```bash + faas-cli template store pull golang-middleware + + faas-cli new --lang golang-middleware singleton \ + --prefix ttl.sh/openfaas-examples + ``` + +=== "Python" + + ```bash + faas-cli template store pull python3-http + + faas-cli new --lang python3-http singleton \ + --prefix ttl.sh/openfaas-examples + ``` + +=== "Node.js" + + ```bash + faas-cli template store pull node24 + + faas-cli new --lang node24 singleton \ + --prefix ttl.sh/openfaas-examples + ``` + +The example uses the public [ttl.sh](https://ttl.sh) registry. Replace the +prefix with your own registry for production use. + +The full source code and `stack.yaml` files are available on GitHub for +[Go](https://github.com/openfaas/function-patterns/tree/master/go/singleton), +[Python](https://github.com/openfaas/function-patterns/tree/master/python/singleton), +and [Node.js](https://github.com/openfaas/function-patterns/tree/master/node/singleton). + +## Implement the function + +Each implementation starts the simulated setup when the function process +starts. The `/ready` path returns `200` only after the resource is available. +Normal requests then reuse the resource. + +=== "Go" + + `handler.go`: + + ```go + package function + + import ( + "fmt" + "net/http" + "time" + ) + + var expensiveResource string + + func init() { + // Initialize once when the function process starts, not on every request. + expensiveResource = setupExpensiveResource() + } + + func setupExpensiveResource() string { + // A real function could create a database client, open a persistent + // connection, or download a large dataset here. + time.Sleep(2 * time.Second) + return "The expensive resource is ready" + } + + func Handle(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/ready" { + fmt.Fprintln(w, "Ready") + return + } + + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + fmt.Fprintln(w, expensiveResource) + } + ``` + +=== "Python" + + `handler.py`: + + ```python + import time + + + def setup_expensive_resource(): + # A real function could create a database client, open a persistent + # connection, or download a large dataset here. + time.sleep(2) + return "The expensive resource is ready" + + + # Initialize once when the function process starts, not on every request. + expensive_resource = setup_expensive_resource() + + + def handle(event, context): + if event.path == "/ready": + return {"statusCode": 200, "body": "Ready"} + + return {"statusCode": 200, "body": expensive_resource} + ``` + +=== "Node.js" + + `handler.js`: + + ```javascript + 'use strict' + + let expensiveResource + let resourceReady = false + + async function setupExpensiveResource () { + // A real function could create a database client, open a persistent + // connection, or download a large dataset here. + await new Promise(resolve => setTimeout(resolve, 2000)) + return 'The expensive resource is ready' + } + + async function initialize () { + expensiveResource = await setupExpensiveResource() + resourceReady = true + } + + // Initialize once when the function process starts, not on every request. + initialize() + + module.exports = async (event, context) => { + // The readiness probe holds traffic until initialization has completed. + if (event.path === '/ready') { + return respond(context, resourceReady ? 200 : 503, + resourceReady ? 'Ready' : 'Initializing') + } + + if (!resourceReady) { + return respond(context, 503, 'Initializing') + } + + return respond(context, 200, expensiveResource) + } + + function respond (context, status, body) { + return context + .status(status) + .headers({ 'Content-Type': 'text/plain; charset=utf-8' }) + .succeed(body) + } + ``` + +## Configure the singleton and readiness + +=== "Go" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + singleton: + lang: golang-middleware + handler: ./singleton + image: ttl.sh/openfaas-examples/singleton:latest + labels: + com.openfaas.scale.min: "1" + com.openfaas.scale.max: "1" + annotations: + com.openfaas.ready.http.path: /ready + com.openfaas.ready.http.initialDelaySeconds: 1 + com.openfaas.ready.http.periodSeconds: 1 + ``` + +=== "Python" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + singleton: + lang: python3-http + handler: ./singleton + image: ttl.sh/openfaas-examples/python-singleton:latest + labels: + com.openfaas.scale.min: "1" + com.openfaas.scale.max: "1" + annotations: + com.openfaas.ready.http.path: /ready + com.openfaas.ready.http.initialDelaySeconds: 1 + com.openfaas.ready.http.periodSeconds: 1 + ``` + +=== "Node.js" + + `stack.yaml`: + + ```yaml + version: 1.0 + provider: + name: openfaas + gateway: http://127.0.0.1:8080 + functions: + singleton: + lang: node24 + handler: ./singleton + image: ttl.sh/openfaas-examples/node-singleton:latest + labels: + com.openfaas.scale.min: "1" + com.openfaas.scale.max: "1" + annotations: + com.openfaas.ready.http.path: /ready + com.openfaas.ready.http.initialDelaySeconds: 1 + com.openfaas.ready.http.periodSeconds: 1 + ``` + +Setting both replica limits to one disables horizontal scaling. With multiple +replicas, every process would perform the setup and keep its own copy of the +resource. See [autoscaling](/architecture/autoscaling/) for more detail. + +The readiness annotations direct the platform probe to `/ready`. They prevent +the replica from receiving traffic while initialization is still running. +For more information, see +[Health and readiness for functions](https://www.openfaas.com/blog/health-and-readiness-for-functions/) +and [custom HTTP health checks](/reference/workloads/#custom-http-health-checks). + +## Deploy and invoke + +Build and deploy the function: + +```bash +faas-cli up --tag=sha +``` + +Confirm that the function is deployed and has one available replica: + +```bash +faas-cli list -v +``` + +Invoke the function after it becomes ready: + +```bash +curl http://127.0.0.1:8080/function/singleton +``` + +The function should return: + +```text +The expensive resource is ready +``` + +## Singleton considerations + +* Process-local resources follow the replica lifecycle. Initialization runs + again after a deployment, restart, or rescheduling, and local state is lost + when the replica is replaced. Store durable state in external persistent + storage. +* A single replica can still process multiple requests concurrently, so shared + resources must be safe for concurrent use. Set `max_inflight: "1"` to process + one request at a time; it limits concurrency, not the replica count. See + [concurrent request limits](/architecture/invocations/#how-many-times-can-a-function-be-invoked). +* Stateful connections such as WebSockets are another use of the Singleton + pattern because their connection state remains in one process. See + [How to Integrate WebSockets with Serverless Functions and OpenFaaS](https://www.openfaas.com/blog/serverless-websockets/). diff --git a/docs/languages/python/index.md b/docs/languages/python/index.md index 1c42297d..e87da0d8 100644 --- a/docs/languages/python/index.md +++ b/docs/languages/python/index.md @@ -484,6 +484,9 @@ def handle(event, context): ## Examples +* [The director pattern](/languages/patterns/director/) +* [The fan-out pattern](/languages/patterns/fan-out/) +* [The singleton pattern](/languages/patterns/singleton/) * [Deploy a function via the OpenFaaS API](examples/openfaas-api.md) * [Access AWS S3 with boto3](examples/s3-boto3.md) * [Call the OpenAI Chat API](examples/openai.md) diff --git a/mkdocs.yml b/mkdocs.yml index 04776b8d..be3d9711 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -53,6 +53,7 @@ theme: tabs: false features: - content.code.copy + - content.tabs.link palette: primary: blue accent: indigo @@ -105,7 +106,7 @@ markdown_extensions: - pymdownx.smartsymbols - pymdownx.superfences - pymdownx.tabbed: - alternate_style: true + alternate_style: true - pymdownx.tasklist: custom_checkbox: true - pymdownx.tilde @@ -132,6 +133,12 @@ nav: - Shell auto completion: ./cli/completion.md - Languages: - Overview: ./languages/overview.md + - Function patterns: + - Overview: ./languages/patterns/index.md + - Director: ./languages/patterns/director.md + - Fan-out: ./languages/patterns/fan-out.md + - Fan-in: ./languages/patterns/fan-in.md + - Singleton: ./languages/patterns/singleton.md - Python: - Overview: ./languages/python/index.md - Examples: @@ -146,7 +153,7 @@ nav: - Stream Server-Sent Events (SSE): ./languages/python/examples/sse.md - Readiness checks for slow start-up: ./languages/python/examples/readiness.md - Node: ./languages/node.md - - Go: ./languages/go.md + - Go: ./languages/go/index.md - C#: ./languages/csharp.md - PHP: ./languages/php.md - Dockerfile: ./languages/dockerfile.md @@ -158,7 +165,7 @@ nav: - Retries: ./openfaas-pro/retries.md - IAM & Policy: - Overview: ./openfaas-pro/iam/overview.md - - IAM Example: ./openfaas-pro/iam/example-auth0.md + - IAM Example: ./openfaas-pro/iam/example-auth0.md - Function Authentication: ./openfaas-pro/iam/function-authentication.md - GitHub Actions Federation: ./openfaas-pro/iam/github-actions-federation.md - GitLab Federation: ./openfaas-pro/iam/gitlab-federation.md