diff --git a/docs/plan.md b/docs/plan.md index c9ba9b6..141610a 100644 --- a/docs/plan.md +++ b/docs/plan.md @@ -197,6 +197,7 @@ The v1.2 design in §4 to §6 stays as reference. A piece of it is built only wh | `games-service` | A new `games` service answers the games I played in the last 14 days at `GET /games/recent`, from the Exophase profile that `BAKHYARD_EXOPHASE_PLAYER` names, digits only; it runs on 8090 under `mise run up`, and its image builds with the others'. The fourth of `games-exophase`'s eleven steps (decision 303) | Done (#178) | | `twitch-token` | The `games` service gets a Twitch app token for IGDB with the client credentials grant, keeps it in memory until 9 tenths of its life, asks Twitch one request at a time, and forgets a token that IGDB refuses. The fifth of `games-exophase`'s eleven steps (decision 304) | Done (#181) | | `igdb-search` | The `games` service's IGDB client finds a title's cover: the top result of a search filtered to main games, expansions, remakes, remasters, expanded games and ports, or a game named exactly the title when the top result has another name; it asks one request at a time, at least 250 ms apart, and with a new token once when IGDB refuses one. The sixth of `games-exophase`'s eleven steps (decision 305) | Done (#183) | +| `games-covers` | Each game in the `games` service's answer carries its IGDB cover when `BAKHYARD_TWITCH_CLIENT_ID` and the Keychain's `twitch-client-secret` are set: each title's cover, or that IGDB has none, is kept a week, the first failure stops the asking until the next fetch, and a missing cover never fails the list. The seventh of `games-exophase`'s eleven steps (decision 306) | Done (#186) | | `vault-edit-sync` | Editing the vault from Bakhyard (§8.2 Phase 6), by the vault session in `../bakhyard-vault/`, which adds its steps' rows under this one and then does `copilot-planner` (decision 195) | In progress | | `vault-plan` | The vault session's first steps: where a save goes waits on the user's answer to §12's question on how edits get back to the Mac, so the version that a save checks and the editor come first; a note's version is the SHA-256 of its bytes, sent as its `ETag`, and a save is a `PUT` that sends it back in `If-Match` (decision 400) | Done (#167) | | `note-etag` | notes answers each note with its version as a strong `ETag`, the SHA-256 of its bytes, and a matching `If-None-Match` with a 304 (decision 400) | Done (#169) | @@ -1281,7 +1282,7 @@ Every item is pending unless it says done. The service an item lands in is in br - `games-service`: a new `games` service answers the games I played in the last 14 days at `GET /games/recent` (decision 303). Done (#178). - `twitch-token`: the service gets a Twitch app token for IGDB, keeps it and renews it before it expires (decision 304). Done (#181). - `igdb-search`: the service's IGDB client finds a title's cover with a filtered search and an exact-name fallback (decisions 68 and 305), within IGDB's 4 requests a second. Done (#183). - - `games-covers`: each game in the answer carries its IGDB cover; each title is matched once and the match kept. + - `games-covers`: each game in the answer carries its IGDB cover; each title is matched once and the match kept for a week (decision 306). Done (#186). - `homelab-games`: in the homelab repo, `games` runs on the laptop with its settings and secret, and the gateway's peers name it. - `games-gateway`: the gateway forwards `/api/games` to `games`. - `games-page`: the Games page lists the games, and marks them stale when the latest fetch failed (decision 67); the Games card's description in the app becomes §7's. @@ -1816,6 +1817,15 @@ Decisions 300 to 399 are the `games-exophase` session's. - **Not yet checked live**: the queries and headers follow the spike, IGDB's docs, and another IGDB client's working use of `name ~` without wildcards. IGDB itself waits for the user's Twitch app, which `games-covers` needs. - **Rejected**: the exact-name search first, which would take any record with that name, such as an edition or a bundle, before the filtered search's better match; the fallback only when the filtered search finds nothing, which would keep the spike's 15th title's wrong match; requests in parallel up to IGDB's 8 open, which would need a rate limiter to save a few seconds on a dozen titles, each matched once; `golang.org/x/time/rate`, a new module for one gap; stripping the quotes from a title rather than escaping them, which would change the exact name; covers that Bakhyard stores and serves, which IGDB already serves. +306. **The covers** (`games-covers`): the lead's choices (2026-10-30), for the user to review. + - **The answer**: each game in `GET /games/recent` carries `cover`, the address of its cover on IGDB (decision 305), left out when it has none, as `playtimeMinutes` and `trophies` are. `api.Game` embeds `exophase.Game` and adds the cover, so each game stays one flat object and the Exophase client knows nothing of IGDB. + - **The settings**: as decision 300 has them. Without `BAKHYARD_TWITCH_CLIENT_ID`, the games come without covers and no secret is read; with it but no secret, the service exits 1 with `read the Twitch client secret:` and why. The ID isn't checked: Twitch refuses one it doesn't know, and the log says so. The secret is read as the service starts, even when Exophase is off. + - **The matches**: `services/games/internal/covers` keeps each title's cover, or that IGDB has none, for a week from when it asked, well within the 30 days that IGDB keeps an image it replaces (decision 305), so a new cover shows within a week. Only the latest list's titles stay, so the matches never outgrow the list. They live in memory (decision 300). + - **When**: as the list is fetched (decision 300), once Exophase has answered, a title at a time in the list's order, so the games played last get their covers first; a title that the list names twice is asked once. Fetches never overlap, since the cache polls one at a time (decision 302), so the matches need no lock. + - **Failures**: the first error stops the asking for that fetch. The games not asked yet, and the one that failed, go without covers, while the covers kept still show. The list itself doesn't fail, so the page shows no failure, and the next fetch, an hour later, asks again. The service logs `find covers` with the error, as home's feed logs `list the items`, unless the fetch's context has ended. + - **A known limit**: a title whose search alone fails, rather than IGDB as a whole, stops the asking at the same place at each fetch, so the titles after it get no cover until it leaves the list; the log names it. + - **Rejected**: the cover in `exophase.Game`, which would tie the Exophase client to IGDB; failing the fetch for a missing cover, which would keep the last list, stale, for want of a picture; asking on past a failure, which would add up to 10 seconds a title while IGDB or Twitch is down; finding the covers after answering, which would show them only with the next fetch, an hour later; keeping errors, which would hide a recovered IGDB until they expire; a `sync.Mutex`, which nothing contends for; an expired match as a fallback when asking fails, which would keep a second kind of match for a rare case. + Decisions 400 to 499 are the vault session's: `vault-edit-sync`, then `copilot-planner`. 400. **Editing notes, the first steps** (`vault-plan`, `note-etag`, `note-editor`): the lead's choices (2026-10-30), for the user to review. diff --git a/docs/setup.md b/docs/setup.md index 8221ed1..ddd6580 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -26,11 +26,12 @@ cp mise.local.example.toml mise.local.toml | `BAKHYARD_BITBUCKET_WORKSPACES` | pulls | The slugs of the workspaces to list your open pull requests in, separated by commas | | `BAKHYARD_GITHUB_API` | github | GitHub's REST API, `https://api.github.com`, where github searches for your open pull requests, review requests and issues ([below](#github)) | | `BAKHYARD_EXOPHASE_PLAYER` | games | The id of the Exophase profile whose games to list, digits only ([below](#exophase)) | +| `BAKHYARD_TWITCH_CLIENT_ID` | games | The client ID of the Twitch app with which games asks IGDB for the games' covers ([below](#igdb)) | | `BAKHYARD_TEST_DATABASE` | Go tests | The test database's URL, password included ([below](#the-test-database)) | -The gateway doesn't start without its four settings, nor notes without its vault: each exits 1 and says what's missing. The gateway also waits for notes, notify, nudge, pulls, time and todos to start. Todos and time run without Jira when none of the three Jira settings is set, pulls without Bitbucket when none of the three Bitbucket settings is set, github without GitHub when `BAKHYARD_GITHUB_API` isn't set, and games without Exophase when `BAKHYARD_EXOPHASE_PLAYER` isn't set. +The gateway doesn't start without its four settings, nor notes without its vault: each exits 1 and says what's missing. The gateway also waits for notes, notify, nudge, pulls, time and todos to start. Todos and time run without Jira when none of the three Jira settings is set, pulls without Bitbucket when none of the three Bitbucket settings is set, github without GitHub when `BAKHYARD_GITHUB_API` isn't set, and games without Exophase when `BAKHYARD_EXOPHASE_PLAYER` isn't set and without covers when `BAKHYARD_TWITCH_CLIENT_ID` isn't set. -On the Mac, the gateway, todos, pulls, time and github read their secrets from the login Keychain, as the sections below show. On the laptop, which has no Keychain, `BAKHYARD_SECRETS` names a folder of files instead, each named as its secret's Keychain account (`google-client-secret`, `postgres-gateway`, `postgres-todos`, `postgres-notify`, `postgres-nudge`, `vapid-key`, `jira-token`, `bitbucket-token`, `github-token`) and holding the secret exactly, with no newline at the end: the files of the Kubernetes Secrets mounted there (decision 154). Leave it unset on the Mac. +On the Mac, the gateway, todos, pulls, time, github and games read their secrets from the login Keychain, as the sections below show. On the laptop, which has no Keychain, `BAKHYARD_SECRETS` names a folder of files instead, each named as its secret's Keychain account (`google-client-secret`, `postgres-gateway`, `postgres-todos`, `postgres-notify`, `postgres-nudge`, `vapid-key`, `jira-token`, `bitbucket-token`, `github-token`, `twitch-client-secret`) and holding the secret exactly, with no newline at the end: the files of the Kubernetes Secrets mounted there (decision 154). Leave it unset on the Mac. ## Tailscale @@ -189,6 +190,22 @@ Games lists the games you played in the last 14 days, the last played first, wit The id must be digits only; with anything else, games exits 1 and says so: `BAKHYARD_EXOPHASE_PLAYER: "…" is not an Exophase profile id, which is digits only`. Without it, `GET /games/recent` answers 404 `Exophase is not set up`. `mise run up` runs games on port 8090, where `GET /games/recent` answers the games, with when the latest fetch to succeed began and why the latest fetch failed, if it did. Games fetches the list only when asked: the first request waits for the first fetch, and later ones get the list at once, while one that finds it an hour old or more starts a fetch in the background. When a fetch fails, for example while Exophase is down, games keeps answering the last list it got, adds the failure, such as `list my Exophase games: 500 Internal Server Error`, which is also what Exophase answers for an id it doesn't know, `list my Exophase games: 403 Forbidden, a Cloudflare challenge` or, for a profile that isn't public, `list my Exophase games: Exophase has no public profile with this id`, and logs `poll Exophase` with it. The next fetch then waits 5 minutes, and twice as long after each failure in a row, up to 6 hours. The list lives in games' memory, so a restart fetches it again. +## IGDB + +Games shows each game's cover from IGDB, which it asks as a Twitch app (decisions 68 and 304 to 306). IGDB's API is free for non-commercial use, under the terms of the Twitch Developer Service Agreement. To set it up: + +1. Turn on two-factor authentication for your Twitch account, under [Security and Privacy](https://www.twitch.tv/settings/security): Twitch's developer console needs it. +2. In the [developer console](https://dev.twitch.tv/console), on the Applications tab, select Register Your Application. Give the app a name that no other Twitch app has, add the OAuth redirect URL `http://localhost`, which IGDB doesn't use, pick any category and the client type Confidential, then select Create. Select Manage beside the new app: the page shows its client ID. Select New Secret and copy the secret. +3. Put the client secret in the Keychain. With `-w` last, `security` asks for it: + + ```sh + security add-generic-password -s bakhyard -a twitch-client-secret -w + ``` + +4. In `mise.local.toml`, set `BAKHYARD_TWITCH_CLIENT_ID` to the app's client ID, then restart games. + +Without the setting, the games come without covers. With it but no secret in the Keychain, games exits 1 and says so: `read the Twitch client secret: no Keychain item with service bakhyard and account twitch-client-secret`. With both, each game in `GET /games/recent` carries `cover`, the address of its cover on IGDB, unless IGDB has none; the app loads the image from IGDB. Games finds the covers as it fetches the list, a title at a time and at most four requests a second, so the first request after a start waits for them too. It keeps each title's cover, or that IGDB has none, for a week. When IGDB or Twitch fails, the games whose covers it hasn't found go without until the next fetch, and games logs `find covers` with the failure, such as `find the cover of "…": ask IGDB: 503 Service Unavailable` or, for a client ID that Twitch doesn't know, `find the cover of "…": get a Twitch token: 400 Bad Request, invalid client`. The covers and Twitch's token live in games' memory, so a restart asks again. A new secret makes the old one fail at once: store it with `-U` added to the command in step 3, then restart games. + ## Run `mise run up` builds the services and runs them with process-compose. A service that exits restarts after 1 s, so a missing setting or secret shows up in that service's log every second until it's fixed. diff --git a/mise.local.example.toml b/mise.local.example.toml index 47524d4..ef26cd6 100644 --- a/mise.local.example.toml +++ b/mise.local.example.toml @@ -33,5 +33,8 @@ BAKHYARD_VAULT = "/Users/you/Documents/Vault" # The id of the Exophase profile whose games the games service lists, digits only: the # playerProfileId in the profile page's source. Leave it out to run without Exophase. # BAKHYARD_EXOPHASE_PLAYER = "1234567" +# The client ID of the Twitch app with which the games service asks IGDB for the games' covers; +# its client secret goes into the Keychain. Leave it out to run without covers. +# BAKHYARD_TWITCH_CLIENT_ID = "0123456789abcdefghijklmnopqrst" # The Go tests' own database. Put its role's password in place of . BAKHYARD_TEST_DATABASE = "postgres://bakhyard_test:@127.0.0.1:5433/bakhyard_test" diff --git a/process-compose.yaml b/process-compose.yaml index f09be10..d11aff1 100644 --- a/process-compose.yaml +++ b/process-compose.yaml @@ -143,7 +143,7 @@ processes: games: <<: *service - description: Lists the games I played lately on Exophase + description: Lists the games I played lately on Exophase, with their IGDB covers command: bin/games environment: - BAKHYARD_LISTEN=tcp://127.0.0.1:8090 diff --git a/services/games/covers.go b/services/games/covers.go new file mode 100644 index 0000000..4ad6609 --- /dev/null +++ b/services/games/covers.go @@ -0,0 +1,72 @@ +package main + +import ( + "context" + "fmt" + "log/slog" + + "github.com/codedogapp/bakhyard/internal/secrets" + "github.com/codedogapp/bakhyard/services/games/internal/api" + "github.com/codedogapp/bakhyard/services/games/internal/covers" + "github.com/codedogapp/bakhyard/services/games/internal/exophase" + "github.com/codedogapp/bakhyard/services/games/internal/igdb" + "github.com/codedogapp/bakhyard/services/games/internal/twitch" +) + +// twitchClientIDVar is the setting of the client ID of the Twitch app with which games asks IGDB +// for the games' covers (docs/setup.md). The games come without covers when it isn't set. +const twitchClientIDVar = "BAKHYARD_TWITCH_CLIENT_ID" + +// twitchAccount is the Keychain account of the Twitch app's client secret, and the name of its +// file (decision 300). +const twitchAccount = "twitch-client-secret" + +// coverCache returns the cache of the covers that IGDB finds for the Twitch app in the settings, +// which gets its tokens from Twitch with the secret that readSecret reads, or nil when the app +// isn't set. +func coverCache( + ctx context.Context, getenv func(string) string, readSecret secrets.Reader, addresses apis, +) (*covers.Cache, error) { + clientID := getenv(twitchClientIDVar) + if clientID == "" { + return nil, nil + } + + secret, err := readSecret(ctx, twitchAccount) + if err != nil { + return nil, fmt.Errorf("read the Twitch client secret: %w", err) + } + + tokens := twitch.New(addresses.twitch, clientID, secret) + + return covers.New(igdb.New(addresses.igdb, clientID, tokens)), nil +} + +// covered returns games, each with its cover from cache, or with none when cache is nil or finds +// none. It logs to logger why cache found no more covers, unless ctx has ended. +func covered( + ctx context.Context, games []exophase.Game, cache *covers.Cache, logger *slog.Logger, +) []api.Game { + answered := make([]api.Game, len(games)) + titles := make([]string, len(games)) + + for i, game := range games { + answered[i].Game = game + titles[i] = game.Title + } + + if cache == nil { + return answered + } + + found, err := cache.Find(ctx, titles) + if err != nil && ctx.Err() == nil { + logger.ErrorContext(ctx, "find covers", "err", err) + } + + for i, cover := range found { + answered[i].Cover = cover + } + + return answered +} diff --git a/services/games/covers_test.go b/services/games/covers_test.go new file mode 100644 index 0000000..b528ec5 --- /dev/null +++ b/services/games/covers_test.go @@ -0,0 +1,239 @@ +package main + +import ( + "bytes" + "context" + "encoding/json/v2" + "errors" + "io" + "log/slog" + "net/http" + "net/http/httptest" + "net/url" + "reflect" + "slices" + "strings" + "sync" + "testing" + "time" + + "github.com/codedogapp/bakhyard/services/games/internal/api" + "github.com/codedogapp/bakhyard/services/games/internal/covers" + "github.com/codedogapp/bakhyard/services/games/internal/exophase" +) + +func TestGamesComeWithTheirCovers(t *testing.T) { + addresses, asked := coversStandIn(t, map[string]string{ + "An hour ago": "co1", "13 days and 23 hours ago": "co2", + }) + addresses.exophase, _ = exophaseStandIn(t, time.Now()) + + router, err := routes(t.Context(), + envOf(map[string]string{exophasePlayerVar: "123", twitchClientIDVar: "app"}), + secretsWith(map[string]string{twitchAccount: "s3cret"}), + addresses) + if err != nil { + t.Fatal(err) + } + + req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/games/recent", nil) + rec := httptest.NewRecorder() + router.ServeHTTP(rec, req) + + var answer struct { + Games []struct { + Title string `json:"title"` + Cover string `json:"cover"` + } `json:"games"` + } + + if err := json.Unmarshal(rec.Body.Bytes(), &answer); err != nil { + t.Fatalf("got %d %s: %v", rec.Code, rec.Body, err) + } + + got := make([]string, 0, len(answer.Games)) + for _, game := range answer.Games { + got = append(got, game.Title+": "+game.Cover) + } + + want := []string{ + "An hour ago: https://images.igdb.com/igdb/image/upload/t_cover_big/co1.jpg", + "13 days and 23 hours ago: https://images.igdb.com/igdb/image/upload/t_cover_big/co2.jpg", + } + if !slices.Equal(got, want) { + t.Errorf("got %s, want the games and covers %q", rec.Body, want) + } + + wantAsked := []string{ + "Twitch: app s3cret", + "IGDB: app, Bearer token", + "IGDB: app, Bearer token", + } + if got := asked(); !slices.Equal(got, wantAsked) { + t.Errorf("got the requests %q, want %q", got, wantAsked) + } +} + +func TestGamesDoesNotStartWithoutTheTwitchSecret(t *testing.T) { + router, err := routes(t.Context(), envOf(map[string]string{twitchClientIDVar: "app"}), + secretsWith(nil), exophaseAt(unasked())) + + want := "read the Twitch client secret: no secret twitch-client-secret" + if router != nil || err == nil || err.Error() != want { + t.Errorf("got %v, %v, want no handler and the error %q", router, err, want) + } +} + +func TestCoversAreOffWithoutTheTwitchApp(t *testing.T) { + cache, err := coverCache(t.Context(), envOf(nil), secretsWith(nil), exophaseAt(unasked())) + if cache != nil || err != nil { + t.Errorf("got %v, %v, want no cache and no error", cache, err) + } +} + +func TestCoveredGivesNoCoversWithoutTheCache(t *testing.T) { + games := []exophase.Game{{Title: "A", Platforms: []string{"PS5"}}, {Title: "B"}} + + got := covered(t.Context(), games, nil, slog.New(slog.DiscardHandler)) + + want := []api.Game{{Game: games[0]}, {Game: games[1]}} + if !reflect.DeepEqual(got, want) { + t.Errorf("got %v, want %v", got, want) + } +} + +func TestCoveredLogsWhyItFoundNoMoreCovers(t *testing.T) { + cache := covers.New(finder(func(_ context.Context, title string) (string, error) { + if title == "B" { + return "", errors.New("IGDB is down") + } + + return title + ".jpg", nil + })) + games := []exophase.Game{{Title: "A"}, {Title: "B"}, {Title: "C"}} + + var log bytes.Buffer + + got := covered(t.Context(), games, cache, slog.New(slog.NewTextHandler(&log, nil))) + + want := []api.Game{{Game: games[0], Cover: "A.jpg"}, {Game: games[1]}, {Game: games[2]}} + if !reflect.DeepEqual(got, want) { + t.Errorf("got %v, want %v", got, want) + } + + if wantLog := `level=ERROR msg="find covers" err="IGDB is down"`; !strings.Contains( + log.String(), wantLog) { + t.Errorf("logged %q, want %q", log.String(), wantLog) + } +} + +func TestCoveredLogsNothingWithoutAFailureToTell(t *testing.T) { + tests := map[string]struct { + ended bool + want []api.Game + }{ + "every cover found": {want: []api.Game{{Title: "A", Cover: "A.jpg"}}}, + "the context has ended": {ended: true, want: []api.Game{{Title: "A"}}}, + } + + for name, test := range tests { + t.Run(name, func(t *testing.T) { + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + + if test.ended { + cancel() + } + + cache := covers.New(finder(func(ctx context.Context, title string) (string, error) { + return title + ".jpg", ctx.Err() + })) + + var log bytes.Buffer + + got := covered( + ctx, + []exophase.Game{{Title: "A"}}, + cache, + slog.New(slog.NewTextHandler(&log, nil)), + ) + + if !reflect.DeepEqual(got, test.want) { + t.Errorf("got %v, want %v", got, test.want) + } + + if log.Len() != 0 { + t.Errorf("logged %q, want nothing", log.String()) + } + }) + } +} + +// finder finds covers as the function does. +type finder func(ctx context.Context, title string) (string, error) + +func (f finder) Cover(ctx context.Context, title string) (string, error) { + return f(ctx, title) +} + +// coversStandIn starts stand-ins for Twitch's OAuth server, which gives the token "token", and +// for IGDB, whose game of each title in images is so named and has the cover with the image id +// that images maps the title to. It returns their addresses, with no Exophase, and a function that +// returns the client credentials that each request came with, in order. +func coversStandIn(t *testing.T, images map[string]string) (apis, func() []string) { + t.Helper() + + var ( + lock sync.Mutex + requests []string + ) + + record := func(request string) { + lock.Lock() + defer lock.Unlock() + + requests = append(requests, request) + } + + twitchAPI := http.NewServeMux() + twitchAPI.HandleFunc("POST /oauth2/token", func(w http.ResponseWriter, r *http.Request) { + record("Twitch: " + r.PostFormValue("client_id") + " " + r.PostFormValue("client_secret")) + + _, _ = io.WriteString(w, `{"access_token":"token","expires_in":5000000}`) + }) + + igdbAPI := http.NewServeMux() + igdbAPI.HandleFunc("POST /v4/games", func(w http.ResponseWriter, r *http.Request) { + record("IGDB: " + r.Header.Get("Client-ID") + ", " + r.Header.Get("Authorization")) + + query, _ := io.ReadAll(r.Body) + _, quoted, _ := strings.Cut(string(query), `"`) + title, _, _ := strings.Cut(quoted, `"`) + + _ = json.MarshalWrite(w, []map[string]any{ + {"name": title, "cover": map[string]string{"image_id": images[title]}}, + }) + }) + + return apis{twitch: serve(t, twitchAPI), igdb: serve(t, igdbAPI)}, func() []string { + lock.Lock() + defer lock.Unlock() + + return slices.Clone(requests) + } +} + +// serve starts a server of handler and returns its address. +func serve(t *testing.T, handler http.Handler) *url.URL { + t.Helper() + + server := httptest.NewServer(handler) + t.Cleanup(server.Close) + + address, err := url.Parse(server.URL) + if err != nil { + t.Fatal(err) + } + + return address +} diff --git a/services/games/exophase.go b/services/games/exophase.go index 943de7c..d2fb233 100644 --- a/services/games/exophase.go +++ b/services/games/exophase.go @@ -9,6 +9,7 @@ import ( "time" "github.com/codedogapp/bakhyard/services/games/internal/api" + "github.com/codedogapp/bakhyard/services/games/internal/covers" "github.com/codedogapp/bakhyard/services/games/internal/exophase" "github.com/codedogapp/bakhyard/services/games/internal/refresh" ) @@ -37,11 +38,12 @@ func exophaseClient(getenv func(string) string, address *url.URL) (*exophase.Cli } // poll returns what keeps the games I played in the last 14 days, which it lists with client -// whenever a request finds them stale, until ctx ends, or nil when Exophase is off and client is -// nil. +// whenever a request finds them stale, until ctx ends, with their covers from cache, or nil when +// Exophase is off and client is nil. func poll( ctx context.Context, client *exophase.Client, + cache *covers.Cache, logger *slog.Logger, ) *refresh.OnRequest[api.Recent] { if client == nil { @@ -50,8 +52,11 @@ func poll( recent := refresh.New("Exophase", func(ctx context.Context) (api.Recent, error) { games, err := client.PlayedSince(ctx, time.Now().Add(-recentWindow)) + if err != nil { + return api.Recent{}, err + } - return api.Recent{Games: games}, err + return api.Recent{Games: covered(ctx, games, cache, logger)}, nil }, logger) go recent.Run(ctx) diff --git a/services/games/exophase_test.go b/services/games/exophase_test.go index 7fc21e5..7e5cbb4 100644 --- a/services/games/exophase_test.go +++ b/services/games/exophase_test.go @@ -15,7 +15,8 @@ import ( func TestGamesListsTheGamesIPlayedInTheLast14Days(t *testing.T) { address, asked := exophaseStandIn(t, time.Now()) - router, err := routes(t.Context(), envOf(map[string]string{exophasePlayerVar: "123"}), address) + router, err := routes(t.Context(), envOf(map[string]string{exophasePlayerVar: "123"}), + secretsWith(nil), exophaseAt(address)) if err != nil { t.Fatal(err) } @@ -62,7 +63,8 @@ func TestGamesTellsWhyExophaseFailed(t *testing.T) { t.Fatal(err) } - router, err := routes(t.Context(), envOf(map[string]string{exophasePlayerVar: "123"}), address) + router, err := routes(t.Context(), envOf(map[string]string{exophasePlayerVar: "123"}), + secretsWith(nil), exophaseAt(address)) if err != nil { t.Fatal(err) } diff --git a/services/games/internal/api/api.go b/services/games/internal/api/api.go index 87632e6..dbcf177 100644 --- a/services/games/internal/api/api.go +++ b/services/games/internal/api/api.go @@ -13,7 +13,16 @@ import ( // Recent is what GET /games/recent answers beside when the latest poll to succeed began and why // the latest poll failed: the games I played in the last 14 days, the last played first. type Recent struct { - Games []exophase.Game `json:"games"` + Games []Game `json:"games"` +} + +// Game is a game in the answer: what Exophase says of it, and its cover. +type Game struct { + exophase.Game + + // Cover is the address of the game's cover on IGDB, or "" when it has none or IGDB isn't set + // up. + Cover string `json:"cover,omitzero"` } // Register serves GET /games/recent on router, which answers what recent has found on Exophase, diff --git a/services/games/internal/api/api_test.go b/services/games/internal/api/api_test.go index 67eb8c9..a037261 100644 --- a/services/games/internal/api/api_test.go +++ b/services/games/internal/api/api_test.go @@ -17,6 +17,11 @@ import ( ) func TestRecentAnswersWhatWasFound(t *testing.T) { + hades := exophase.Game{ + Title: "Hades", Platforms: []string{"PS5"}, + LastPlayed: time.Date(1999, 12, 31, 22, 0, 0, 0, time.UTC), + } + for _, test := range []struct { name string found api.Recent @@ -25,13 +30,16 @@ func TestRecentAnswersWhatWasFound(t *testing.T) { {"no games", api.Recent{}, `{"games":[],"searched":"2000-01-01T00:00:00Z"}`}, { "a game", - api.Recent{Games: []exophase.Game{{ - Title: "Hades", Platforms: []string{"PS5"}, - LastPlayed: time.Date(1999, 12, 31, 22, 0, 0, 0, time.UTC), - }}}, + api.Recent{Games: []api.Game{{Game: hades}}}, `{"games":[{"title":"Hades","platforms":["PS5"],"lastPlayed":"1999-12-31T22:00:00Z"}],` + `"searched":"2000-01-01T00:00:00Z"}`, }, + { + "a game with its cover", + api.Recent{Games: []api.Game{{Game: hades, Cover: "https://images.example/hades.jpg"}}}, + `{"games":[{"title":"Hades","platforms":["PS5"],"lastPlayed":"1999-12-31T22:00:00Z",` + + `"cover":"https://images.example/hades.jpg"}],"searched":"2000-01-01T00:00:00Z"}`, + }, } { t.Run(test.name, func(t *testing.T) { // A bubble's clock starts at midnight UTC on 1 January 2000. diff --git a/services/games/internal/covers/covers.go b/services/games/internal/covers/covers.go new file mode 100644 index 0000000..ca88823 --- /dev/null +++ b/services/games/internal/covers/covers.go @@ -0,0 +1,86 @@ +// Package covers keeps the covers found for the games' titles, each for a week at most +// (docs/plan.md decisions 68, 300 and 306). +package covers + +import ( + "context" + "time" +) + +// keepFor is how long a title's cover is kept: a week, well within the 30 days for which IGDB +// keeps an image that it has removed or replaced. +const keepFor = 7 * 24 * time.Hour + +// Finder finds the address of a title's cover, or "" when the title has none. +type Finder interface { + Cover(ctx context.Context, title string) (string, error) +} + +// Cache keeps the covers that its Finder finds for the titles of the latest list. Its Find must +// not run while another runs, as the games' fetches don't. +type Cache struct { + finder Finder + matches map[string]match +} + +// match is a title's cover, or "" when it has none, and when the Finder was asked for it. +type match struct { + cover string + asked time.Time +} + +// New returns a Cache, with nothing kept yet, of the covers that finder finds. +func New(finder Finder) *Cache { + return &Cache{finder: finder, matches: map[string]match{}} +} + +// Find returns the cover of each title in titles, in order: the one kept when the Finder was asked +// for it less than a week ago, or else the one that it finds now. It asks for the titles one after +// another, each once, and then keeps the covers of these titles only. At the Finder's first error +// it asks no more, and returns the error with the covers it has, "" for the others. +func (c *Cache) Find(ctx context.Context, titles []string) ([]string, error) { + matches := make(map[string]match, len(titles)) + + var err error + + for _, title := range titles { + found, ok := c.kept(title, matches) + if !ok && err == nil { + found, err = c.ask(ctx, title) + ok = err == nil + } + + if ok { + matches[title] = found + } + } + + c.matches = matches + + covers := make([]string, len(titles)) + for i, title := range titles { + covers[i] = matches[title].cover + } + + return covers, err +} + +// kept returns title's match from matches, those of the Find that runs, or else from the last +// Find's when the Finder was asked for it less than a week ago. +func (c *Cache) kept(title string, matches map[string]match) (match, bool) { + if found, ok := matches[title]; ok { + return found, true + } + + found, ok := c.matches[title] + + return found, ok && time.Since(found.asked) < keepFor +} + +// ask asks the Finder for title's cover. +func (c *Cache) ask(ctx context.Context, title string) (match, error) { + asked := time.Now() + cover, err := c.finder.Cover(ctx, title) + + return match{cover: cover, asked: asked}, err +} diff --git a/services/games/internal/covers/covers_test.go b/services/games/internal/covers/covers_test.go new file mode 100644 index 0000000..30645ad --- /dev/null +++ b/services/games/internal/covers/covers_test.go @@ -0,0 +1,141 @@ +package covers_test + +import ( + "context" + "errors" + "slices" + "testing" + "testing/synctest" + "time" + + "github.com/codedogapp/bakhyard/services/games/internal/covers" +) + +func TestFindAsksForEachTitleOnceInOrder(t *testing.T) { + finder := finderOf(t, map[string]string{"B": "b.jpg", "A": "a.jpg"}) + + find(t, covers.New(finder), []string{"B", "A", "B", "No cover", "A", "No cover"}, + "b.jpg", "a.jpg", "b.jpg", "", "a.jpg", "") + finder.wantAsked("B", "A", "No cover") +} + +func TestFindOfNoTitlesAsksNothing(t *testing.T) { + finder := finderOf(t, nil) + + find(t, covers.New(finder), nil) + finder.wantAsked() +} + +func TestFindKeepsACoverForAWeek(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + finder := finderOf(t, map[string]string{"A": "a.jpg", "C": "c.jpg"}) + cache := covers.New(finder) + + find(t, cache, []string{"A", "No cover"}, "a.jpg", "") + time.Sleep(7*24*time.Hour - time.Nanosecond) + find(t, cache, []string{"A", "No cover", "C"}, "a.jpg", "", "c.jpg") + time.Sleep(time.Nanosecond) + find(t, cache, []string{"C", "A", "No cover"}, "c.jpg", "a.jpg", "") + + finder.wantAsked("A", "No cover", "C", "A", "No cover") + }) +} + +func TestFindCountsTheWeekFromTheAsking(t *testing.T) { + synctest.Test(t, func(t *testing.T) { + finder := finderOf(t, map[string]string{"A": "a.jpg"}) + finder.delay = time.Minute + cache := covers.New(finder) + + find(t, cache, []string{"A"}, "a.jpg") + time.Sleep(7*24*time.Hour - time.Minute) + find(t, cache, []string{"A"}, "a.jpg") + + finder.wantAsked("A", "A") + }) +} + +func TestFindKeepsTheCoversOfTheLatestTitlesOnly(t *testing.T) { + finder := finderOf(t, map[string]string{"A": "a.jpg", "B": "b.jpg"}) + cache := covers.New(finder) + + find(t, cache, []string{"A", "B"}, "a.jpg", "b.jpg") + find(t, cache, []string{"A"}, "a.jpg") + find(t, cache, []string{"B", "A"}, "b.jpg", "a.jpg") + + finder.wantAsked("A", "B", "B") +} + +func TestFindAsksNoMoreAfterAFailure(t *testing.T) { + finder := finderOf(t, map[string]string{ + "Kept": "kept.jpg", "New": "new.jpg", "Down": "down.jpg", "Later": "later.jpg", + }) + cache := covers.New(finder) + + find(t, cache, []string{"Kept"}, "kept.jpg") + + down := errors.New("IGDB is down") + finder.fails = map[string]error{"Down": down} + + got, err := cache.Find(t.Context(), []string{"New", "Down", "Kept", "Later", "Down", "New"}) + + want := []string{"new.jpg", "", "kept.jpg", "", "", "new.jpg"} + if !errors.Is(err, down) || !slices.Equal(got, want) { + t.Errorf("got %q, %v, want %q, %v", got, err, want, down) + } + + finder.fails = nil + + find(t, cache, []string{"New", "Down", "Kept", "Later"}, + "new.jpg", "down.jpg", "kept.jpg", "later.jpg") + + finder.wantAsked("Kept", "New", "Down", "Down", "Later") +} + +// finder is a stand-in Finder, which gives a title's cover from found, with the title's error from +// fails, after delay, and records the titles that it was asked for. +type finder struct { + t *testing.T + found map[string]string + fails map[string]error + delay time.Duration + asked []string +} + +// finderOf returns a finder of the covers in found, whose titles may be asked for only with t's +// context. +func finderOf(t *testing.T, found map[string]string) *finder { + t.Helper() + + return &finder{t: t, found: found} +} + +func (f *finder) Cover(ctx context.Context, title string) (string, error) { + if ctx != f.t.Context() { + f.t.Errorf("%q: asked with another context", title) + } + + f.asked = append(f.asked, title) + time.Sleep(f.delay) + + return f.found[title], f.fails[title] +} + +// wantAsked fails the test unless the finder was asked for want, in order. +func (f *finder) wantAsked(want ...string) { + f.t.Helper() + + if !slices.Equal(f.asked, want) { + f.t.Errorf("asked for %q, want %q", f.asked, want) + } +} + +// find has cache find the covers of titles, and fails the test unless they're want. +func find(t *testing.T, cache *covers.Cache, titles []string, want ...string) { + t.Helper() + + got, err := cache.Find(t.Context(), titles) + if err != nil || !slices.Equal(got, want) { + t.Errorf("%q: got %q, %v, want %q", titles, got, err, want) + } +} diff --git a/services/games/main.go b/services/games/main.go index 2908f21..8f8648e 100644 --- a/services/games/main.go +++ b/services/games/main.go @@ -11,16 +11,22 @@ import ( "github.com/go-chi/chi/v5/middleware" "github.com/codedogapp/bakhyard/internal/health" + "github.com/codedogapp/bakhyard/internal/keychain" + "github.com/codedogapp/bakhyard/internal/secrets" "github.com/codedogapp/bakhyard/internal/service" "github.com/codedogapp/bakhyard/services/games/internal/api" ) func main() { - // Exophase's API, where the games list comes from: the same for everyone, so not a setting - // (docs/plan.md decision 300). - exophaseAPI := &url.URL{Scheme: "https", Host: "api.exophase.com"} + // The APIs that games asks: the same for everyone, so not settings (docs/plan.md decision 300). + addresses := apis{ + exophase: &url.URL{Scheme: "https", Host: "api.exophase.com"}, + twitch: &url.URL{Scheme: "https", Host: "id.twitch.tv"}, + igdb: &url.URL{Scheme: "https", Host: "api.igdb.com"}, + } + readSecret := secrets.New(os.Getenv, keychain.NewReader(keychain.Security).Password) served := func(ctx context.Context, getenv func(string) string) (http.Handler, error) { - return routes(ctx, getenv, exophaseAPI) + return routes(ctx, getenv, readSecret, addresses) } err := service.Run(os.Getenv, served) @@ -30,14 +36,31 @@ func main() { } } +// apis are the addresses of the APIs that games asks. +type apis struct { + // exophase is Exophase's, where the games list comes from. + exophase *url.URL + // twitch is Twitch's OAuth server, which gives the tokens that IGDB takes. + twitch *url.URL + // igdb is IGDB's, where the covers come from. + igdb *url.URL +} + // routes returns the handler of games, which serves its API. With Exophase set up, it lists the -// games I played lately from exophaseAPI when a request finds them stale, until ctx ends. +// games I played lately when a request finds them stale, until ctx ends, and with a Twitch app set +// up, whose secret readSecret reads, it finds their covers on IGDB. func routes( ctx context.Context, getenv func(string) string, - exophaseAPI *url.URL, + readSecret secrets.Reader, + addresses apis, ) (http.Handler, error) { - client, err := exophaseClient(getenv, exophaseAPI) + client, err := exophaseClient(getenv, addresses.exophase) + if err != nil { + return nil, err + } + + cache, err := coverCache(ctx, getenv, readSecret, addresses) if err != nil { return nil, err } @@ -46,7 +69,7 @@ func routes( // Answer HEAD wherever GET is served. router.Use(middleware.GetHead) health.Register(router) - api.Register(router, poll(ctx, client, slog.Default())) + api.Register(router, poll(ctx, client, cache, slog.Default())) return router, nil } diff --git a/services/games/main_test.go b/services/games/main_test.go index 955ef97..30c53ed 100644 --- a/services/games/main_test.go +++ b/services/games/main_test.go @@ -1,14 +1,18 @@ package main import ( + "context" + "fmt" "net/http" "net/http/httptest" "net/url" "testing" + + "github.com/codedogapp/bakhyard/internal/secrets" ) func TestGamesServesWithoutExophase(t *testing.T) { - router, err := routes(t.Context(), envOf(nil), unasked()) + router, err := routes(t.Context(), envOf(nil), secretsWith(nil), exophaseAt(unasked())) if err != nil { t.Fatal(err) } @@ -37,7 +41,7 @@ func TestGamesServesWithoutExophase(t *testing.T) { func TestGamesDoesNotStartWithABadPlayer(t *testing.T) { router, err := routes(t.Context(), envOf(map[string]string{exophasePlayerVar: "me"}), - unasked()) + secretsWith(nil), exophaseAt(unasked())) want := `BAKHYARD_EXOPHASE_PLAYER: "me" is not an Exophase profile id, which is digits only` if router != nil || err == nil || err.Error() != want { @@ -45,12 +49,30 @@ func TestGamesDoesNotStartWithABadPlayer(t *testing.T) { } } -// unasked returns an address of Exophase's API that the tests never ask. +// unasked returns an address of an API that the tests never ask. func unasked() *url.URL { - return &url.URL{Scheme: "https", Host: "api.exophase.example"} + return &url.URL{Scheme: "https", Host: "api.unasked.example"} +} + +// exophaseAt returns the addresses of the APIs, with Exophase's at address and the others +// unasked. +func exophaseAt(address *url.URL) apis { + return apis{exophase: address, twitch: unasked(), igdb: unasked()} } // envOf returns an environment that holds only settings. func envOf(settings map[string]string) func(string) string { return func(name string) string { return settings[name] } } + +// secretsWith returns a fake secrets reader that has only items, by name. +func secretsWith(items map[string]string) secrets.Reader { + return func(_ context.Context, name string) (string, error) { + secret, ok := items[name] + if !ok { + return "", fmt.Errorf("no secret %s", name) + } + + return secret, nil + } +}