Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion docs/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ The v1.2 design in §4 to §6 stays as reference. A piece of it is built only wh
| `games-cache` | The `games` service polls Exophase only on request: the first request waits for the first poll, later ones get the latest list at once and start a poll in the background once the list is an hour old; after a failed poll the last good list stays, and the next poll waits 5 minutes, twice as long after each failure in a row, up to 6 hours. The third of `games-exophase`'s eleven steps (decision 302) | Done (#175) |
| `games-service` | A new `games` service answers the games I played in the last 14 days at `GET /games/recent`, from the Exophase profile that `BAKHYARD_EXOPHASE_PLAYER` names, digits only; it runs on 8090 under `mise run up`, and its image builds with the others'. The fourth of `games-exophase`'s eleven steps (decision 303) | Done (#178) |
| `twitch-token` | The `games` service gets a Twitch app token for IGDB with the client credentials grant, keeps it in memory until 9 tenths of its life, asks Twitch one request at a time, and forgets a token that IGDB refuses. The fifth of `games-exophase`'s eleven steps (decision 304) | Done (#181) |
| `igdb-search` | The `games` service's IGDB client finds a title's cover: the top result of a search filtered to main games, expansions, remakes, remasters, expanded games and ports, or a game named exactly the title when the top result has another name; it asks one request at a time, at least 250 ms apart, and with a new token once when IGDB refuses one. The sixth of `games-exophase`'s eleven steps (decision 305) | Done (#183) |
| `vault-edit-sync` | Editing the vault from Bakhyard (§8.2 Phase 6), by the vault session in `../bakhyard-vault/`, which adds its steps' rows under this one and then does `copilot-planner` (decision 195) | In progress |
| `vault-plan` | The vault session's first steps: where a save goes waits on the user's answer to §12's question on how edits get back to the Mac, so the version that a save checks and the editor come first; a note's version is the SHA-256 of its bytes, sent as its `ETag`, and a save is a `PUT` that sends it back in `If-Match` (decision 400) | Done (#167) |
| `note-etag` | notes answers each note with its version as a strong `ETag`, the SHA-256 of its bytes, and a matching `If-None-Match` with a 304 (decision 400) | Done (#169) |
Expand Down Expand Up @@ -1275,7 +1276,7 @@ Every item is pending unless it says done. The service an item lands in is in br
- `games-cache`: the service fetches on request: the first request waits for the first fetch, later ones get the latest list at once and start a fetch in the background once it's an hour old; after a failure, the last good list stays and the next fetch backs off (decision 302). Done (#175).
- `games-service`: a new `games` service answers the games I played in the last 14 days at `GET /games/recent` (decision 303). Done (#178).
- `twitch-token`: the service gets a Twitch app token for IGDB, keeps it and renews it before it expires (decision 304). Done (#181).
- `igdb-search`: the service's IGDB client finds a title's cover with a filtered search and an exact-name fallback (decision 68), within IGDB's 4 requests a second.
- `igdb-search`: the service's IGDB client finds a title's cover with a filtered search and an exact-name fallback (decisions 68 and 305), within IGDB's 4 requests a second. Done (#183).
- `games-covers`: each game in the answer carries its IGDB cover; each title is matched once and the match kept.
- `homelab-games`: in the homelab repo, `games` runs on the laptop with its settings and secret, and the gateway's peers name it.
- `games-gateway`: the gateway forwards `/api/games` to `games`.
Expand Down Expand Up @@ -1793,6 +1794,17 @@ Decisions 300 to 399 are the `games-exophase` session's.
- **Failures**: a failed request changes nothing, so the next call asks again. A status other than 200 gives `get a Twitch token:` and the status, with Twitch's `message` when it gives one, such as `400 Bad Request, invalid client`; an answer with no token, or with a lifetime of 0 seconds or less, gives `read Twitch's answer:` and why. No error shows the secret or a token.
- **Rejected**: checking the token every hour at `/oauth2/validate`, which Twitch requires of apps that keep a user's session, not of an app token; renewing only when IGDB answers 401, which costs a refused search per token; asking for a token as the service starts, since only a title not matched yet needs one; a `sync.Mutex`, which a waiting call's context can't end; a user agent, which only Exophase's Cloudflare needed (decision 301).

305. **The IGDB search** (`igdb-search`): the lead's choices (2026-10-30), for the user to review.
- **The request**: `services/games/internal/igdb` posts its query to IGDB's games endpoint, `https://api.igdb.com/v4/games`, with the Twitch app's `Client-ID`, the app token (decision 304) as `Authorization: Bearer`, and `Accept: application/json`, as IGDB's docs show, and the query as `text/plain; charset=utf-8`. Each request gives up after 10 seconds, and an answer over 64 KiB is refused, as Twitch's are.
- **The match**: the first query searches for the title among main games, expansions, standalone expansions, remakes, remasters, expanded games and ports that aren't versions of another game, `game_type = (0,2,4,8,9,10,11) & version_parent = null` as in the spike (§2), and takes the top result. When its name is the title, ignoring case, it's the match. Otherwise a second query looks for a game of any type named exactly the title, ignoring case, with `name ~`. A game it finds wins, since the top result may be another game, as it was for the spike's 15th title. When it finds none, the top result stays, since Exophase's titles often differ from IGDB's names in subtitles or numerals. A title that IGDB knows by the same name costs one request, and any other two.
- **The cover**: the match's 264×374 cover, `https://images.igdb.com/igdb/image/upload/t_cover_big/<image_id>.jpg` (decision 68), or none when the match has no cover or there's no match. The page loads it from IGDB, which the gateway's content security policy allows. IGDB keeps an image that it removes or replaces for 30 days, so `games-covers` keeps a match for a week at most. An empty title, which Exophase can give, gets no cover and asks nothing.
- **The title in a query**: a backslash before each backslash and quote, as the query language's strings take them, and no control characters, which no title needs.
- **IGDB's limits**: requests go one at a time, through a one-slot channel, and each starts at least 250 ms after the previous one began, so IGDB gets at most 4 a second and one open request, within its 4 a second and 8 open. A call whose context ends while it waits returns the context's error.
- **A refused token**: when IGDB answers 401, the client drops the token and asks once more with a new one. A second 401 is an error, so a client ID that doesn't fit its token can't make the client ask Twitch without end.
- **Failures**: errors start with `find the cover of "<title>":`, then `ask IGDB:` and the status or the transport's error, `read IGDB's answer:`, or the token's error. The client retries no 429 and no 5xx: `games-covers` keeps no error, so a later fetch asks again. No error shows the token, which travels only in a header.
- **Not yet checked live**: the queries and headers follow the spike, IGDB's docs, and another IGDB client's working use of `name ~` without wildcards. IGDB itself waits for the user's Twitch app, which `games-covers` needs.
- **Rejected**: the exact-name search first, which would take any record with that name, such as an edition or a bundle, before the filtered search's better match; the fallback only when the filtered search finds nothing, which would keep the spike's 15th title's wrong match; requests in parallel up to IGDB's 8 open, which would need a rate limiter to save a few seconds on a dozen titles, each matched once; `golang.org/x/time/rate`, a new module for one gap; stripping the quotes from a title rather than escaping them, which would change the exact name; covers that Bakhyard stores and serves, which IGDB already serves.

Decisions 400 to 499 are the vault session's: `vault-edit-sync`, then `copilot-planner`.

400. **Editing notes, the first steps** (`vault-plan`, `note-etag`, `note-editor`): the lead's choices (2026-10-30), for the user to review.
Expand Down
127 changes: 127 additions & 0 deletions services/games/internal/igdb/ask.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
package igdb

import (
"context"
"encoding/json/v2"
"errors"
"fmt"
"io"
"net/http"
"strings"
"time"
)

const (
// gap is the least time between the starts of two requests, so IGDB gets 4 a second at most.
gap = 250 * time.Millisecond
// maxAnswer bounds the size of each answer from IGDB: many times what a game's needs.
maxAnswer = 64 << 10
// requestTimeout bounds each request to IGDB.
requestTimeout = 10 * time.Second
// asking starts the errors of a request that IGDB didn't answer with games.
asking = "ask IGDB"
)

// errRefused is the error of a request whose token IGDB refused.
var errRefused = errors.New(asking + ": 401 Unauthorized")

// game is a game in IGDB's answer, with the fields that the queries ask for.
type game struct {
Name string `json:"name"`
Cover struct {
ImageID string `json:"image_id"`
} `json:"cover"`
}

// ask returns the games that IGDB answers to query. When IGDB refuses the token, which it may
// have revoked, ask asks once more with a new one.
func (c *Client) ask(ctx context.Context, query string) ([]game, error) {
games, err := c.askWithToken(ctx, query)
if errors.Is(err, errRefused) {
return c.askWithToken(ctx, query)
}

return games, err
}

// askWithToken asks query with the token that the Tokens give, and drops it when IGDB refuses it.
func (c *Client) askWithToken(ctx context.Context, query string) ([]game, error) {
token, err := c.tokens.Token(ctx)
if err != nil {
return nil, err
}

games, err := c.send(ctx, token, query)
if errors.Is(err, errRefused) {
c.tokens.Drop(token)
}

return games, err
}

// send sends query with token in its turn, once the previous request started gap ago. If ctx
// ends while it waits, it returns ctx's error.
func (c *Client) send(ctx context.Context, token, query string) ([]game, error) {
select {
case c.turn <- struct{}{}:
case <-ctx.Done():
return nil, ctx.Err()
}

defer func() { <-c.turn }()

pause := time.NewTimer(time.Until(c.next))
defer pause.Stop()

select {
case <-pause.C:
case <-ctx.Done():
return nil, ctx.Err()
}

c.next = time.Now().Add(gap)

return c.post(ctx, token, query)
}

// post posts query to IGDB's games endpoint with token, and returns the games it answers.
func (c *Client) post(ctx context.Context, token, query string) ([]game, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.endpoint,
strings.NewReader(query))
if err != nil {
return nil, fmt.Errorf("%s: %w", asking, err)
}

req.Header.Set("Client-ID", c.clientID)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Accept", "application/json")
req.Header.Set("Content-Type", "text/plain; charset=utf-8")

res, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("%s: %w", asking, err)
}

defer func() { _ = res.Body.Close() }()

switch res.StatusCode {
case http.StatusOK:
return answered(res.Body)
case http.StatusUnauthorized:
return nil, errRefused
default:
return nil, fmt.Errorf("%s: %s", asking, res.Status)
}
}

// answered returns the games in body, IGDB's answer to a query.
func answered(body io.Reader) ([]game, error) {
var games []game

err := json.UnmarshalRead(io.LimitReader(body, maxAnswer), &games)
if err != nil {
return nil, fmt.Errorf("read IGDB's answer: %w", err)
}

return games, nil
}
180 changes: 180 additions & 0 deletions services/games/internal/igdb/ask_internal_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
package igdb

import (
"context"
"errors"
"io"
"net/http"
"net/url"
"slices"
"strings"
"sync"
"testing"
"testing/synctest"
"time"
)

// memory stands in for IGDB in memory, so that it answers in a bubble: it answers each request
// with no games once delay has passed, and records when each request began.
type memory struct {
delay time.Duration
began []time.Time
}

func (m *memory) RoundTrip(*http.Request) (*http.Response, error) {
m.began = append(m.began, time.Now())
time.Sleep(m.delay)

return &http.Response{
StatusCode: http.StatusOK,
Body: io.NopCloser(strings.NewReader("[]")),
}, nil
}

// hanging stands in for an IGDB that never answers.
type hanging struct{}

func (hanging) RoundTrip(r *http.Request) (*http.Response, error) {
<-r.Context().Done()

return nil, r.Context().Err()
}

// aToken is a token that IGDB never refuses.
type aToken struct{}

func (aToken) Token(context.Context) (string, error) { return "a-token", nil }

func (aToken) Drop(string) {}

func TestRequestsStartAQuarterOfASecondApart(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
igdb := &memory{}
client := standIn(igdb)
start := time.Now()

wantNoCover(t, client)
wantNoCover(t, client)
time.Sleep(time.Second)
wantNoCover(t, client)

want := []time.Duration{0, 250, 500, 750, 1750, 2000}
if got := since(start, igdb.began); !slices.Equal(got, want) {
t.Errorf("the requests began after %v ms, want %v", got, want)
}
})
}

func TestRequestsGoOneAtATime(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
igdb := &memory{delay: time.Second}
client := standIn(igdb)
start := time.Now()

var calls sync.WaitGroup
for range 2 {
calls.Go(func() { wantNoCover(t, client) })
}

calls.Wait()

want := []time.Duration{0, 1000, 2000, 3000}
if got := since(start, igdb.began); !slices.Equal(got, want) {
t.Errorf("the requests began after %v ms, want %v", got, want)
}
})
}

func TestACallThatWaitsForTheRequestUnderWayEndsWithItsContext(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
igdb := &memory{delay: time.Second}
client := standIn(igdb)

var first sync.WaitGroup
first.Go(func() { wantNoCover(t, client) })
synctest.Wait()

wantGivenUp(t, client)
first.Wait()

if len(igdb.began) != 2 {
t.Errorf("IGDB got %d requests, want the first call's 2", len(igdb.began))
}
})
}

func TestACallThatWaitsForItsQuarterOfASecondEndsWithItsContext(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
igdb := &memory{}
client := standIn(igdb)

wantNoCover(t, client)
wantGivenUp(t, client)

if len(igdb.began) != 2 {
t.Errorf("IGDB got %d requests, want the first call's 2", len(igdb.began))
}
})
}

func TestARequestGivesUpAfter10Seconds(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
began := time.Now()

got, err := standIn(hanging{}).Cover(t.Context(), "Moon Garden")
if got != "" || !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("got %q, %v, want the timeout's error", got, err)
}

if waited := time.Since(began); waited != 10*time.Second {
t.Errorf("gave up after %s, want 10s", waited)
}
})
}

// standIn returns the Client whose requests go to igdb instead of IGDB.
func standIn(igdb http.RoundTripper) *Client {
client := New(&url.URL{Scheme: "https", Host: "api.igdb.com"}, "an-id", aToken{})
client.client.Transport = igdb

return client
}

// wantNoCover checks that client finds no cover for Moon Garden, and no error.
func wantNoCover(t *testing.T, client *Client) {
t.Helper()

if got, err := client.Cover(t.Context(), "Moon Garden"); got != "" || err != nil {
t.Errorf("got %q, %v, want no cover", got, err)
}
}

// wantGivenUp checks that client gives up on Moon Garden's cover with its context's error, when
// the context ends 100 ms after the call.
func wantGivenUp(t *testing.T, client *Client) {
t.Helper()

ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
defer cancel()

began := time.Now()

got, err := client.Cover(ctx, "Moon Garden")
if got != "" || !errors.Is(err, context.DeadlineExceeded) {
t.Errorf("got %q, %v, want the context's error", got, err)
}

if waited := time.Since(began); waited != 100*time.Millisecond {
t.Errorf("gave up after %s, want 100ms", waited)
}
}

// since returns how many milliseconds after start each of times is.
func since(start time.Time, times []time.Time) []time.Duration {
waits := make([]time.Duration, 0, len(times))
for _, at := range times {
waits = append(waits, at.Sub(start)/time.Millisecond)
}

return waits
}
Loading
Loading