diff --git a/docs/Navigation.md b/docs/Navigation.md index 620a8710..751e105d 100644 --- a/docs/Navigation.md +++ b/docs/Navigation.md @@ -49,17 +49,22 @@ search: - [Collections](using/collections.md) - [Smart Collections](using/smart-collections.md) - [Virtual Collections](using/virtual-collections.md) + - [Physical Games](using/physical-games.md) + - [Recommendations](using/recommendations.md) - [Downloads](using/downloads.md) - [Uploads](using/uploads.md) - In-Browser Play - [EmulatorJS](using/in-browser-play/emulatorjs.md) - [`js-dos`](using/in-browser-play/js-dos.md) - [MS-DOS](using/in-browser-play/ms-dos.md) + - [PICO-8](using/in-browser-play/pico-8.md) - [Ruffle](using/in-browser-play/ruffle.md) - [Emulator Streaming](using/emulator-streaming.md) - [Migrating to webstation](using/emulator-streaming-migration.md) + - [Jukebox](using/jukebox.md) - [Saves & States](using/saves-and-states.md) - [RetroAchievements](using/retroachievements.md) + - [Walkthroughs](using/walkthroughs.md) - [ROM Patcher](using/rom-patcher.md) - [Netplay](using/netplay.md) - [Account & Profile](using/account-and-profile.md) diff --git a/docs/administration/firmware-management.md b/docs/administration/firmware-management.md index f429545f..dd8c55c2 100644 --- a/docs/administration/firmware-management.md +++ b/docs/administration/firmware-management.md @@ -19,9 +19,14 @@ RomM does not ship games or firmware, and the team cannot help you obtain BIOS f ## Ingesting firmware 1. Put the file in the right `bios/` folder (see [Folder Structure](../getting-started/folder-structure.md)) -2. Run a scan → Firmware is picked up alongside ROMs -3. Navigate to the platform's gallery and click the `CPU` icon in the top left -4. Firmware files will display at the bottom of the page +2. Run a scan, and firmware is picked up alongside ROMs +3. The platform's firmware is then listed on that platform's page, and offered to the in-browser player + +Where `bios/` lives is up to you. The default is `bios/{platform}` at the library root, and you can move it with the [`filesystem.structure.firmware`](../reference/configuration-file.md#filesystemstructure) template. + +## Missing firmware + +Delete a file from `bios/` and the next scan **flags it missing** instead of dropping it from the database. Put the file back and the next scan clears the flag. If you're never replacing it, the **Cleanup missing firmware** task deletes every flagged row in one go (see [Scheduled Tasks](scheduled-tasks.md#triggering-a-task-manually)). ## Platform-specific firmware diff --git a/docs/administration/index.md b/docs/administration/index.md index 80f82b39..0555f83d 100644 --- a/docs/administration/index.md +++ b/docs/administration/index.md @@ -5,7 +5,7 @@ description: Running RomM for yourself and others. # Administration -Administration is everything you do **as the operator** of a RomM instance: managing accounts, controlling access, configuring metadata sources, scheduling scans, watching the library for changes, monitoring the server, and keeping data safe. +Administration is everything you do **as the server owner** of a RomM instance: managing accounts, controlling access, configuring metadata sources, scheduling scans, watching the library for changes, monitoring the server, and keeping data safe. The end-user equivalent (how to actually play the games, build collections, upload saves) lives in [Using RomM](../using/index.md). diff --git a/docs/administration/oidc/index.md b/docs/administration/oidc/index.md index 490728be..547bea1d 100644 --- a/docs/administration/oidc/index.md +++ b/docs/administration/oidc/index.md @@ -134,6 +134,8 @@ environment: - OIDC_USERNAME_ATTRIBUTE=preferred_username ``` +Whatever that attribute holds gets sanitised before it becomes a username to prevent invalid characters from being used. RomM also sends a PKCE challenge on every authorization request, so it's safe to mark PKCE required for the client if your IdP offers that. + ## Important notes - **Email must match** between OIDC and any existing local account, otherwise OIDC creates a new account alongside the old one. diff --git a/docs/administration/scanning-and-watcher.md b/docs/administration/scanning-and-watcher.md index c233aa5c..24886a62 100644 --- a/docs/administration/scanning-and-watcher.md +++ b/docs/administration/scanning-and-watcher.md @@ -20,7 +20,7 @@ Every scan picks one mode. Modes differ in what they touch, so use the most-targ | Mode | What it does | When to use | | ----------------- | -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | **New Platforms** | Only scans platform folders not already in the DB. | After mounting a new ROM set (very fast). | -| **Quick** | Skips files that already exist in the DB, with no metadata refresh. | Default for scheduled runs and the watcher. | +| **Quick** | Adds new games and catches file updates, with no metadata refresh. | Default for scheduled runs and the watcher. | | **Unmatched** | Re-runs metadata matching against ROMs currently missing external IDs. | After adding a new metadata provider, or when some titles didn't match on the first scan. | | **Update** | Re-fetches metadata for all already-matched ROMs. | When metadata providers have meaningfully changed (e.g. IGDB restructured). | | **Hashes** | Recalculates CRC/MD5/SHA1 hashes. | After upgrading from a version that didn't hash or when you suspect file corruption. | @@ -36,12 +36,14 @@ A manual scan can be scoped to specific platforms and a chosen subset of metadat Configured via env vars (full table in [Scheduled Tasks](scheduled-tasks.md)): -| Variable | Default | Purpose | -| ----------------------- | ----------- | ------------------------------------------------------------------------------------------------------------ | -| `SCHEDULED_RESCAN_CRON` | `0 0 * * *` | Cron expression for the scheduled library scan. Runs a **Quick** scan by default. | -| `SCAN_TIMEOUT` | `14400` | Hard cap in seconds: scans that exceed this are killed and logged. | -| `SCAN_WORKERS` | `1` | Concurrent worker processes for scanning; leave as auto unless you're tuning. | -| `SEVEN_ZIP_TIMEOUT` | `60` | Per-archive timeout in seconds for `.7z` extraction during scan; raise if scanning huge compressed ROM sets. | +| Variable | Default | Purpose | +| ----------------------- | ----------- | ---------------------------------------------------------------------------------------------------------- | +| `SCHEDULED_RESCAN_CRON` | `0 3 * * *` | Cron expression for the scheduled library scan, which runs a **Quick** scan | +| `SCAN_TIMEOUT` | `14400` | Hard cap in seconds, after which the scan is killed and the clients watching it are told why | +| `SCAN_WORKERS` | `4` | How many ROMs a scan processes at once | +| `SEVEN_ZIP_TIMEOUT` | `60` | Per-archive timeout in seconds for `.7z` extraction during scan, raise it if scanning huge compressed sets | + +Scans get their own queue and worker now, so a long library scan won't hold up the shorter background tasks behind it. To disable scheduled scans entirely, either unset the cron or set it to something unreachable (`SCHEDULED_RESCAN_CRON=0 0 31 2 *`). @@ -100,7 +102,19 @@ exclude: extensions: [nfo] ``` -Full schema in [Configuration File](../reference/configuration-file.md). +Whatever you list here is **added** to the defaults, not swapped in for them. The system folders and the frontend media folders stay excluded either way. Full schema in [Configuration File](../reference/configuration-file.md). + +## Platform folder names + +Every platform folder has to resolve to a [known slug](../platforms/supported-platforms.md). The [folder name aliases](../platforms/supported-platforms.md#folder-name-aliases) already cover what Batocera, RetroBat and ES-DE call things, so a library from one of those works mostly untouched. Anything else needs a [`system.platforms`](../reference/configuration-file.md#systemplatforms) mapping. + +## Title IDs read from the binary + +The following platforms stamp a **native Title ID** into the game binary, which scans can read: PSX, PS2, PS3, PSP, PS Vita, Switch, 3DS, Wii, Wii U, GameCube, Dreamcast, Xbox and Xbox 360. + +That ID identifies games on the platforms RomM doesn't hash, which is how a renamed or moved file holds on to its saves and collections, and marks the location where the game writes its saves, which is needed for device sync. + +To skip the whole thing, use [`filesystem.skip_title_id_extraction`](../reference/configuration-file.md#filesystemskip_title_id_extraction). ## Region and language preference diff --git a/docs/administration/scheduled-tasks.md b/docs/administration/scheduled-tasks.md index f3bfaf04..734bd084 100644 --- a/docs/administration/scheduled-tasks.md +++ b/docs/administration/scheduled-tasks.md @@ -25,11 +25,13 @@ Every scheduled task takes a standard 5-field cron expression: - `*/30 * * * *`: every 30 minutes - `0 2 * * 0`: 2 AM every Sunday -Set the env var and restart the container; the scheduler picks up the new schedule the moment RomM comes back up. +Set the env var and restart the container. The scheduler picks up the new schedule as soon as RomM is back. ## Enabling a scheduled task -Most tasks have an `ENABLE_*` environment variable, like `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA=true` which enables the LaunchBox sync, and every one of them is off by default. Set both the enable var and its cron var, since a task with an empty cron string has nothing to schedule and stays unscheduled even when enabled. +Most tasks have an `ENABLE_*` environment variable, like `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA=true` which enables the LaunchBox sync. Set both the enable var and its cron var, since a task with an empty cron string has nothing to schedule and stays unscheduled even when enabled. + +Unlike other tasks, **build recommendations index** ships enabled, because the [recommendation](../using/recommendations.md) sections read that index and similar games sits empty without it. Setting `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS=false` stops the nightly rebuild, but doesn't hide either section; users turn those off in their own settings. The housekeeping tasks (netplay cleanup, upload tmp cleanup, ZIP cache cleanup) are always on and have no env vars. Check the [env var reference](../reference/environment-variables.md) for the full list. @@ -37,7 +39,7 @@ The housekeeping tasks (netplay cleanup, upload tmp cleanup, ZIP cache cleanup) ### From the Administration page -**Administration → Tasks** shows every task with a "Run" button. Admins (anyone with `tasks.run` scope) can trigger: +**Administration → Tasks** lists every task with its status and a way to run it. Anyone with the `tasks.run` scope can fire one off, scheduled tasks included, which saves waiting for the next cron tick after a config change. ### From the API @@ -59,6 +61,7 @@ A task that's been "running" for hours is usually a scan that hit `SCAN_TIMEOUT` On a Raspberry Pi or NAS with 2 GB of RAM and/or a single CPU core: - Raise the cron intervals (daily → weekly) for the nightlies -- Set `SCAN_WORKERS=1` to avoid concurrent scan processes +- Set `SCAN_WORKERS=1` and `WEB_SERVER_CONCURRENCY=1`, both of which default to `4` - Enable the watcher but raise `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` to 30+ minutes -- Disable image conversion if you don't care about WebP (`ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP=false`). +- Disable image conversion if you don't care about WebP (`ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP=false`) +- On a big library, set `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS=false` to skip the nightly build diff --git a/docs/administration/server-stats.md b/docs/administration/server-stats.md index 7cfea37e..695629dd 100644 --- a/docs/administration/server-stats.md +++ b/docs/administration/server-stats.md @@ -11,14 +11,14 @@ description: "The numbers Mason! What do they mean?" ### Top-line counts -| Metric | What it counts | -| ---------------- | --------------------------------------------------------------------------------- | -| **Platforms** | Every platform RomM has seen at least one ROM for. Deleted platforms don't count. | -| **Games** | Total ROM entries. A multi-file game (folder with multiple files) counts as 1. | -| **Saves** | User save files across all users. | -| **States** | Emulator save states across all users. | -| **Screenshots** | User-uploaded screenshots. Provider-fetched screenshots aren't counted here. | -| **Size on disk** | Total disk usage of all ROMs, saves, states, and screenshots. | +| Metric | What it counts | +| ---------------- | ----------------------------------------------------------------------------------------- | +| **Platforms** | Every platform RomM has seen at least one ROM for. Deleted platforms don't count. | +| **Games** | Total ROM entries. Multi-file (folder with multiple files) and physical games count as 1. | +| **Saves** | User save files across all users. | +| **States** | Emulator save states across all users. | +| **Screenshots** | User-uploaded screenshots. Provider-fetched screenshots aren't counted here. | +| **Size on disk** | Total disk usage of all ROMs, saves, states, and screenshots. | ### Per-platform breakdown @@ -29,8 +29,6 @@ Under the summary, it's a table sorted by name, size or game count. For each pla - Region distribution (how many games tagged USA, Japan, Europe, World, etc.) - Metadata coverage (how many games have metadata from each provider) -When you want to know "which platform is eating my disk?" or "which platform has the worst match rate?" - ## API The same data is available programmatically: diff --git a/docs/developers/architecture.md b/docs/developers/architecture.md index 1e42d955..a8deff10 100644 --- a/docs/developers/architecture.md +++ b/docs/developers/architecture.md @@ -83,7 +83,7 @@ Environment variables (100+ of them, all listed in `env.template`) cover infrast ### Background jobs -RQ workers run scheduled jobs (rescans, Switch TitleDB refresh, LaunchBox refresh, image-to-WebP conversion, RA progress sync, netplay cleanup) and manual tasks (`cleanup_missing_roms`, `cleanup_orphaned_resources`, `sync_folder_scan`). Each scheduled task is gated by an `ENABLE_SCHEDULED_*` env var and tunable via the matching `*_CRON`. Operator-side detail in [Scheduled Tasks](../administration/scheduled-tasks.md). +RQ workers run scheduled jobs (rescans, Switch TitleDB refresh, LaunchBox refresh, image-to-WebP conversion, RA progress sync, netplay cleanup) and manual tasks (`cleanup_missing_roms`, `cleanup_orphaned_resources`, `sync_folder_scan`). Each scheduled task is gated by an `ENABLE_SCHEDULED_*` env var and tunable via the matching `*_CRON`. Server owner detail in [Scheduled Tasks](../administration/scheduled-tasks.md). ## Frontend diff --git a/docs/developers/development-setup.md b/docs/developers/development-setup.md index bd9b9ebf..e87e61de 100644 --- a/docs/developers/development-setup.md +++ b/docs/developers/development-setup.md @@ -7,6 +7,15 @@ description: Run RomM locally for development # Setting up RomM for development +## Prerequisites + +| Tool | Needed for | +| ------- | ------------------------------------------------------------ | +| Python | 3.14 or newer, pinned in `.python-version` | +| Node.js | 24, with npm 11.10 or newer, per `frontend/package.json` | +| uv | Dependencies, and it fetches the Python in `.python-version` | +| Docker | For the database, Valkey, and the optional streaming stack | + ## Option 1: Using Docker If you prefer to use Docker for development, you can set up RomM using the provided Docker Compose configuration. This method simplifies the setup process by encapsulating all dependencies within Docker containers. @@ -111,6 +120,15 @@ uv sync --all-extras --dev docker compose up -d ``` +Two optional stacks have their own compose files: + +```sh +docker compose -f docker-compose.oidc.yml up -d # Authentik, for testing OIDC +docker compose -f docker-compose.streaming.yml up -d # webstation, for emulator streaming +``` + +The streaming image is amd64 only and runs to several GB, hence opt-in. + #### Run the backend _Migrations will be run automatically when running the backend._ @@ -126,7 +144,6 @@ uv run python3 main.py ```sh cd frontend -# npm version >= 9 needed npm install ``` diff --git a/docs/developers/index.md b/docs/developers/index.md index f20781f5..12b8a24d 100644 --- a/docs/developers/index.md +++ b/docs/developers/index.md @@ -5,7 +5,7 @@ description: Build on top of RomM or contribute to it # API & Development -Everything you need to **build on top of RomM** (third-party apps, scripts, integrations) or **contribute to it** (code, translations, docs). End-user content lives in [Using RomM](../using/index.md). Operator content lives in [Administration](../administration/index.md). +Everything you need to **build on top of RomM** (third-party apps, scripts, integrations) or **contribute to it** (code, translations, docs). End-user content lives in [Using RomM](../using/index.md). Server owner content lives in [Administration](../administration/index.md). ## Calling the API diff --git a/docs/ecosystem/first-party-apps.md b/docs/ecosystem/first-party-apps.md index b561c558..43536e14 100644 --- a/docs/ecosystem/first-party-apps.md +++ b/docs/ecosystem/first-party-apps.md @@ -62,7 +62,7 @@ See [Getting Started](https://github.com/rommapp/argosy-launcher#getting-started - **Pulls** ROMs from your instance to the handheld's SD card, organised into muOS/NextUI's expected folder layout - **Pushes** saves and states back to your instance when you finish a session - **Schedules** sync runs: on idle, on session end, or on a cron -- Works fully offline between syncs, so the handheld doesn't need your instance to play +- Works fully offline between syncs so the handheld doesn't need your instance to play ### Setup diff --git a/docs/getting-started/folder-structure.md b/docs/getting-started/folder-structure.md index 814d0ef8..8e541ed0 100644 --- a/docs/getting-started/folder-structure.md +++ b/docs/getting-started/folder-structure.md @@ -53,7 +53,24 @@ See the [reference Docker Compose](quick-start.md) for where `/romm/library` liv ## Multi-file games -Some games come as **folders** instead of single files, which could include multiple disc, DLCs, manuals, or patches. These sub-folder names are recognised and surfaced as tags in the UI: `dlc`, `hack`, `manual`, `mod`, `patch`, `update`, `demo`, `translation`, `prototype`, `screenshots`. +Some games come as **folders** instead of single files, holding multiple discs, DLC, manuals, or patches alongside the game itself. These sub-folder names are recognised and surfaced as tags in the UI, in singular or plural form: + +| Folder | Holds | +| ---------------- | --------------------------------------------------- | +| `dlc` | Downloadable content | +| `update` | Game updates | +| `patch` | [Patch files](../using/rom-patcher.md) | +| `hack`, `mod` | Community modifications | +| `translation` | Fan translations | +| `demo` | Demo builds | +| `prototype` | Prototype builds | +| `manual` \* | Manuals | +| `walkthrough` \* | [Walkthroughs](../using/walkthroughs.md) | +| `soundtrack` \* | Audio tracks for the [Jukebox](../using/jukebox.md) | +| `cheat` \* | Cheat files | +| `screenshot` \* | Screenshots | + +\* These never contain a ROM binary, so title-id extraction skips them. They are still hashed. ## Visual reference @@ -98,7 +115,10 @@ Some games come as **folders** instead of single files, which could include mult │ │ ├─ demo │ │ ├─ translation │ │ ├─ prototype - │ │ └─ screenshots + │ │ ├─ walkthrough + │ │ ├─ soundtrack + │ │ ├─ cheat + │ │ └─ screenshot │ │ │ └─ ps/ │ ├─ game_5/ @@ -139,7 +159,10 @@ Some games come as **folders** instead of single files, which could include mult │ │ ├─ demo │ │ ├─ translation │ │ ├─ prototype - │ │ └─ screenshots + │ │ ├─ walkthrough + │ │ ├─ soundtrack + │ │ ├─ cheat + │ │ └─ screenshot │ │ │ └─ bios/ │ └─ gba_bios.bin @@ -214,7 +237,7 @@ Game identity is content-based, so a template isn't a cage: move or rename a gam !!! warning "Relocation needs an identity" - Matching a moved file to its entry needs all three of its hashes (CRC, MD5, SHA-1), so hashing has to be on (see [`filesystem.skip_hash_calculation`](../reference/configuration-file.md#filesystemskip_hash_calculation)). Platforms RomM doesn't hash (Switch, PS3, PS4, the PC and mobile platforms) fall back to the title id read out of the binary. With neither available, or when two entries missing from the same platform share an identity, the file is imported as a new game and the old entry stays flagged as missing from the filesystem. + Matching a moved file to its entry needs all three of its hashes (CRC, MD5, SHA-1), so hashing has to be on (see [`filesystem.skip_hash_calculation`](../reference/configuration-file.md#filesystemskip_hash_calculation)). Platforms RomM doesn't hash (Switch, PS3, PS4, the PC and mobile platforms) fall back to the Title ID read out of the binary. With neither available, or when two entries missing from the same platform share an identity, the file is imported as a new game and the old entry stays flagged as missing from the filesystem. ### Notes diff --git a/docs/getting-started/metadata-providers.md b/docs/getting-started/metadata-providers.md index 6b395742..3904532d 100644 --- a/docs/getting-started/metadata-providers.md +++ b/docs/getting-started/metadata-providers.md @@ -103,6 +103,8 @@ You must run a LaunchBox metadata update (either manually, or scheduled via cron Simply set `HASHEOUS_API_ENABLED=true` in your environment variables, and future scans will start using the [Hasheous API](https://hasheous.org/swagger/index.html). +RomM uses the public instance at `hasheous.org` by default, but users running their own [self-hosted Hasheous](https://github.com/gaseous-project/hasheous) can point `HASHEOUS_API_URL` at its API base, e.g. `https://hasheous.example.com/api/v1`. + ### Playmatch [Playmatch](https://github.com/RetroRealm/playmatch) is a free, open source and community driven hash based matching service supporting multiple metadata providers such as IGDB, ScreenScraper, SteamGridDB, Retroachievements and more, hosted by a member of our community. @@ -142,7 +144,32 @@ The [Flashpoint Project Database](https://flashpointproject.github.io/flashpoint The [HowLongToBeat](https://howlongtobeat.com/) project provides game completion times for more than 84,000 games. Enable this metadata source with the `HLTB_API_ENABLED=true` environment variable. If you are adding this provider to an existing setup, perform a `UNMATCHED` scan with HowLongToBeat selected to update an existing platform. -Game completion times will be added to a new tab on the details page for supported matched games. +Game completion times will be added to a new tab on the details page for supported matched games. Once those are populated the gallery can sort and filter by game length. + +### Steam + +[Steam](https://store.steampowered.com/) is a metadata source for the `win`, `linux` and `mac` platforms. Enable it with `STEAM_API_ENABLED=true`; there is no API key, no account, and no rate-limit sign-up. You get the title, description, capsule art, screenshots, genres, developers, publishers, release date, game modes and the Metacritic score. + +```yaml +scan: + priority: + metadata: + - steam # PC platforms only +``` + +### Demozoo, Pouët and CSDb + +These demoscene databases cover productions (demos, intros, cracktros, musicdisks) rather than commercial games. All three are public APIs needing no key, and all three are off by default: + +| Provider | Variable | Covers | +| ------------------------------- | --------------------- | ------------------------------------------------------ | +| [Demozoo](https://demozoo.org/) | `DEMOZOO_API_ENABLED` | `win`, `dos`, `amiga`, `c64`, `nes`, `snes`, `genesis` | +| [Pouët](https://www.pouet.net/) | `POUET_API_ENABLED` | Whatever platforms the production itself declares | +| [CSDb](https://csdb.dk/) | `CSDB_API_ENABLED` | `c64`, used for stills Demozoo doesn't have | + +All three match on the production id, which you can give them three ways: a [filename tag](#metadata-tags-in-filenames), the bare ID pasted into the ROM editor, or a production URL pasted into the ROM editor (`https://demozoo.org/productions/108/`, `https://www.pouet.net/prod.php?which=108`, `https://csdb.dk/release/?id=75330`). + +Demozoo will also search by title, and if a Demozoo production references a CSDb release, the scan follows it automatically. Pouët only accepts a title search that lands on exactly one production. An ambiguous title is left unmatched rather than guessed at. ### ES-DE gamelist.xml @@ -282,6 +309,10 @@ Scans will now parse custom metadata tags in the filename that match specific pa (ssfr-xxxx) for [ScreenScraper](https://screenscraper.fr/) (launchbox-xxxx) for [Launchbox](https://gamesdb.launchbox-app.com/) (hltb-xxxx) for [HowLongToBeat](https://howlongtobeat.com/) +(steam-xxxx) for [Steam](https://store.steampowered.com/) +(demozoo-xxxx) for [Demozoo](https://demozoo.org/) +(pouet-xxxx) for [Pouët](https://www.pouet.net/) +(csdb-xxxx) for [CSDb](https://csdb.dk/) Filenames will not be renamed to add tags, as they are a non-standard formatting system and could create conflicts with other software. @@ -329,33 +360,9 @@ To use an alternate style end-to-end: ## Priority and conflict resolution -When multiple providers return different values for the same field, the winner is determined by `scan.priority.metadata` and `scan.priority.artwork` in `config.yml`. Defaults: +When multiple providers return different values for the same field, the winner is decided by `scan.priority.metadata` and `scan.priority.artwork` in `config.yml`. The current defaults and the full slug table live in [Configuration File → `scan.priority.metadata`](../reference/configuration-file.md#scanprioritymetadata). -```yaml -scan: - priority: - metadata: - - igdb - - moby - - ss - - ra - - launchbox - - gamelist - - hasheous - - flashpoint - - hltb - artwork: - - igdb - - moby - - ss - - ra - - launchbox - - libretro - - gamelist - - hasheous - - flashpoint - - hltb -``` +A provider that isn't enabled is skipped wherever it sits in the list, so leaving all of them in place costs nothing. Reorder these lists to taste. For example, put `ss` first if you prefer ScreenScraper boxart, or move `hltb` up if you care about completion times more than descriptions. diff --git a/docs/index.md b/docs/index.md index 71bdff8b..aba2b73b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,7 +17,7 @@ Welcome to the **RomM Project**, the premier self-hosted, open source ROM manage Website · Demo · Discord -RomM (ROM Manager) lets you scan, enrich, organise, and play your game collection from a clean web UI, with metadata from IGDB, ScreenScraper, MobyGames, RetroAchievements, Hasheous, LaunchBox, and more. In-browser play via EmulatorJS and Ruffle, companion apps for Android, handhelds, and desktop, and a first-class multi-user experience with OIDC SSO. +Scan, enrich, browse and play your ROM collection from one beautiful & free self-hosted app. Metadata from 10+ providers, save sync across your devices, and support for over 400 platforms. RomM is a must-have for anyone who plays on emulators. ## Where do you want to go? @@ -43,7 +43,7 @@ RomM (ROM Manager) lets you scan, enrich, organise, and play your game collectio *** - Library, collections, saves & states, ROM Patcher, Netplay. + Library, collections, saves & states, streaming, netplay. [Using RomM →](using/index.md) diff --git a/docs/install/databases.md b/docs/install/databases.md index f8145332..0c7f7d3f 100644 --- a/docs/install/databases.md +++ b/docs/install/databases.md @@ -124,6 +124,15 @@ services: retries: 5 ``` +## Connection pooling + +Database connections are pooled and discarded after `DB_POOL_RECYCLE_SECONDS` (default `300`). Without it, a quiet instance eventually hands out a connection the database already closed for being idle. Keep this **below** whatever idle timeout sits in front of your database: MySQL/MariaDB's `wait_timeout`, a provider's cap, or a proxy in between. `-1` disables recycling entirely, which is only a good idea if you're certain nothing is closing idle connections. + +```yaml +environment: + - DB_POOL_RECYCLE_SECONDS=180 # a proxy in front drops idle connections at 240s +``` + ## Extra connection parameters `DB_QUERY_JSON` takes a JSON blob of extra parameters appended to the connection string, e.g. for enabling TLS to an external DB, a longer connection timeout, or a non-default port: diff --git a/docs/platforms/supported-platforms.md b/docs/platforms/supported-platforms.md index fa4f9b77..899055c1 100644 --- a/docs/platforms/supported-platforms.md +++ b/docs/platforms/supported-platforms.md @@ -22,7 +22,7 @@ Name your folder after the **platform slug** in the table, or after one of the [ - **Slug**: the folder name RomM expects. Matches the IGDB platform slug where possible. Matched case-insensitively, so `SNES/` and `snes/` are the same platform. - **Name**: the human-readable platform name - **Providers**: which metadata providers have at least partial coverage (see [Metadata Providers](../getting-started/metadata-providers.md)). -- **EmulatorJS**: a playable in-browser core exists (see [Configuration File → `emulatorjs`](../reference/configuration-file.md#emulatorjs) for operator-level tuning). +- **EmulatorJS**: a playable in-browser core exists (see [Configuration File → `emulatorjs`](../reference/configuration-file.md#emulatorjs) for server owner-level tuning). - **Firmware**: platform needs BIOS files for emulation (see [Firmware Management](../administration/firmware-management.md)). ## Folder name aliases diff --git a/docs/reference/configuration-file.md b/docs/reference/configuration-file.md index 634cad4c..27d71229 100644 --- a/docs/reference/configuration-file.md +++ b/docs/reference/configuration-file.md @@ -35,7 +35,7 @@ exclude: Drop files with these extensions before matching, only for files that aren't inside a multi-file folder. -**Default:** `["db", "ini", "tmp", "bak", "lock", "log", "cache", "crdownload"]` +**Default:** `["db", "tmp", "bak", "lock", "log", "cache", "crdownload", "assembling"]` ```yaml exclude: @@ -48,7 +48,7 @@ exclude: Unix-glob file-name patterns to skip. -**Default:** `[".DS_Store", ".localized", ".Trashes", ".stfolder", "@SynoResource", "gamelist.xml"]` +**Default:** `[".DS_Store", ".localized", ".Trashes", ".stfolder", "@SynoResource", "*:Zone.Identifier", "gamelist.xml", "metadata.pegasus.txt"]` ```yaml exclude: @@ -61,7 +61,7 @@ exclude: Skip whole folders. Used for multi-disc/multi-file games you want invisible. -**Default:** `["@eaDir", "__MACOSX", "$RECYCLE.BIN", ".Trash-*", ".stfolder", ".Spotlight-V100", ".fseventsd", ".DocumentRevisions-V100", "System Volume Information"]` +The default already covers the system folders that are never a platform, plus every per-media-type folder ES-DE, Batocera and the [Pegasus export](exports.md) drop beside your ROMs (`covers`, `screenshots`, `manuals` and the rest). Your list gets added to that rather than replacing it. ```yaml exclude: @@ -74,7 +74,7 @@ exclude: Files **inside** a multi-file ROM folder to ignore (e.g. `.nfo`, `._*` macOS attributes, similar noise from multi-disc sets). -**Default:** `[".DS_Store", ".localized", ".Trashes", ".stfolder", "@SynoResource", "gamelist.xml"]` +**Default:** the same list as [`exclude.roms.single_file.names`](#excluderomssingle_filenames) ```yaml exclude: @@ -88,7 +88,7 @@ exclude: Extensions to ignore inside a multi-file ROM folder. -**Default:** `["db", "ini", "tmp", "bak", "lock", "log", "cache", "crdownload"]` +**Default:** the same list as [`exclude.roms.single_file.extensions`](#excluderomssingle_fileextensions) ```yaml exclude: @@ -180,6 +180,28 @@ filesystem: skip_hash_calculation: true ``` +### `filesystem.skip_title_id_extraction` + +Skip reading the platform-native Title ID out of ROM binaries. That ID is what identifies a game on non-hashed platforms, and it records where the game writes its saves, so expect worse matching on those platforms with this on. See [Title ids read from the binary](../administration/scanning-and-watcher.md#title-ids-read-from-the-binary) for which platforms have one and what the ID carries. + +**Default:** `false` + +```yaml +filesystem: + skip_title_id_extraction: true +``` + +### `filesystem.embed_switch_title_ids` + +Rename Switch ROMs on disk so their filename ends in `[TITLEID][vVERSION]`, which is what most Switch tooling expects to see. Disabled by default because it rewrites file names. + +**Default:** `false` + +```yaml +filesystem: + embed_switch_title_ids: true +``` + --- ## `scan` @@ -188,7 +210,7 @@ filesystem: Order metadata providers are queried during a scan. First match wins for descriptive fields (title, description, release date, etc.). -**Default:** `["igdb", "moby", "ss", "ra", "launchbox", "gamelist", "hasheous", "flashpoint", "hltb"]` +**Default:** `["igdb", "moby", "ss", "ra", "launchbox", "gamelist", "hasheous", "tgdb", "flashpoint", "steam", "hltb", "demozoo", "pouet", "csdb"]` ```yaml scan: @@ -201,25 +223,31 @@ scan: Values are the provider slugs. Full list: -| Slug | Provider | -| ------------ | --------------------- | -| `igdb` | IGDB | -| `moby` | MobyGames | -| `ss` | ScreenScraper | -| `ra` | RetroAchievements | -| `launchbox` | LaunchBox | -| `gamelist` | gamelist.xml importer | -| `hasheous` | Hasheous | -| `flashpoint` | Flashpoint | -| `hltb` | HowLongToBeat | -| `tgdb` | TheGamesDB | -| `libretro` | Libretro metadata | +| Slug | Provider | +| ------------ | ------------------------------- | +| `igdb` | IGDB | +| `moby` | MobyGames | +| `ss` | ScreenScraper | +| `ra` | RetroAchievements | +| `launchbox` | LaunchBox | +| `gamelist` | gamelist.xml importer | +| `hasheous` | Hasheous | +| `playmatch` | Playmatch | +| `flashpoint` | Flashpoint | +| `steam` | Steam, on the PC platforms only | +| `hltb` | HowLongToBeat | +| `demozoo` | Demozoo, demoscene productions | +| `pouet` | Pouët, demoscene productions | +| `csdb` | CSDb, C64 demoscene productions | +| `tgdb` | TheGamesDB | +| `sgdb` | SteamGridDB, artwork only | +| `libretro` | Libretro metadata, artwork only | See [Metadata Providers](../getting-started/metadata-providers.md) for context on each. ### `scan.priority.artwork` -Same idea, for cover art and screenshots. Defaults to the same order as `scan.priority.metadata` but can differ. +Same idea but for cover art and screenshots, with a default of its own: `["sgdb", "igdb", "moby", "ss", "libretro", "ra", "launchbox", "gamelist", "hasheous", "tgdb", "flashpoint", "steam", "hltb", "demozoo", "pouet", "csdb"]`. ```yaml scan: @@ -251,6 +279,8 @@ scan: Preferred region for titles, cover art, and regional variants. ScreenScraper uses this directly, and other providers respect it where possible. +This also decides which dump the gallery shows when you own several copies of a game. Siblings collapse into one card, and the winner is whichever region sits highest in this list, with pre-release dumps pushed below full releases. Regions you haven't listed come last, so a Japan-only release still wins when it's the only one there. + **Default:** `["us", "wor", "ss", "eu", "jp"]` ```yaml @@ -341,6 +371,8 @@ scan: image: screenshot ``` +`media.thumbnail` and `media.image` pick which [`scan.media`](#scanmedia) type fills the `` and `` tags. + ### `scan.pegasus.export` Export metadata in Pegasus-frontend format (`metadata.pegasus.txt`). @@ -351,6 +383,8 @@ scan: export: true ``` +Both exports are covered in full in [Exports](exports.md). + --- ## `emulatorjs` @@ -379,13 +413,39 @@ emulatorjs: ### `emulatorjs.disable_batch_bootup` -DOS-specific knob that skips the `autorun.bat` step. Toggle if DOS games won't boot. +Multi-disc games hand EmulatorJS every disc at once, which lets the emulator swap between them from its own menu without a reload. Set this to go back to booting only the disc you launched, which is worth trying if a core doesn't cope with the batch. + +**Default:** `false` ```yaml emulatorjs: disable_batch_bootup: true ``` +### `emulatorjs.default_cores` + +Preselect the libretro core for a platform, keyed by [platform slug](../platforms/supported-platforms.md). Players who have already chosen a core on their device will keep defaulting to it. + +```yaml +emulatorjs: + default_cores: + nds: desmume + nintendo-dsi: melonds +``` + +Core names have to be exact, and anything you don't list keeps EmulatorJS's own default. + +### `emulatorjs.auto_save_sync` + +Upload a save every time the emulator writes one, rather than only on save-and-quit. Closing the tab or suffering a crash mid-game then loses nothing. + +**Default:** `false` + +```yaml +emulatorjs: + auto_save_sync: true +``` + ### `emulatorjs.disable_auto_unload` By default, EmulatorJS stops the emulator when you leave its page. Disable to keep it running across navigation. @@ -475,15 +535,15 @@ emulatorjs: 1: { value: "'", value2: "BUTTON_3" } ``` -### Operator-level vs per-user +### Server owner vs per-user Most settings under `emulatorjs.settings` and `emulatorjs.controls` can be overridden by users in-game (Menu → Settings, Menu → Controls). Per-user values take precedence, the config.yml setting is the fallback. -| Where the setting lives | Who it affects | Survives upgrades? | -| --------------------------------- | -------------------- | ------------------ | -| Operator: `config.yml`/env vars | Everyone, as default | Yes | -| Per-user: in-game Menu → Settings | Just that user | Yes | -| Per-user: in-game Menu → Controls | Just that user | Yes | +| Where the setting lives | Who it affects | Survives upgrades? | +| ----------------------------------- | -------------------- | ------------------ | +| Server owner: `config.yml`/env vars | Everyone, as default | Yes | +| Per-user: in-game Menu → Settings | Just that user | Yes | +| Per-user: in-game Menu → Controls | Just that user | Yes | --- @@ -502,36 +562,49 @@ streaming: ### `streaming.containers` -One entry per emulator container, where each entry maps a [platform slug](../platforms/supported-platforms.md) to the container that streams it. +**One entry per container**, not per platform. A container serves every platform listed in its `platforms` map, and its own keys are the defaults for all of them. -| Key | Required | Purpose | -| ------------------ | -------- | -------------------------------------------------------------------------------------------------------------- | -| `platform` | Yes | Platform slug this container serves (e.g. `ps2`, `ngc`, `wii`, `xbox`) | -| `host` | Yes | Browser-facing Selkies web UI (must be reachable from clients and served over **HTTPS**) | -| `broker_host` | No | Server-side broker API (derived from `host` if omitted) | -| `label` | Yes | Text shown on the play action (e.g. `PCSX2`) | -| `broker_secret` | No | Secret for this container, used only when the `STREAMING_BROKER_SECRET` env var is unset | -| `memory_card_sync` | No | Sync the whole memory card to the RomM library on `ps2` and `ngc` (GameCube), ignored on cardless platforms | -| `library_path` | No | In-container path to the RomM library if it is mounted somewhere other than the default `/romm/library` | -| `emulator` | No | Lowercased name grouping this container's states and memory cards, defaults to `label`, then the platform slug | +| Key | Required | Purpose | +| ------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `host` | Yes | Browser-facing Selkies web UI, served over **HTTPS**, or a path when reverse proxied onto RomM's own origin | +| `platforms` | Yes | Map of [platform slug](../platforms/supported-platforms.md) to the emulator serving it, or to an override block | +| `protocol` | No | `webstation`. Omitted, the entry is read as a deprecated per-emulator broker mod | +| `label` | No | Name for the container, shown in the fleet view. The play action is named after the emulator instead | +| `subfolder` | No | URL prefix the broker is served under, matching the container's `SUBFOLDER` | +| `broker_host` | No | Server-to-broker API base. Derived from `host` when omitted, and **required** when `host` is a path | +| `broker_secret` | No | Secret for this container, used only when the `STREAMING_BROKER_SECRET` env var is unset | +| `library_path` | No | In-container path to the RomM library, if it is mounted somewhere other than the default `/romm/library` | +| `emulator` | No | Lowercased name grouping states and memory cards. Ignored when `platforms` is used, since each platform's own emulator names them | +| `memory_card_sync` | No | Sync the whole memory card to the RomM library. Ignored only on platforms known to have no card (`wii`, `psx`, `ps3`, `ps4`, `xbox`, `xbox360`, `wiiu`, `3ds`, `switch`) | -See [Emulator Streaming → Memory cards](../using/emulator-streaming.md#memory-cards) for how `memory_card_sync` behaves. +Each `platforms` value is either the emulator name on its own, or a block overriding `emulator`, `label` and `memory_card_sync` for that platform. ```yaml streaming: enabled: true containers: - - platform: ps2 - host: https://192.168.1.51:3001 # browser-facing, must be HTTPS - broker_host: http://192.168.1.51:8000 # server-to-container, HTTP ok - label: PCSX2 - memory_card_sync: true # keep the PS2 memory card in your RomM library - - platform: ngc # ngc/wii can share one Dolphin container - host: https://192.168.1.51:3002 - broker_host: http://192.168.1.51:8001 - label: Dolphin + - protocol: webstation + host: https://192.168.1.56:3010 # browser-facing, must be HTTPS + subfolder: /streaming # matches the container's SUBFOLDER + library_path: /romm # where it mounts your ROM library + broker_secret: change-me # matches the container's BROKER_SECRET + label: Emulation station + platforms: + snes: retroarch # the emulator name directly... + ps2: # ...or a block overriding container keys + emulator: pcsx2 + label: PCSX2 + memory_card_sync: true + ngc: + emulator: dolphin + label: Dolphin + memory_card_sync: true ``` +Platforms listed on several containers form a pool, with each claim taking the first free lane. Pool members have to agree on `emulator`, `memory_card_sync` and `protocol`, and are differentiated by broker host, so give each one a distinct `broker_host` (see [Emulator Streaming → How a session works](../using/emulator-streaming.md#how-a-session-works)). + +See [Emulator Streaming → Memory cards](../using/emulator-streaming.md#memory-cards) for how `memory_card_sync` behaves, and [Migrating to webstation](../using/emulator-streaming-migration.md) if you still run the per-emulator broker mods. + --- ## Related diff --git a/docs/reference/exports.md b/docs/reference/exports.md index dd0b5a9c..cbce47b8 100644 --- a/docs/reference/exports.md +++ b/docs/reference/exports.md @@ -24,6 +24,8 @@ scan: With `export: true`, every scan writes a `gamelist.xml` into the platform folder, and downloads the selected media into sibling folders (`covers/`, `screenshots/`, etc.) that ES-DE expects. +Existing entries in `gamelist.xml` are rewritten with new data, so exporting into a library your frontend already scraped won't throw that work away. Games in subfolders keep their folder in the exported ``, relative to the platform folder, and their media mirrors the same folders. + Standard ES-DE/EmulationStation format: ```xml @@ -85,6 +87,8 @@ scan: export: true ``` +An existing `metadata.pegasus.txt` gets merged rather than overwritten. Both exports also **share their media folders**, so turning both on gives you one copy of each cover and screenshot rather than two. + Human-readable text format: ```text diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index b36c8720..3519149d 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -29,11 +29,13 @@ Every term the docs, UI, and API use consistently, with foundational concepts ge **EmulatorJS**: the bundled in-browser retro emulator. Handles NES, SNES, N64, PSX, Saturn, and 20+ more cores (see [In-Browser Play → EmulatorJS](../using/in-browser-play/emulatorjs.md)). +**Facet**: one axis a [recommendation](../using/recommendations.md) is explained by, such as a shared franchise, genre, theme or developer. A facet counts for as much as it is rare in your own library. + **Feed**: a URL endpoint that exposes a filtered library view in a third-party tool's expected format (see [Feed Clients](../ecosystem/feed-clients.md)). -**Firmware**: BIOS or system firmware required for certain emulators (PS1, GBA, Saturn, etc.). Lives under `/romm/library/bios/{platform}/` or `/romm/library/{platform}/bios/` (depending on which folder structure you chose). Uploaded via the UI and managed by admins and users with the `firmware.write` scope (see [Firmware Management](../administration/firmware-management.md)). +**Firmware**: BIOS or system firmware required for certain emulators (PS1, GBA, Saturn, etc.). Lives wherever the [`filesystem.structure.firmware`](configuration-file.md#filesystemstructure) template points, `bios/{platform}` by default. Uploaded via the UI and managed by admins and users with the `firmware.write` scope (see [Firmware Management](../administration/firmware-management.md)). -**Full image**: the default container variant, including EmulatorJS + Ruffle. `rommapp/romm:X.Y.Z` (see [Image Variants](../install/image-variants.md)). +**Full image**: the default container variant, bundling all four browser runtimes (EmulatorJS, Ruffle, `js-dos`, FAKE-08). `rommapp/romm:X.Y.Z` (see [Image Variants](../install/image-variants.md)). **Game Data tab**: the ROM detail page tab for saves, states, and screenshots. User-specific. @@ -41,11 +43,15 @@ Every term the docs, UI, and API use consistently, with foundational concepts ge **Invite link**: single-use URL that lets a new user register with a pre-assigned role (see [Invitations & Registration](../administration/invitations-and-registration.md)). +**Jukebox**: the library-wide soundtrack player, reading the audio files in each game's `soundtrack/` folder (see [Jukebox](../using/jukebox.md)). + **Kekatsu**: Nintendo DS multiboot loader that reads RomM's feed (see [Kekatsu](../ecosystem/feed-clients.md#kekatsu)). **Kiosk mode**: server-side setting (`KIOSK_MODE=true`) that turns every read endpoint into unauthenticated access. Anonymous visitors can browse but nobody can write, which fits public demos and wall displays (see [Authentication → Kiosk mode](../administration/authentication.md#kiosk-mode)). -**Library**: your ROM files on disk. Mounted (usually read-only) at `/romm/library` inside the container, with platforms as subdirectories. The catalogue is built from what's found there (see [Folder Structure](../getting-started/folder-structure.md)). +**Library**: your ROM files on disk. Mounted at `/romm/library` inside the container, and laid out according to your [structure templates](../getting-started/folder-structure.md#custom-library-structure). The catalogue is built from what's found there (see [Folder Structure](../getting-started/folder-structure.md)). + +**Memory card**: on PS2 and GameCube, the emulator's whole card kept in your RomM library rather than on the streaming container, with versions and sharing (see [Emulator Streaming → Memory cards](../using/emulator-streaming.md#memory-cards)). **Metadata provider**: external source of game data, queried during a scan, with results merged. Configured via env vars + priority in `config.yml` (see [Metadata Providers](../getting-started/metadata-providers.md)). @@ -59,6 +65,8 @@ Every term the docs, UI, and API use consistently, with foundational concepts ge **Personal tab**: the ROM detail page tab for per-user data (rating, status, notes, playtime). +**Physical game**: a library entry for a copy you own on cartridge or disc, with no file on disk. Added by name or by barcode, and excluded from anything that needs a real file (see [Physical Games](../using/physical-games.md)). + **Platform**: a gaming system: SNES, PlayStation, Game Boy Advance, DOS, etc. ~400 platforms ship supported. Each has a **slug** (`snes`, `psx`, `gba`) that doubles as the folder name expected in your library, alongside the [folder name aliases](../platforms/supported-platforms.md#folder-name-aliases) other frontends use. Override the folder-name → slug mapping via `config.yml` (see [Supported Platforms](../platforms/supported-platforms.md)). **Play session**: a timestamped record of someone playing a ROM (start, end, duration, device). Used by the stats, the Continue Playing ribbon, and per-ROM playtime totals. Ingested automatically when playing in-browser, and companion apps push them via API. @@ -77,6 +85,8 @@ Every term the docs, UI, and API use consistently, with foundational concepts ge **Scope**: a coarse OAuth permission derived from a user's effective [permission-group](../administration/users-and-roles.md#permission-groups) grants. Client API Tokens and OIDC sessions carry a subset of the user's scopes (see [Users & Roles → API tokens](../administration/users-and-roles.md#api-tokens-advanced)). +**Session (streaming)**: one claim on a streaming container. A container drives one display, so it holds one session at a time, bound to the user who claimed it (see [Emulator Streaming](../using/emulator-streaming.md#how-a-session-works)). + **Setup Wizard**: first-run flow that creates the admin user. Shown before any user exists. **Smart Collection**: rule-based auto-populating collection (see [Smart Collections](../using/smart-collections.md)). @@ -87,13 +97,19 @@ Every term the docs, UI, and API use consistently, with foundational concepts ge **Tinfoil**: Nintendo Switch homebrew that installs from RomM's feed (see [Tinfoil](../ecosystem/feed-clients.md#tinfoil)). +**Title id**: the platform-native identifier read out of a ROM's own binary during a scan, on the platforms that have one. Identifies a game where RomM doesn't hash, and says where the game writes its saves (see [Scanning & Watcher](../administration/scanning-and-watcher.md#title-ids-read-from-the-binary)). + **User**: an account. Its role is either User (access from a permission group plus per-user overrides) or Admin (full access). Can be created by the Setup Wizard, an admin, an invite link, or OIDC auto-provisioning (see [Users & Roles](../administration/users-and-roles.md)). **Valkey**: open-source Redis fork, drop-in compatible (see [Redis or Valkey](../install/redis-or-valkey.md)). **Virtual Collection**: auto-generated collection by genre/developer/year/tag. Read-only (see [Virtual Collections](../using/virtual-collections.md)). -**Watcher**: filesystem watcher that triggers scans on file events. `WATCHER_ENABLED=true` (see [Scanning & Watcher](../administration/scanning-and-watcher.md#filesystem-watcher)). +**Walkthrough**: a guide document attached to a game, stored in its `walkthrough/` folder beside its manual, uploaded or imported from a GameFAQs URL, with per-user reading progress (see [Walkthroughs](../using/walkthroughs.md)). + +**Watcher**: filesystem watcher that triggers scans on file events. `ENABLE_RESCAN_ON_FILESYSTEM_CHANGE=true` (see [Scanning & Watcher](../administration/scanning-and-watcher.md#filesystem-watcher)). + +**webstation**: the single container every streamed emulator runs in, serving as many platforms as you point at it (see [Emulator Streaming](../using/emulator-streaming.md)). --- diff --git a/docs/resources/snippets/env-vars.md b/docs/resources/snippets/env-vars.md index 9abdc432..e2864883 100644 --- a/docs/resources/snippets/env-vars.md +++ b/docs/resources/snippets/env-vars.md @@ -12,16 +12,17 @@ ### Database -| Variable | Default | Required | Description | -| ---------------- | --------- | :------: | ------------------------------------------------------------------------ | -| `ROMM_DB_DRIVER` | `mariadb` | | Database driver to use (mariadb, mysql, postgresql) | -| `DB_HOST` | | `✓` | Host name of the database instance | -| `DB_PORT` | `3306` | | Port number of the database instance | -| `DB_NAME` | `romm` | | Database name (should match MYSQL_DATABASE in MariaDB) | -| `DB_USER` | | `✓` | Database username (should match MARIADB_USER in MariaDB) | -| `DB_PASSWD` | | `✓` | Database password (should match MARIADB_PASSWORD in MariaDB) | -| `DB_ROOT_PASSWD` | | | Database root user password (only used by the bundled MariaDB container) | -| `DB_QUERY_JSON` | | | Extra query parameters for the database connection, as JSON | +| Variable | Default | Required | Description | +| ------------------------- | --------- | :------: | ----------------------------------------------------------------------------------------------------------- | +| `ROMM_DB_DRIVER` | `mariadb` | | Database driver to use (mariadb, mysql, postgresql) | +| `DB_HOST` | | `✓` | Host name of the database instance | +| `DB_PORT` | `3306` | | Port number of the database instance | +| `DB_NAME` | `romm` | | Database name (should match MYSQL_DATABASE in MariaDB) | +| `DB_USER` | | `✓` | Database username (should match MARIADB_USER in MariaDB) | +| `DB_PASSWD` | | `✓` | Database password (should match MARIADB_PASSWORD in MariaDB) | +| `DB_ROOT_PASSWD` | | | Database root user password (only used by the bundled MariaDB container) | +| `DB_QUERY_JSON` | | | Extra query parameters for the database connection, as JSON | +| `DB_POOL_RECYCLE_SECONDS` | `300` | | Retire a pooled connection after this long, before the server drops it for being idle (-1 to never recycle) | ### Redis/Valkey @@ -76,46 +77,60 @@ ### Metadata Providers -| Variable | Default | Required | Description | -| -------------------------------------- | ------- | :------: | --------------------------------------------------------- | -| `IGDB_CLIENT_ID` | | | Client ID for the IGDB API | -| `IGDB_CLIENT_SECRET` | | | Client secret for the IGDB API | -| `MOBYGAMES_API_KEY` | | | MobyGames secret API key | -| `SCREENSCRAPER_USER` | | | Screenscraper username | -| `SCREENSCRAPER_PASSWORD` | | | Screenscraper password | -| `STEAMGRIDDB_API_KEY` | | | SteamGridDB secret API key | -| `RETROACHIEVEMENTS_API_KEY` | | | RetroAchievements secret API key | -| `REFRESH_RETROACHIEVEMENTS_CACHE_DAYS` | `30` | | RetroAchievements metadata cache refresh interval in days | -| `PLAYMATCH_API_ENABLED` | `false` | | Enable PlayMatch API integration | -| `LAUNCHBOX_API_ENABLED` | `false` | | Enable LaunchBox API integration | -| `HASHEOUS_API_ENABLED` | `false` | | Enable Hasheous API integration | -| `FLASHPOINT_API_ENABLED` | `false` | | Enable Flashpoint API integration | -| `HLTB_API_ENABLED` | `false` | | Enable HowLongToBeat API integration | -| `TGDB_API_ENABLED` | `false` | | Enable TheGamesDB API integration | +| Variable | Default | Required | Description | +| -------------------------------------- | ------- | :------: | --------------------------------------------------------------------------------- | +| `IGDB_CLIENT_ID` | | | Client ID for the IGDB API | +| `IGDB_CLIENT_SECRET` | | | Client secret for the IGDB API | +| `MOBYGAMES_API_KEY` | | | MobyGames secret API key | +| `SCREENSCRAPER_USER` | | | Screenscraper username | +| `SCREENSCRAPER_PASSWORD` | | | Screenscraper password | +| `STEAMGRIDDB_API_KEY` | | | SteamGridDB secret API key | +| `RETROACHIEVEMENTS_API_KEY` | | | RetroAchievements secret API key | +| `REFRESH_RETROACHIEVEMENTS_CACHE_DAYS` | `30` | | RetroAchievements metadata cache refresh interval in days | +| `PLAYMATCH_API_ENABLED` | `false` | | Enable PlayMatch API integration | +| `LAUNCHBOX_API_ENABLED` | `false` | | Enable LaunchBox API integration | +| `HASHEOUS_API_ENABLED` | `false` | | Enable Hasheous API integration | +| `FLASHPOINT_API_ENABLED` | `false` | | Enable Flashpoint API integration | +| `HLTB_API_ENABLED` | `false` | | Enable HowLongToBeat API integration | +| `DEMOZOO_API_ENABLED` | `false` | | Enable Demozoo (filename tags (demozoo-N) / paste ID; no API key) | +| `POUET_API_ENABLED` | `false` | | Enable Pouët (filename tags (pouet-N) / paste ID; no API key) | +| `CSDB_API_ENABLED` | `false` | | Enable CSDb (filename tags (csdb-N) / paste ID / Demozoo CsdbRelease; no API key) | +| `STEAM_API_ENABLED` | `false` | | Enable Steam API integration (PC platforms only) | +| `TGDB_API_ENABLED` | `false` | | Enable TheGamesDB API integration | + +### Physical Games + +| Variable | Default | Required | Description | +| -------------------- | --------------------------------------------- | :------: | ------------------------------------------------------ | +| `UPC_LOOKUP_ENABLED` | `true` | | Look a barcode up by UPC when adding a physical game | +| `UPC_LOOKUP_API_KEY` | | | Key for the UPC lookup service, if your plan needs one | +| `UPC_LOOKUP_URL` | `https://api.upcitemdb.com/prod/trial/lookup` | | UPC lookup endpoint | ### Scans & Tasks -| Variable | Default | Required | Description | -| -------------------------------------------------- | ----------- | :------: | ----------------------------------------------------------------------------------------- | -| `SCAN_TIMEOUT` | `14400` | | Timeout for background scan/rescan tasks in seconds | -| `SCAN_WORKERS` | `4` | | How many ROMs a scan processes at once | -| `TASK_TIMEOUT` | `300` | | Timeout for other background tasks in seconds | -| `TASK_RESULT_TTL` | `86400` | | How long to keep task results in Valkey in seconds | -| `SEVEN_ZIP_TIMEOUT` | `60` | | Timeout for 7-Zip operations in seconds | -| `ENABLE_RESCAN_ON_FILESYSTEM_CHANGE` | `false` | | Re-scan the library automatically when the filesystem changes | -| `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` | `5` | | Delay in minutes before re-scanning after a filesystem change | -| `ENABLE_SCHEDULED_RESCAN` | `false` | | Enable scheduled library re-scans | -| `SCHEDULED_RESCAN_CRON` | `0 3 * * *` | | Cron expression for scheduled re-scans | -| `ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB` | `false` | | Enable scheduled Switch TitleDB index updates | -| `SCHEDULED_UPDATE_SWITCH_TITLEDB_CRON` | `0 4 * * *` | | Cron expression for scheduled Switch TitleDB updates | -| `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA` | `false` | | Enable scheduled LaunchBox metadata updates | -| `SCHEDULED_UPDATE_LAUNCHBOX_METADATA_CRON` | `0 4 * * *` | | Cron expression for scheduled LaunchBox metadata updates | -| `ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP` | `false` | | Enable scheduled conversion of images to WebP | -| `SCHEDULED_CONVERT_IMAGES_TO_WEBP_CRON` | `0 4 * * *` | | Cron expression for scheduled WebP conversion | -| `ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES` | `false` | | Enable scheduled cleanup of orphaned resources (covers, screenshots) left by deleted ROMs | -| `SCHEDULED_CLEANUP_ORPHANED_RESOURCES_CRON` | `0 5 * * *` | | Cron expression for scheduled orphaned resource cleanup | -| `ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC` | `false` | | Enable scheduled RetroAchievements progress sync | -| `SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC_CRON` | `0 4 * * *` | | Cron expression for scheduled RetroAchievements sync | +| Variable | Default | Required | Description | +| -------------------------------------------------- | ------------ | :------: | ----------------------------------------------------------------------------------------- | +| `SCAN_TIMEOUT` | `14400` | | Timeout for background scan/rescan tasks in seconds | +| `SCAN_WORKERS` | `4` | | How many ROMs a scan processes at once | +| `TASK_TIMEOUT` | `300` | | Timeout for other background tasks in seconds | +| `TASK_RESULT_TTL` | `86400` | | How long to keep task results in Valkey in seconds | +| `SEVEN_ZIP_TIMEOUT` | `60` | | Timeout for 7-Zip operations in seconds | +| `ENABLE_RESCAN_ON_FILESYSTEM_CHANGE` | `false` | | Re-scan the library automatically when the filesystem changes | +| `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` | `5` | | Delay in minutes before re-scanning after a filesystem change | +| `ENABLE_SCHEDULED_RESCAN` | `false` | | Enable scheduled library re-scans | +| `SCHEDULED_RESCAN_CRON` | `0 3 * * *` | | Cron expression for scheduled re-scans | +| `ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB` | `false` | | Enable scheduled Switch TitleDB index updates | +| `SCHEDULED_UPDATE_SWITCH_TITLEDB_CRON` | `0 4 * * *` | | Cron expression for scheduled Switch TitleDB updates | +| `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA` | `false` | | Enable scheduled LaunchBox metadata updates | +| `SCHEDULED_UPDATE_LAUNCHBOX_METADATA_CRON` | `0 4 * * *` | | Cron expression for scheduled LaunchBox metadata updates | +| `ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP` | `false` | | Enable scheduled conversion of images to WebP | +| `SCHEDULED_CONVERT_IMAGES_TO_WEBP_CRON` | `0 4 * * *` | | Cron expression for scheduled WebP conversion | +| `ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES` | `false` | | Enable scheduled cleanup of orphaned resources (covers, screenshots) left by deleted ROMs | +| `SCHEDULED_CLEANUP_ORPHANED_RESOURCES_CRON` | `0 5 * * *` | | Cron expression for scheduled orphaned resource cleanup | +| `ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC` | `false` | | Enable scheduled RetroAchievements progress sync | +| `SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC_CRON` | `0 4 * * *` | | Cron expression for scheduled RetroAchievements sync | +| `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS` | `true` | | Enable the scheduled rebuild of the recommendations index | +| `SCHEDULED_BUILD_RECOMMENDATIONS_CRON` | `30 5 * * *` | | Cron expression for the recommendations index rebuild | ### Sync @@ -130,11 +145,12 @@ ### Emulation -| Variable | Default | Required | Description | -| --------------------- | ------- | :------: | ----------------------------------------------------------- | -| `DISABLE_EMULATOR_JS` | `false` | | Disable in-browser play via EmulatorJS | -| `DISABLE_RUFFLE_RS` | `false` | | Disable in-browser Flash playback via RuffleRS | -| `DISABLE_JSDOS` | `false` | | Disable in-browser Windows 3.x and 9x playback via `js-dos` | +| Variable | Default | Required | Description | +| --------------------- | ------- | :------: | -------------------------------------------------------- | +| `DISABLE_EMULATOR_JS` | `false` | | Disable in-browser play via EmulatorJS | +| `DISABLE_RUFFLE_RS` | `false` | | Disable in-browser Flash playback via RuffleRS | +| `DISABLE_JSDOS` | `false` | | Disable in-browser Win3.x and Win9.x playback via js-dos | +| `DISABLE_PICO8` | `false` | | Disable in-browser PICO-8 playback via FAKE-08 | ### Integrations @@ -202,7 +218,9 @@ ### Emulator Streaming -| Variable | Default | Required | Description | -| ------------------------- | ------- | :------: | --------------------------------------------------------------------------------- | -| `STREAMING_BROKER_SECRET` | | | - | -| `STREAMING_SAVE_TIMEOUT` | `45` | | Seconds to wait for a broker save-and-exit (raise if a broker has SAVE_WAIT > 45) | +| Variable | Default | Required | Description | +| ------------------------------- | ------- | :------: | -------------------------------------------------------------------------------------------------- | +| `STREAMING_BROKER_SECRET` | | | Shared secret matching streaming containers' BROKER_SECRET, required for broker auth | +| `STREAMING_SAVE_TIMEOUT` | `45` | | Seconds to wait for a broker save-and-exit (raise if a broker has SAVE_WAIT > 45) | +| `STREAMING_LAUNCH_TIMEOUT` | `600` | | Seconds a webstation activate may take, covering pkg/archive extraction before the emulator starts | +| `STREAMING_STATE_HISTORY_LIMIT` | `50` | | Save states kept per ROM, emulator and user; oldest are pruned past this (0 disables) | diff --git a/docs/resources/snippets/scheduled-tasks.md b/docs/resources/snippets/scheduled-tasks.md index 79c86ae9..b920895a 100644 --- a/docs/resources/snippets/scheduled-tasks.md +++ b/docs/resources/snippets/scheduled-tasks.md @@ -2,18 +2,20 @@ | Task | Type | Default schedule | Enable var | Schedule/delay var | Purpose | | ----------------------------------------- | --------- | ---------------- | -------------------------------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| Scheduled netplay cleanup | Scheduled | `*/30 * * * *` | `-` | `-` | Cleans up empty netplay rooms. Always on, not configurable. | -| Scheduled ZIP cache cleanup | Scheduled | `0 4 * * *` | `-` | `-` | Removes stale cached ZIP files based on tiered TTL. Always on, not configurable. | -| Scheduled upload tmp cleanup | Scheduled | `0 * * * *` | `-` | `-` | Cleans up orphaned chunked-upload temp directories. Always on, not configurable. | -| Cleanup orphaned resources | Scheduled | `0 5 * * *` | `ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES` | `SCHEDULED_CLEANUP_ORPHANED_RESOURCES_CRON` | Clean up orphaned resources in the ROMs directory. | | Scheduled rescan | Scheduled | `0 3 * * *` | `ENABLE_SCHEDULED_RESCAN` | `SCHEDULED_RESCAN_CRON` | Rescans the entire library. | -| Scheduled Switch TitleDB update | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB` | `SCHEDULED_UPDATE_SWITCH_TITLEDB_CRON` | Updates the Nintendo Switch TitleDB file. | | Scheduled LaunchBox metadata update | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA` | `SCHEDULED_UPDATE_LAUNCHBOX_METADATA_CRON` | Updates the LaunchBox metadata store. | +| Scheduled Switch TitleDB update | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB` | `SCHEDULED_UPDATE_SWITCH_TITLEDB_CRON` | Updates the Nintendo Switch TitleDB file. | +| Build recommendations index | Scheduled | `30 5 * * *` | `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS` | `SCHEDULED_BUILD_RECOMMENDATIONS_CRON` | Rebuilds the similar-games index from library metadata, play history and collections. | | Convert images to WebP | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP` | `SCHEDULED_CONVERT_IMAGES_TO_WEBP_CRON` | Convert existing image files (PNG, JPG, BMP, TIFF, GIF) to WebP format for better performance. | +| Scheduled ZIP cache cleanup | Scheduled | `0 4 * * *` | `-` | `-` | Removes stale cached ZIP files based on tiered TTL. Always on, not configurable. | +| Cleanup orphaned resources | Scheduled | `0 5 * * *` | `ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES` | `SCHEDULED_CLEANUP_ORPHANED_RESOURCES_CRON` | Clean up orphaned resources in the ROMs directory. | +| Scheduled netplay cleanup | Scheduled | `*/30 * * * *` | `-` | `-` | Cleans up empty netplay rooms. Always on, not configurable. | +| Scheduled upload tmp cleanup | Scheduled | `0 * * * *` | `-` | `-` | Cleans up orphaned chunked-upload temp directories. Always on, not configurable. | | Scheduled RetroAchievements progress sync | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC` | `SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC_CRON` | Updates RetroAchievements progress for all users. | | Push-Pull Sync | Scheduled | `*/30 * * * *` | `ENABLE_SYNC_PUSH_PULL` | `SYNC_PUSH_PULL_CRON` | Sync saves with devices via SSH/SFTP. | | Cleanup missing ROMs | Manual | `-` | `-` | `-` | Delete all ROMs flagged as missing from the filesystem from the database. | -| Recompute save content hashes | Manual | `-` | `-` | `-` | Re-scan every save row and rewrite content_hash with the current compute_content_hash algorithm. One-time recovery after the zip-hash dispatch fix. | +| Cleanup missing firmware | Manual | `-` | `-` | `-` | Delete all firmware flagged as missing from the filesystem from the database. | | Sync Folder Scan | Manual | `-` | `ENABLE_SYNC_FOLDER_WATCHER` | `-` | Scan device sync folders for new save files. | +| Recompute save content hashes | Manual | `-` | `-` | `-` | Re-scan every save row and rewrite content_hash with the current compute_content_hash algorithm. One-time recovery after the zip-hash dispatch fix. | | Filesystem watcher | Watcher | `-` | `ENABLE_RESCAN_ON_FILESYSTEM_CHANGE` | `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` | Watch the library folder and trigger a rescan on changes. | | Sync folder watcher | Watcher | `-` | `ENABLE_SYNC_FOLDER_WATCHER` | `SYNC_FOLDER_SCAN_DELAY` | Watch the sync folder and trigger a scan on changes. | diff --git a/docs/resources/snippets/supported-platforms.md b/docs/resources/snippets/supported-platforms.md index 2b0e1b83..428e5d10 100644 --- a/docs/resources/snippets/supported-platforms.md +++ b/docs/resources/snippets/supported-platforms.md @@ -123,6 +123,7 @@ | Dreamcast | `dc` | igdb logo screenscraper logo mobygames logo launchbox logo hasheous logo retroachivements logo howlongtobeat logo libretro logo | | DVD Player | `dvd-player` | igdb logo mobygames logo howlongtobeat logo | | e-Reader / Card-e Reader | `e-reader-slash-card-e-reader` | igdb logo | +| EasyRPG | `rpg-maker` | screenscraper logo libretro logo | | ECD Micromind | `ecd-micromind` | mobygames logo | | EDSAC | `edsac` | igdb logo | | Elektor TV Games Computer | `elektor` | igdb logo retroachivements logo | diff --git a/docs/troubleshooting/in-browser-play.md b/docs/troubleshooting/in-browser-play.md index 5bd0cd7b..5ecf065d 100644 --- a/docs/troubleshooting/in-browser-play.md +++ b/docs/troubleshooting/in-browser-play.md @@ -7,10 +7,14 @@ description: Diagnose EmulatorJS and Ruffle issues ## EmulatorJS won't load at all -- **On the slim image without internet?** The slim image fetches EmulatorJS cores from a CDN at runtime rather than bundling them, so without outbound network the container can't load games. Either switch to the full image (cores bundled) or open outbound access (see [Image Variants](../install/image-variants.md)). +- **On the slim image without internet?** The slim image fetches EmulatorJS cores from a CDN at runtime rather than bundling them, so without outbound network the browser can't load games. Either switch to the full image (cores bundled) or open outbound access (see [Image Variants](../install/image-variants.md)). Ruffle and PICO-8 are full-image only. - Check the **browser console** and look for 404s on `/assets/emulatorjs/...`, which indicate the EmulatorJS bundle didn't install correctly in the container. Check `docker logs romm` for entrypoint install-step failures. - **Browser compatibility**: EmulatorJS uses SharedArrayBuffer, which needs a modern Chrome/Firefox/Safari and an HTTPS-served instance (cross-origin isolation requires HTTPS). If you're still on plain HTTP, set up TLS first (see [Reverse Proxy](../install/reverse-proxy.md)). +## "This core needs a secure connection" + +Threaded cores need the `SharedArrayBuffer` API, and browsers only hand that out on a secure, cross-origin-isolated origin. Over plain HTTP the player now tells you this directly instead of throwing a generic error at you. Fix it by serving the app over `https://` (see [Reverse Proxy](../install/reverse-proxy.md)), or pick a single-threaded core for the platform. It comes up with the PSP core, with [`js-dos`](../using/in-browser-play/js-dos.md) for Windows 3.x and 9x, and with a few of the heavier EmulatorJS cores. + ## Black screen or no audio - **Core incompatibility.** Some cores have issues with specific ROMs. Try a different core via in-game Menu → **Core**. diff --git a/docs/troubleshooting/netplay.md b/docs/troubleshooting/netplay.md index dd1db0ee..0d04f9b3 100644 --- a/docs/troubleshooting/netplay.md +++ b/docs/troubleshooting/netplay.md @@ -9,10 +9,10 @@ Netplay uses WebRTC + ICE servers for peer-to-peer connections, so most issues a ## `Failed to start game` -The most common error, and almost always the operator-side config: +The most common error, and almost always the server owner config: 1. **Is Netplay enabled?** `emulatorjs.netplay.enabled: true` in `config.yml`. -2. **Are ICE servers configured?** The operator needs at least one STUN server in `emulatorjs.netplay.ice_servers`. Without any, NAT traversal can't begin. +2. **Are ICE servers configured?** The server owner needs at least one STUN server in `emulatorjs.netplay.ice_servers`. Without any, NAT traversal can't begin. 3. **ICE server URLs reachable?** RomM can't talk to `stun.l.google.com:19302` if your server has no outbound internet. Sounds silly but happens in air-gapped labs. Full config: [Configuration File → `emulatorjs.netplay`](../reference/configuration-file.md#emulatorjsnetplay). diff --git a/docs/using/account-and-profile.md b/docs/using/account-and-profile.md index 1f526c04..5689b611 100644 --- a/docs/using/account-and-profile.md +++ b/docs/using/account-and-profile.md @@ -9,7 +9,7 @@ Every user (User or Admin) can manage their own profile, but Admins can edit _ot ## Preferred username in OIDC -If you're an OIDC user and want to show your `preferred_username` from the token instead of your email local-part, the operator can set `OIDC_USERNAME_ATTRIBUTE=preferred_username` (see [OIDC Setup](../administration/oidc/index.md)). +If you're an OIDC user and want to show your `preferred_username` from the token instead of your email local-part, the server owner can set `OIDC_USERNAME_ATTRIBUTE=preferred_username` (see [OIDC Setup](../administration/oidc/index.md)). ## Client API tokens diff --git a/docs/using/emulator-streaming-migration.md b/docs/using/emulator-streaming-migration.md index 0a2fdaca..9bc0dd3f 100644 --- a/docs/using/emulator-streaming-migration.md +++ b/docs/using/emulator-streaming-migration.md @@ -9,7 +9,7 @@ Emulator streaming used to mean one container per emulator, each running its own !!! warning "The per-emulator broker mods are deprecated" - A `config.yml` container without `protocol: webstation` still works today, logs a startup warning telling you to migrate, and will stop being supported in a future release. + A `config.yml` container without `protocol: webstation` still works today, but will stop being supported in a future release. ## Why @@ -63,7 +63,7 @@ The full field reference lives in [Configuration File → `streaming`](../refere 2. Point `library_path` in `config.yml` at that same mount. 3. Move each platform you were streaming into the new container's `platforms:` map (see the before/after above). 4. Confirm streaming works for each platform, then remove the old per-emulator containers. -5. Restart RomM. The startup warning about legacy containers disappears once no `config.yml` container is missing `protocol: webstation`. +5. Restart RomM. Every container is on the new shape once none of them is missing `protocol: webstation`. ## Per-emulator notes diff --git a/docs/using/emulator-streaming.md b/docs/using/emulator-streaming.md index 3e9c7e60..d5f17922 100644 --- a/docs/using/emulator-streaming.md +++ b/docs/using/emulator-streaming.md @@ -5,88 +5,169 @@ description: Launch games into a native emulator running in a container # Emulator Streaming -Emulator streaming launches a game into a **native** emulator running in a separate container and streams the picture, sound, and input back to your browser. Unlike [in-browser play](in-browser-play/emulatorjs.md), the emulation runs server-side on real emulator binaries (PCSX2, Dolphin, xemu), so the heavy lifting happens on the host rather than in the client. The server claims a session, tells the emulator which ROM to launch, and shows the live stream inside a player view, with save-state and volume controls in the toolbar. +Emulator streaming runs your game in a **real** emulator on the server and streams the video, audio and input to your browser. Where [in-browser play](in-browser-play/emulatorjs.md) compiles emulators to WebAssembly and runs them on your machine, this runs actual PCSX2, Dolphin, RPCS3 and RetroArch binaries on the host. Your browser is just a screen. -Each emulator runs in its own [linuxserver](https://docs.linuxserver.io) container with a [Selkies](https://github.com/selkies-project/selkies) WebRTC stream and a small HTTP broker sidecar that RomM talks to. Nothing appears in the UI until you configure at least one container. +It all runs in one [docker-webstation](https://github.com/linuxserver/docker-webstation) container: a [Selkies](https://github.com/selkies-project/selkies) WebRTC desktop with the emulators installed, plus the [romm-broker](https://github.com/romm-streaming/romm-broker) sidecar RomM sends commands to. A single container handles **every** platform you point at it. Until you configure one, none of this shows up in the UI. - -!!! info "Already running per-emulator containers?" - Streaming is moving to a single webstation container that serves every platform. See [Migrating to webstation](emulator-streaming-migration.md). +If a platform can do both, you get separate actions for browser play and streaming, so turning on streaming doesn't take browser play away. -!!! warning "Work in progress" - This is the first release of the streaming framework and ships with three emulators. More integrations (rpcs3 for PS3, and others) are planned as separate follow-ups. +!!! info "Coming from the per-emulator broker mods?" + A container without `protocol: webstation` still works, but will stop being supported in a future release. See [Migrating to webstation](emulator-streaming-migration.md) for the config rewrite. + +## How a session works + +There's one display per container, so a container runs **one session at a time** no matter which platform it's playing. Sessions are stored in [Valkey](../install/redis-or-valkey.md) and claimed atomically, which keeps multiple API workers from stepping on each other. Whoever claims a session owns it, and only they or an admin can control or release it. + +List the same platform on several containers and you get a **pool**. RomM walks them in config order and grabs the first free one, so two people can play SNES at once if you've got two containers. If they're all busy, RomM checks for sessions whose heartbeat has gone quiet (someone closed a tab, a browser crashed) and clears those out before telling you the platform is in use. -## Supported emulators +Containers only pool together if they agree on `emulator`, `memory_card_sync` and `protocol`. Those three decide where saves end up and which controls the player offers, and it would be a bad surprise to land on a pool member and find your saves missing. Containers that differ are treated as separate setups. -Each emulator ships as a companion Docker mod repo with the broker sidecar and a worked `docker-compose.yml`. +## Supported platforms -| Platform slug | Emulator | Save states | Manual slots | Autosave slot | Broker repo | -| ------------- | -------- | ----------- | ------------ | ------------- | ------------------------------------------------------------------------------------- | -| `ps2` | PCSX2 | Yes | 9 | Slot 10 | [pcsx2-romm-integration](https://github.com/LoneAngelFayt/pcsx2-romm-integration) | -| `ngc`, `wii` | Dolphin | Yes | 7 | Slot 8 | [dolphin-romm-integration](https://github.com/LoneAngelFayt/dolphin-romm-integration) | -| `xbox` | xemu | Yes | 9 | Slot 10 | [xemu-romm-integration](https://github.com/LoneAngelFayt/xemu-romm-integration) | +The broker ships standalone emulators for the platforms below, and RetroArch for everything else. -The broker launches ROMs as direct files, so **archive extraction is not supported**. +| Platform slug | Emulator | Save states | Memory card | Disc swap | +| ------------------- | ----------- | --------------------------- | ----------- | --------- | +| `ps2` | PCSX2 | Slots 1-9, autosave slot 10 | Yes | Manual | +| `ngc` | Dolphin | Slots 1-7, autosave slot 8 | Yes | - | +| `wii` | Dolphin | Slots 1-7, autosave slot 8 | - | - | +| `psx` | DuckStation | One resume state | - | - | +| `ps3` | RPCS3 | One resume state | - | - | +| `xbox` | xemu | - | - | - | +| `xbox360` | Xenia | - | - | - | +| `wiiu` | Cemu | - | - | - | +| `switch` | Eden | - | - | - | +| `3ds` | Azahar | - | - | - | +| `ps4` | shadPS4 | - | - | - | +| `psp` | PPSSPP | - | - | - | +| `dc` | Flycast | - | - | Yes | +| _(anything else)_ | RetroArch | One resume state | - | Varies | +| _(adventure games)_ | ScummVM | One resume state | - | - | -Only **one session per platform** can be active at a time, since there is a single emulator container behind it. Sessions are stored in [Valkey](../install/redis-or-valkey.md) with an atomic claim, so multiple workers stay consistent. The session is bound to the user who claimed it, and only that owner or an admin can control or release it, though an admin can force-release a stuck session. +RetroArch covers dozens of platforms from the one container. The broker picks the core, not RomM, and RomM just labels the action with whatever core that is (`RA Snes9x`, `RA mGBA`). If you want to change the mapping it's the broker's `retroarch_platforms.json`. + +ROMs are launched as plain files, so **archives won't work**. Extract them first. ## Saves and save states -**Save states** are the emulator's own quick-save slots: numbered manual slots plus a dedicated autosave slot per platform (see the table above). The autosave slot is reserved for **Save & Exit** and is overwritten on the next exit. +Three separate things move between the container and your library. Which ones apply depends on the platform. + +### Save states + +These are the emulator's own snapshots, and the table above says which of three shapes each platform gets: + +- **Numbered slots with an autosave.** The autosave slot belongs to save-and-exit and is overwritten every time you exit, so don't keep anything there. +- **A single resume state.** These emulators only write a state as they shut down, so there's no grid to pick from, just the one state that saving and resuming both use. +- **No states at all.** You rely on the game's own save data instead. -Each state is copied off the container into your library as it is written, with a thumbnail alongside it, and the state written by **Save & Exit** is filed when the session ends. Your stored states are offered the next time you launch that game, so you can carry on from one instead of booting fresh. RomM keeps the newest states per game, emulator, and user, and prunes the rest past `STREAMING_STATE_HISTORY_LIMIT` (default `50`, `0` to keep everything). +Whenever a state is written, RomM copies it off the container, along with a thumbnail grabbed from the video. The state from save-and-exit is collected when the session closes. Claim a container later and RomM pushes your stored states back onto it, which is why they follow you between containers and survive a container being rebuilt. -These streaming states are stored separately from RomM's [per-user saves and states](saves-and-states.md) from in-browser play. +RomM keeps the most recent `STREAMING_STATE_HISTORY_LIMIT` states per game, per emulator, per user (default `50`, or `0` to keep everything) and prunes the rest. -## Memory cards +These are kept apart from the [saves and states](saves-and-states.md) that in-browser play produces, because the formats aren't interchangeable. -On platforms with a memory card (**PS2** and **GameCube**), you can opt a container into whole-card sync with `memory_card_sync: true`. The card then lives in your RomM library rather than on the container: RomM loads your card in when a session starts, copies it back when you exit, and leaves the container's slot blank in between. Your most recently used card for that emulator loads by default, and you can keep several cards and pick which one a session mounts. +### In-game saves -PCSX2 only serves its card when Slot 1 holds a **Folder** card rather than a **File** card. With a File card the broker refuses the transfer and the session will not start, so set the card type before turning the flag on (see the [PCSX2 broker's memory card setup](https://github.com/LoneAngelFayt/pcsx2-romm-integration#memory-card-setup)). Dolphin's broker pins its own folder card, so GameCube needs no extra setup. +This is the save data the game itself writes: NAND, battery saves, the emulated user profile. When a session ends, RomM zips the lot off the container and stores it as a single save, so it all stays together. On platforms with no save states, this is the only thing keeping your progress. -Because each session starts from your library, the first time RomM uses a container that already has a card on it, it stops and asks what to do: +### Memory cards -- Importing it stores the container's card in your library, as a new version of your current card for that emulator, or as your first card if you have none. -- Starting fresh erases the container's card, which is confirmed first because it cannot be undone. +On **PS2** and **GameCube** you can set `memory_card_sync: true` and have RomM manage the whole card instead of individual save files. The card lives in your library: RomM loads it in when a session starts, copies it back when you exit, and leaves the container's slot empty in between. This **replaces** the in-game save handling above for that container. -RomM records your answer per container, so it only asks once. +Cards are their own little library. Keep as many as you like, name and rename them, snapshot the current state as a version and roll back to it later, share one with another user, or download it. Whichever card you used last loads by default. + +Set the flag on a platform with no memory card (Wii, xemu) and RomM logs a warning and ignores it, syncing individual save files instead. Obeying it would mean shuffling an empty card back and forth while the saves those platforms actually use stopped syncing. + +PCSX2 needs Slot 1 set to a **Folder** card, not a **File** card. Given a File card, the broker refuses to hand it over and the session won't start at all. Change the card type before you turn the flag on (see the [PCSX2 memory card setup](https://github.com/LoneAngelFayt/pcsx2-romm-integration#memory-card-setup)). Dolphin pins its own folder card, and GameCube works out of the box. + +Every session starts from your library's copy, so the first time RomM meets a container that already has a card sitting on it, it stops and asks: + +- **Import it**, and the container's card goes into your library, either as a new version of your current card or as your first one. +- **Start fresh**, and the container's card is wiped. You get a confirmation first, since there's no undo. + +RomM remembers your answer per container and won't ask again. !!! warning "Playing on the container directly" - Syncing only happens around a RomM streaming session. If you play on the emulator container directly, outside RomM, those saves and cards stay on the container and are not pulled into your library. A later RomM session loads your library's copy over them, so make your progress through RomM to keep it. + Syncing only happens at the start and end of a RomM session. Play on the container directly, outside RomM, and those saves and cards never reach your library. Worse, the next RomM session will write your library's copy over them. Go through RomM if you want to keep the progress. + +## Multi-disc games + +Multi-disc games boot their full playlist. On `dc`, `saturn`, `segacd`, `turbografx-cd` and `dos` you can change discs without restarting the emulator, choosing from that game's own discs. + +RomM records which disc was mounted when you saved a state and remounts the same one when you load it, so a save on disc 3 comes back on disc 3. + +PS2 has no swap control, but you can change discs from inside PCSX2 itself. + +## Joining a session + +You can open a session up to a second player. Anyone can see which sessions are joinable and ask to join. Joining hooks into the running session rather than claiming a container of its own, so it doesn't eat a second one. + +## Administration + +Admins get two extra things: + +**Desktop sessions** give you the container's desktop with no game running. This is how you set an emulator up from the inside: BIOS paths, controllers, whatever per-emulator settings it needs. A desktop takes the same lock a game does, so it blocks players out and a running game blocks you out. + +**The container fleet** shows every container you've configured, what it's running and who claimed it, and lets you force-release any session. Rows are per container, not per platform, since a container serving five platforms still only holds one session. Anything RomM can't claim (a `host` missing its scheme, a broker it can't reach) shows as unconfigured rather than idle, so a typo is visible instead of silent. + +Force-release is also your way out when a platform is stuck as in-use because someone closed their browser without releasing it. ## Setup -### Run the emulator containers +### Run the webstation container + +Start [docker-webstation](https://github.com/linuxserver/docker-webstation) with [romm-broker](https://github.com/romm-streaming/romm-broker), mounting your ROM library read-only. Two things on it matter to RomM: + +- The **Selkies web UI**, which is what your browser loads the stream from. +- The **broker API**, which is where RomM sends launch, save, state and disc commands. It's served under the container's `SUBFOLDER` on the same origin. -Pick the broker repos for the platforms you want and follow each repo's `docker-compose.yml`. Each container exposes two things we need: +There's a working compose file at [`docker-compose.streaming.yml`](https://github.com/rommapp/romm/blob/master/docker-compose.streaming.yml) upstream. Note the image is amd64 only. -- The **Selkies web UI** the browser loads the stream from (an HTTPS port). -- The **broker API** RomM sends launch, save, and volume commands to (default port `8000`). +### Set up `config.yml` -### Setup `config.yml` +Add a `streaming` block with **one entry per container** (full schema in [Configuration File → `streaming`](../reference/configuration-file.md#streaming)). Whatever you set at the container level applies to every platform it serves, and a platform block only needs to name the things that differ. -Add a `streaming` block with one entry per emulator container, full schema in [Configuration File → `streaming`](../reference/configuration-file.md#streaming). +```yaml +streaming: + enabled: true + containers: + - protocol: webstation + host: https://192.168.1.56:3010 + subfolder: /streaming + label: Emulation station + platforms: + snes: retroarch # just the emulator name... + ps2: # ...or a block overriding container keys + emulator: pcsx2 + memory_card_sync: true +``` -- `host` must be reachable from clients and served over **HTTPS** (Selkies WebRTC requires a secure context). Use the container's built-in self-signed cert or a [reverse proxy with TLS](../install/reverse-proxy.md). -- `broker_host` is called server-side, so HTTP is fine. If the containers share a Docker network, use the container name (e.g. `http://pcsx2:8000`). If `broker_host` is omitted, it gets derived from `host`. -- `label` is the text shown on the play action. -- `memory_card_sync: true` opts a **PS2** or **GameCube** container into whole-card sync (see [Memory cards](#memory-cards)). It has no effect on platforms without a memory card. -- `library_path` overrides the in-container library path if the container mounts the RomM library somewhere other than the default `/romm/library`. -- `emulator` sets an explicit name used to group this container's states and memory cards, lowercased. It defaults to `label`, then the platform slug, so setting it keeps stored states and cards attached when you rename a label later. +Four of those keys have consequences worth knowing before you pick their values: -Multiple platforms can share one container (point `ngc` and `wii` at the same Dolphin instance) or each use their own. +- `host` is what the browser connects to, and it has to be **HTTPS**: Selkies WebRTC won't run without a secure context. Use the container's self-signed cert, or put it behind a [reverse proxy with TLS](../install/reverse-proxy.md). A path like `/streaming` works if you've proxied the container onto RomM's own origin, but then you must set `broker_host` yourself, because a bare path gives RomM no address to call. +- `broker_host` is only ever called server to server, so plain HTTP is fine. Pooled containers are identified by it, so two serving the same platform need different ones. +- `library_path` is where the container sees your library. Don't change it casually, since your state and save history is keyed to it. +- The per-platform `emulator` names what that platform's states and memory cards are filed under, so renaming it later orphans everything stored under the old name. A container-level `emulator` is ignored whenever `platforms` is used. ### Set the shared secret -`STREAMING_BROKER_SECRET` authenticates calls to the broker. Set the **same value** in every container. If a broker needs a different secret, set `broker_secret` on that entry in `config.yml` and leave `STREAMING_BROKER_SECRET` unset, because the env var wins over the per-container value whenever it carries one. +`STREAMING_BROKER_SECRET` is what authenticates RomM to the broker, and it has to match the container's `BROKER_SECRET`. Use the **same value** everywhere. + +If one broker needs its own secret, put `broker_secret` on that entry in `config.yml` and leave `STREAMING_BROKER_SECRET` unset entirely. The env var beats the per-container value whenever it's set, so you can't mix the two. + +### Tune the timeouts + +Three env vars bound how long streaming waits, listed with their defaults in [Environment Variables → Emulator Streaming](../reference/environment-variables.md#emulator-streaming). -If a broker's save wait exceeds the default 45 seconds, raise `STREAMING_SAVE_TIMEOUT` (seconds) so Save & Exit doesn't time out. `STREAMING_STATE_HISTORY_LIMIT` (default `50`) caps how many save states RomM keeps per game, emulator, and user before pruning the oldest, and `0` keeps every state. All are set as env vars (see [Environment Variables](../reference/environment-variables.md)). +`STREAMING_LAUNCH_TIMEOUT` covers the whole launch, including any unpacking before the emulator starts. `STREAMING_SAVE_TIMEOUT` covers save-and-exit, and needs raising if a broker's own `SAVE_WAIT` is higher. `STREAMING_STATE_HISTORY_LIMIT` caps how many states are kept per ROM, emulator and user. ## Troubleshooting -- **No Play on `