Skip to content

feat(contexts): data model for grid pages - #3131

Merged
blizzz merged 6 commits into
mainfrom
feat/noid/grid-page-model
Nov 20, 2026
Merged

blizzz merged 6 commits into
mainfrom
feat/noid/grid-page-model

Conversation

@rubenvdlinde

@rubenvdlinde rubenvdlinde commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Part of #3099. This adds the data model from #3100, and nothing uses it yet. It's the first of seven pull requests that build the backend for grid pages and application menus. They're stacked, each on the one before:

  1. feat(contexts): data model for grid pages #3131 (this one): the data model.
  2. feat(contexts): keep widgets in step with pages and applications #3132: keep widgets in step with their pages and applications.
  3. feat(api): pages and widgets on api/2 #3133: pages and widgets on api/2.
  4. feat(contexts): give applications a menu with its own API #3135: give applications a menu.
  5. feat(contexts): save the menu of an application #3136: save the menu of an application.
  6. feat(contexts): nest menu items one level deep #3137: nest menu items one level deep.
  7. feat(contexts): carry the menu through export and import #3138: carry the menu through export and import.

Splitting it this way lets you review the schema on its own, before any behaviour depends on it.

Size: 1380 lines: 681 code and 691 tests, plus the version bump.

What changes

  • Pages (tables_contexts_page) get title, icon and description. A new page type grid sits next to startpage.
  • Every page gets a slug, unique within its application, that doesn't change once it's set. The column, the unique index, the entity and the export and import key all use slug. Existing start pages get dashboard.
  • Widgets (tables_contexts_widget): the page, the type, an optional title, icon and description, the position and size on the 12-column grid, the settings in one JSON column, and for a data widget its table or view as a nullable, indexed node_rel_id, the way menu items keep theirs. Settings are read back ordered by name, because MySQL's JSON column type does not keep the order they were written in, so every database answers the same.
  • An entity and mapper for widgets, with tests against the database. When a table or view leaves the application, its widgets are unlinked and stay.
  • Imported pages are checked: an import takes startpage and grid pages only, and at most one start page per application. Other pages are skipped with a warning in the log, like other invalid import data. The import of an application runs in one transaction and keeps a page's title, icon and description.

One migration, Version2500Date20261109000000, adds the columns and tables, including the unique index on the page's application and slug, and then fills in a slug for every existing page. Tables runs on PostgreSQL, MySQL and SQLite, which all treat NULL values in a unique index as distinct (not empty strings, which the code never writes), so the index can be created before the names are filled in. It also serves lookups by application, so there's no separate index on context_id. The column itself allows NULL: a new page gets its page_<id> right after it's inserted, the same way views get theirs.

Where it differs from #3100

We agreed these with @enjeck:

  • Settings are one JSON column on the widget instead of a table with one row per setting. A setting is never queried on its own, and the server checks the whole set against the widget type's schema whenever it saves.
  • Pages have a slug. Views and columns have a technical name for the same reasons: a re-imported scheme finds its pages instead of adding them again, and a menu item or an external tool can address a page by name.

And two small ones on top:

  • Pages have an icon instead of an emoji. It's a named icon from the same set applications use, so applications, pages, menu items and widgets share one icon picker.
  • Widgets get an optional icon and description, next to the title.

How the slugs work

They work the same way as technical names for views and columns:

  • Without a name, a page gets page_<id>, like view_<id> and column_<id>. The start page of an application is dashboard.
  • A unique index on the application and the name is the guarantee. In feat(api): pages and widgets on api/2 #3133 the API turns a name that's already used into a 400.
  • page_<number> is reserved for the automatic names, the way columns keep reserved names, so a name someone types can never collide with a later automatic one. The API refuses it from feat(api): pages and widgets on api/2 #3133 on; imports treat it as no name.
  • One shared check for the format (^[a-z][a-z0-9_]*$, at most 200 characters, no trailing newline), now also used by views and columns, so the rule lives in one place.

Why nothing breaks

The new tables start empty, and the new page columns are empty, except the slug. Existing start pages get dashboard, and new applications get the same when their start page is created. Nothing reads the rest yet, so existing applications, start pages and API responses stay as they are.

The app version goes to 2.5.0-dev.0 (in info.xml, package.json and package-lock.json, the way main went to 2.4.0-dev.0 after 2.3.2), so instances on the released 2.4.0 run the migration. The migrations of this stack are named Version2500Date… for the same reason.

How it was tested

On Nextcloud 34 with PostgreSQL:

  • WidgetTest, plus new database tests WidgetMapperTest and PageTest
  • the slugs: the default page_<id>, filling in existing pages, refusing a name that's already used, a trailing newline
  • importing pages: kept and default names, an unknown page type and a second start page
  • the migration: a second run changes nothing
  • the existing view and column tests, which now use the shared technical name check
  • the full unit suite on PostgreSQL and SQLite: 454 tests
  • the upgrade path: an instance on the released 2.4.0, with three applications, upgraded with occ upgrade to this stack; both migrations ran, the start pages became dashboard and another page page_<id>
  • composer cs:check, psalm and rector, all clean

🖼️ Screenshots

Backend only, no UI changes.

🏁 Checklist

  • ⛑️ Tests (unit and/or integration) are included or not needed
  • 🔙 Backport requests are created or not needed: /backport to stableX.X
  • 📅 Milestone is set
  • 🌸 PR title is meaningful (if it should be in the changelog: is it meaningful to users?)

🤖 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@rubenvdlinde
rubenvdlinde added this pull request to stack #3134 October 22, 2026 11:01
@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch from 9467b93 to 2f8eafc Compare October 22, 2026 11:37
@rubenvdlinde
rubenvdlinde removed this pull request from stack #3134 October 22, 2026 11:42
@rubenvdlinde
rubenvdlinde added this pull request to stack #3139 October 22, 2026 11:42
@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch 4 times, most recently from 4489199 to 5d26395 Compare October 26, 2026 09:24
@rubenvdlinde
rubenvdlinde marked this pull request as ready for review October 26, 2026 09:25
$table->addColumn('grid_y', Types::INTEGER, ['notnull' => true, 'default' => 0]);
$table->addColumn('grid_width', Types::SMALLINT, ['notnull' => true, 'default' => 1]);
$table->addColumn('grid_height', Types::SMALLINT, ['notnull' => true, 'default' => 1]);
$table->addColumn('settings', Types::TEXT, ['notnull' => false]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i thought we said we're using JSON?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did, sorry. The column is Types::JSON now. Nextcloud maps it per database: json on PostgreSQL, JSON on MySQL, LONGTEXT on MariaDB and text on SQLite. I ran the tests on PostgreSQL and SQLite; nothing compares the stored text, so MySQL normalising the JSON doesn't matter. The Widget entity maps the field as Types::JSON too and reads and writes it as an array. That part is in #3133, where the settings type moves into ResponseDefinitions anyway.

Comment thread lib/Service/ContextService.php Resolved
Comment thread lib/Db/Page.php Resolved
Comment thread tests/unit/Db/PageTest.php Resolved
@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch 3 times, most recently from d20afd0 to ada0004 Compare November 3, 2026 14:51
@rubenvdlinde
rubenvdlinde removed this pull request from stack #3139 November 4, 2026 10:41
rubenvdlinde added a commit that referenced this pull request Nov 13, 2026
- a table or view added to an application later gets a widget at the bottom of the
  start page, so the start page follows the Edit application dialog and Import scheme
- a new, migrated or imported start page is filled with one call, its header passing
  the header schema with the description cut to 2000 characters; a start page that
  cannot show every table or view logs how many it shows
- an imported application that gets a new start page has it filled as well
- the migration step sorts after the steps of #2476 and #3131, copies its layout
  instead of using GridWidgetTypes, reads all start pages with two queries, prepares
  its inserts once, commits per hundred pages and reports progress
- the migration test runs in a transaction it rolls back, instead of emptying every
  Tables table of the instance
- the docs say page where the start page now counts as well

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch from c142eed to b2f3392 Compare November 16, 2026 09:22
rubenvdlinde added a commit that referenced this pull request Nov 16, 2026
- a table or view added to an application later gets a widget at the bottom of the
  start page, so the start page follows the Edit application dialog and Import scheme
- a new, migrated or imported start page is filled with one call, its header passing
  the header schema with the description cut to 2000 characters; a start page that
  cannot show every table or view logs how many it shows
- an imported application that gets a new start page has it filled as well
- the migration step sorts after the steps of #2476 and #3131, copies its layout
  instead of using GridWidgetTypes, reads all start pages with two queries, prepares
  its inserts once, commits per hundred pages and reports progress
- the migration test runs in a transaction it rolls back, instead of emptying every
  Tables table of the instance
- the docs say page where the start page now counts as well

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch from b2f3392 to 9a723b0 Compare November 17, 2026 09:11
rubenvdlinde added a commit that referenced this pull request Nov 17, 2026
- a table or view added to an application later gets a widget at the bottom of the
  start page, so the start page follows the Edit application dialog and Import scheme
- a new, migrated or imported start page is filled with one call, its header passing
  the header schema with the description cut to 2000 characters; a start page that
  cannot show every table or view logs how many it shows
- an imported application that gets a new start page has it filled as well
- the migration step sorts after the steps of #2476 and #3131, copies its layout
  instead of using GridWidgetTypes, reads all start pages with two queries, prepares
  its inserts once, commits per hundred pages and reports progress
- the migration test runs in a transaction it rolls back, instead of emptying every
  Tables table of the instance
- the docs say page where the start page now counts as well

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>

@enjeck enjeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me except for the unresolved comments. We can merge if those are addressed now or plan to be addressed later

@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch from 9a723b0 to 94493d0 Compare November 19, 2026 08:27
rubenvdlinde added a commit that referenced this pull request Nov 19, 2026
- a table or view added to an application later gets a widget at the bottom of the
  start page, so the start page follows the Edit application dialog and Import scheme
- a new, migrated or imported start page is filled with one call, its header passing
  the header schema with the description cut to 2000 characters; a start page that
  cannot show every table or view logs how many it shows
- an imported application that gets a new start page has it filled as well
- the migration step sorts after the steps of #2476 and #3131, copies its layout
  instead of using GridWidgetTypes, reads all start pages with two queries, prepares
  its inserts once, commits per hundred pages and reports progress
- the migration test runs in a transaction it rolls back, instead of emptying every
  Tables table of the instance
- the docs say page where the start page now counts as well

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch from 94493d0 to 544f2e0 Compare November 19, 2026 13:35
rubenvdlinde added a commit that referenced this pull request Nov 19, 2026
- a table or view added to an application later gets a widget at the bottom of the
  start page, so the start page follows the Edit application dialog and Import scheme
- a new, migrated or imported start page is filled with one call, its header passing
  the header schema with the description cut to 2000 characters; a start page that
  cannot show every table or view logs how many it shows
- an imported application that gets a new start page has it filled as well
- the migration step sorts after the steps of #2476 and #3131, copies its layout
  instead of using GridWidgetTypes, reads all start pages with two queries, prepares
  its inserts once, commits per hundred pages and reports progress
- the migration test runs in a transaction it rolls back, instead of emptying every
  Tables table of the instance
- the docs say page where the start page now counts as well

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
Pages get a title, a named icon, a description and a technical name
that is unique within their application. Without a name a page is
called page_<id>, the start page dashboard, and page_<number> is kept
for those automatic names.

Widgets get their own table: the page, the type, an optional title,
icon and description, the position and size on the grid, and the
settings as one JSON object. A widget source links a data widget to a
table or view of the same application.

One migration adds all of it and names the pages that already exist.
The technical name format is checked by one validator, now also used
for views and columns. The app version moves to 2.5.0-dev.0 so that
instances on 2.4.0 run the migration.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
…e atomically

Types::JSON is deprecated for columns since Nextcloud 33. An import reads pages in the
snake case the mapper exports, so the camel case fallbacks go.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
- chunk the IN lists of the widget mappers for Oracle
- freeze the migration: literals instead of class constants, and a date after the
  steps of #2476 so it sorts after them
- an imported grid page without a title is titled by its technical name, and an
  imported application without a start page gets one with all its tables and views
- the reserved automatic name lives in TechnicalNameValidator
- the stricter technical name rule refuses a trailing newline for views and columns
  too; importView() drops one from names stored by older versions
- widgets have one serialised shape, so Widget is no longer JsonSerializable
- naming a page after its id only updates its technical name
- the icon check calls IconHelper directly, the view name test goes through create(),
  the database error stub moves to the pull request that uses it, and the source
  mapper test uses ids no real row has

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
…dget

A data widget now stores its node relation in a nullable, indexed node_rel_id
column, as menu items do, instead of in a separate tables_contexts_widget_source
table. WidgetSource and its mapper go; WidgetMapper::clearNodeRelIds() unlinks
the widgets of removed relations.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
…le, icon and description of its pages

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
A page is named by its slug, as agreed in #3094: the column, the unique
index, the entity field, the export and import key. A new page gets
page_<id> and the start page dashboard. Columns and views keep their
technical name.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
@rubenvdlinde
rubenvdlinde force-pushed the feat/noid/grid-page-model branch from 544f2e0 to e52a7b7 Compare November 20, 2026 10:52
rubenvdlinde added a commit that referenced this pull request Nov 20, 2026
- a table or view added to an application later gets a widget at the bottom of the
  start page, so the start page follows the Edit application dialog and Import scheme
- a new, migrated or imported start page is filled with one call, its header passing
  the header schema with the description cut to 2000 characters; a start page that
  cannot show every table or view logs how many it shows
- an imported application that gets a new start page has it filled as well
- the migration step sorts after the steps of #2476 and #3131, copies its layout
  instead of using GridWidgetTypes, reads all start pages with two queries, prepares
  its inserts once, commits per hundred pages and reports progress
- the migration test runs in a transaction it rolls back, instead of emptying every
  Tables table of the instance
- the docs say page where the start page now counts as well

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
@blizzz blizzz added 4. to release Ready to be released and/or waiting for tests to finish and removed feedback-requested labels Nov 20, 2026
@blizzz
blizzz merged commit ec81152 into main Nov 20, 2026
81 checks passed
@blizzz
blizzz deleted the feat/noid/grid-page-model branch November 20, 2026 13:19

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this should be renamed in a follow-up.

rubenvdlinde added a commit that referenced this pull request Nov 21, 2026
- a table or view added to an application later gets a widget at the bottom of the
  start page, so the start page follows the Edit application dialog and Import scheme
- a new, migrated or imported start page is filled with one call, its header passing
  the header schema with the description cut to 2000 characters; a start page that
  cannot show every table or view logs how many it shows
- an imported application that gets a new start page has it filled as well
- the migration step sorts after the steps of #2476 and #3131, copies its layout
  instead of using GridWidgetTypes, reads all start pages with two queries, prepares
  its inserts once, commits per hundred pages and reports progress
- the migration test runs in a transaction it rolls back, instead of emptying every
  Tables table of the instance
- the docs say page where the start page now counts as well

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ruben van der Linde <ruben@conduction.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4. to release Ready to be released and/or waiting for tests to finish AI assisted

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants