Repository navigation
Conversation
blizzz
left a comment
There was a problem hiding this comment.
Apart of the comments, we get some more overhead on PUT and DELETE requests:
(RowOCSController.php:190,241). It checks the read permission, loads the view and runs a filtered scope query. Then updateRowData/delete do all of that again: getRowById, the permission checks, and isRowInViewPresent (RowService.php:707,831).
(by code, without benchmarking)
I would prefer keeping the UUID out of the core logic as much as possible. Handle them on the Controller parts, where necessary, and continue working with the ID. The main way to interact with Tables should still be the id.
All in all there are too many other things creeping into this PR, and the changes are more complicated than necessary.
| use OCP\Migration\IOutput; | ||
| use OCP\Migration\SimpleMigrationStep; | ||
|
|
||
| class Version2500Date20261103000000 extends SimpleMigrationStep { |
There was a problem hiding this comment.
When adding a new migration, also bump the version in https://github.com/nextcloud/tables/blob/main/appinfo/info.xml#L29, now to 2.4.1-dev.0. Otherwise this will not be picked up on anything that uses existing setups.
There was a problem hiding this comment.
Done, with one adjustment: main moved to 2.5.0-dev.0 when #3131 was merged today, so the PR now bumps to 2.5.0-dev.1 (in package.json too), and the migration is named Version2500Date20261120000000 to sort after main's newest one.
| #[NoAdminRequired] | ||
| #[RequirePermission(permission: Application::PERMISSION_READ, typeParam: 'nodeCollection')] | ||
| #[ApiRoute(verb: 'GET', url: '/api/2/{nodeCollection}/{nodeId}/rows/{rowId}', requirements: ['nodeCollection' => '(tables|views)', 'nodeId' => '(\d+)'])] | ||
| public function getRow(string $nodeCollection, int $nodeId, string $rowId): DataResponse { |
There was a problem hiding this comment.
not sure this addition belongs to this PR, what is the motivation?
There was a problem hiding this comment.
#3093 asked for it: the route section says "this needs a GET for both uuid and integer id, as this route does not exist yet", and one of the acceptance criteria covers GET, PUT and DELETE on that path. It was also where most of the extra logic came from: API v2 has no single-row read yet, so reading through a view needed its own permission, filter and column handling, including the zero-column changes you asked about below. Given your preference to keep UUIDs out of the core logic, I've removed it together with the UUID addressing, so this PR doesn't cover that acceptance criterion.
| #[RequirePermission(permission: Application::PERMISSION_UPDATE, typeParam: 'nodeCollection')] | ||
| #[ApiRoute(verb: 'PUT', url: '/api/2/{nodeCollection}/{nodeId}/rows/{rowId}', requirements: ['nodeCollection' => '(tables|views)', 'nodeId' => '(\d+)'])] | ||
| public function updateRow(string $nodeCollection, int $nodeId, int $rowId, mixed $data): DataResponse { | ||
| public function updateRow(string $nodeCollection, int $nodeId, string $rowId, mixed $data): DataResponse { |
There was a problem hiding this comment.
did you consider creating a new endpoint to retrieve a row by UUID, and what let you to this decision?
There was a problem hiding this comment.
I followed the design in #3093, which puts the UUID into the existing {rowId} routes and notes that rowId switches from int to string in their specification. The issue considered a separate /api/2/rows/{uuid} and rejected it, because API v2 already puts the table or view in the route and a UUID alone would leave the permission context ambiguous. I didn't evaluate a separate route under the node path. I've now taken UUID addressing out of this PR, so PUT and DELETE are unchanged from main.
| #[RequirePermission(permission: Application::PERMISSION_DELETE, typeParam: 'nodeCollection')] | ||
| #[ApiRoute(verb: 'DELETE', url: '/api/2/{nodeCollection}/{nodeId}/rows/{rowId}', requirements: ['nodeCollection' => '(tables|views)', 'nodeId' => '(\d+)'])] | ||
| public function deleteRow(string $nodeCollection, int $nodeId, int $rowId): DataResponse { | ||
| public function deleteRow(string $nodeCollection, int $nodeId, string $rowId): DataResponse { |
| */ | ||
| private function getRows(array $rowIds, array $columnIds): array { | ||
| if (empty($rowIds) || empty($columnIds)) { | ||
| if (empty($rowIds)) { |
There was a problem hiding this comment.
It let the new single-row GET return a row of a table or view without columns (just the sleeve). Since Row2Mapper::find() goes through getRows(), it affected the other single-row callers too, and the findAll() guard kept listings as they were. Both are gone now.
| $cellMapperCache = []; | ||
|
|
||
| while ($rowData = $result->fetchAssociative()) { | ||
| while ($result !== null && ($rowData = $result->fetchAssociative())) { |
There was a problem hiding this comment.
Same reason, removed as well.
| reset($this->data); | ||
| } | ||
|
|
||
| public static function createFromInputArray(array $cells): self { |
There was a problem hiding this comment.
This addition also does not really belong into this PR, does it?
There was a problem hiding this comment.
Right, it isn't about UUIDs. It checked each cell of the PUT body at the controller boundary (required keys, integer column IDs), so malformed cells got a 400. Removed.
| try { | ||
| $this->rowService->updateSet($rowId, $this->viewId, $data, $this->userId, $this->tableId); | ||
| $this->countUpdatedRows++; | ||
| } catch (NotFoundError $e) { |
There was a problem hiding this comment.
this is a behavorial change and also outside of the scope
There was a problem hiding this comment.
It followed from the error changes in RowService: upsertRow() counts an InternalError as one failed line but rethrows a NotFoundError, which ends the import. With those changes, a spreadsheet ID that matches no row would have stopped the whole import. Removed together with them.
| $e = new \Exception('Update row is not allowed.'); | ||
| $this->logger->error($e->getMessage(), ['exception' => $e]); | ||
| throw new InternalError(static::class . ' - ' . __FUNCTION__ . ': ' . $e->getMessage()); | ||
| throw new NotFoundError('Row not found in the requested view.'); |
There was a problem hiding this comment.
Here and later one, the changes errors don't seem fitting to the PR either and are a behavioral change (not judging them).
There was a problem hiding this comment.
Agreed, they're out, along with the other incidental changes in RowService. Since PUT and DELETE resolved the row before calling the service, I had switched the later checks to NotFoundError so they gave 404 as well. #3178 covers that change on its own.
Add a nullable, uniquely indexed uuid column to tables_row_sleeves. Every sleeve insertion, including legacy transfers with explicit IDs, assigns a UUIDv7 before writing. An assigned identity is normalized to lowercase and cannot change or be cleared. Stored values load unchanged, so rows created before this release keep NULL until they are backfilled. User migration archives now carry each row's UUID. Import keeps an archived UUID unless another row already has it, in which case the imported row gets a fresh one. Archives without a UUID, or with one that is not a valid UUIDv7, get a fresh one as well, as table imports treat invalid table UUIDs. The UUID is looked up before inserting, as TableService::create() does for table UUIDs, because the import runs in one transaction that a unique constraint violation would cancel. Refs: nextcloud#3093 Assisted-by: Codex:gpt-6 Assisted-by: Codex:gpt-6-astra Assisted-by: ClaudeCode:claude-opus-5-5 Assisted-by: ClaudeCode:claude-fable-5-2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
Register a periodic job from the migration that assigns UUIDv7 values to sleeves still holding NULL. A run processes batches of up to 500 rows for about five seconds and commits each batch. The update only touches rows whose uuid is still NULL, so concurrent assignments and deletions are respected and audit metadata stays unchanged. Parallel runs are disabled. Progress is a lazy app-config cursor written once per run. A lagging cursor only repeats a cheap NULL scan, while writing it after every batch would invalidate the shared app-config cache. When a pass finds nothing left, the job rescans from the start for rows below the cursor before it removes itself. On SQLite the job pauses briefly between batches so requests can take the single writer. Refs: nextcloud#3093 Assisted-by: Codex:gpt-6 Assisted-by: Codex:gpt-6-astra Assisted-by: ClaudeCode:claude-opus-5-5 Assisted-by: ClaudeCode:claude-fable-5-2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
Row responses now include the persisted uuid next to the integer id, including public-share rows and the row returned by POST. Rows created before the backfill reaches them report null. Row responses forwarded from a federated server keep the remote uuid, or report null when the remote server does not send one. Regenerate openapi.json and the TypeScript types for the new field. Refs: nextcloud#3093 Assisted-by: Codex:gpt-6 Assisted-by: Codex:gpt-6-astra Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
Existing installations run the row UUID migration only when the declared app version is higher than the installed one. Bump appinfo/info.xml along with package.json and package-lock.json, which this repository keeps in sync. Refs: nextcloud#3093 Assisted-by: ClaudeCode:claude-opus-5-5 Assisted-by: ClaudeCode:claude-fable-5-2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jiajun Xu <foolfitz@gmail.com>
266c10c to
8ab689f
Compare
|
Thanks for the review. I've cut the PR down to the row identity itself: the UUID column with its unique index, assignment on insert, the background backfill, keeping UUIDs in user migration archives, and Addressing rows by UUID is no longer part of it. The API v2 row routes are back to main's numeric IDs (only federated responses still fill in To get there I dropped the last two commits and rebased the rest onto current main, which also resolves the conflicts. The PR is now the first three commits, with the archive import change folded into the first, plus the version bump. I tested the upgrade from 2.4.0, the backfill, user migration imports and the unit suite on MySQL, MariaDB, PostgreSQL and SQLite. |
|
Hello there, We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process. Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6 Thank you for contributing to Nextcloud and we hope to hear from you soon! (If you believe you should not receive this message, you can add yourself to the blocklist.) |
Summary
Related issue: #3093
Add a persistent UUIDv7 to each row so that it can still be identified after export and import through user migration, even if its integer ID changes. The existing integer primary keys and cell associations remain unchanged.
Main changes
tables_row_sleevesand add a unique index covering all rows across the instance. Once assigned, a UUID cannot be replaced or cleared.null, and rows remain accessible by numeric ID./api/2/{tables|views}/{nodeId}/rows/{rowId}in API v2, and allow GET, PUT, and DELETE to accept either an integer ID or a UUID. UUID lookups retain the existing permissions, view filters, and column restrictions. Malformed identifiers return 400; rows not found within the authorized scope return 404. Adduuidto row responses and update the OpenAPI and TypeScript definitions together.The UUID API provided by this PR can read, update, or delete rows on the instance receiving the request. For example, after importing data from Site A into Site B, UUID requests sent to Site B operate only on Site B's data and are not forwarded to Site A.
This defines the scope of this PR. Work left for later includes federated UUID forwarding, UUID addressing on public share token routes, cross-site synchronization, and full
NOT NULLenforcement. UUID preservation applies to user migration archives; CSV import does not yet support migrating rows while retaining their source UUIDs.Regarding the database migration: before merging, confirm with the maintainers which Tables release will include this change and align the migration's version identifier accordingly. This ensures that upgrading an existing installation automatically adds the UUID column and index and schedules background backfill for existing rows.
Testing
Manual testing: preserving UUIDs after archive migration
Using version
0e70ed8c, data with two text columns, Item and Status, was migrated through a User migration archive between two independent Nextcloud 33.0.8 / SQLite environments, with Site A as the source and Site B as the destination.1. Create data on Site A and record the original ID and UUID
A row containing
Transfer notebook/keepwas created, and its identifiers were queried. The request returned HTTP 200, integer ID 4, and UUID01a1ab40-6328-75f1-8976-3cd3e2a98dd8.2. Export a user migration archive from Site A
The
user:export --types=tablescommand exported the Tables data, and the terminal reported a successful export. A separate inspection of the archive confirmed that it preserved the row's UUID and original content.3. Import the same archive into Site B
The
user:importcommand imported the archive into a dedicated empty account on Site B. The terminal reported that the Tables import completed, with the data written to Site B's own database.4. Query the source row on Site A using the original UUID
A GET request using the original UUID was sent from Site A's browser session. It returned HTTP 200, with integer ID 4 and the same UUID as before export.
5. Query the imported row on Site B using the same UUID
A GET request using the same UUID was sent from a separate browser session on Site B. It returned HTTP 200. The integer ID had been reassigned to 108, while the UUID still matched Site A's and the content remained
Transfer notebook/keep. This confirmed that the original UUID could still locate the data after migration.6. Update the content on Site B using the UUID
A PUT request using the UUID path changed Item to
Notebook after transfer. It returned HTTP 200, with integer ID 108, an unchanged UUID, and Status still set tokeep. The table in the screenshot had not yet been reloaded; the PUT response confirms the update.7. Query Site B again to confirm that the update was saved
A separate GET request using the UUID returned HTTP 200, with Item set to
Notebook after transferand both the integer ID and UUID unchanged. The reloaded table also displayed the new value.8. Query Site A again to confirm that the source data was unaffected
A final GET request to Site A using the original UUID returned HTTP 200, with integer ID 4, the same UUID, and the original content,
Transfer notebook/keep. This confirmed that Site B's update had not synchronized back to Site A.This manual test covers archive migration and local UUID GET/PUT operations. It does not cover DELETE, UUID collisions, historical backfill, view-sharing permissions, or federation operations.
Automated verification and upgrade measurements
Local verification records show that the full PHPUnit suite, MySQL/MariaDB/PostgreSQL database tests, Psalm, coding standards, PHP syntax checks, and TypeScript checks all passed. Regenerating the OpenAPI specification and types produced no differences.
An earlier SQLite benchmark with 1,000,014 rows recorded approximately 1.44 seconds for the schema/index migration and 41.48 seconds of total background worker time. All 165 sampled HTTP requests during backfill returned 200. This measurement predates the final cursor-write adjustment and has not been repeated; total worker time does not represent elapsed completion time under the normal cron schedule.
🏁 Checklist
/backport to stableX.X🤖 AI (if applicable)
Codex assisted with developing the code and tests, and Claude Code assisted with their review. I performed the manual migration test and captured the screenshots; Codex assisted with drafting the PR description.