diff --git a/docs/build/.pages b/docs/build/.pages index aa964096b..10984d381 100644 --- a/docs/build/.pages +++ b/docs/build/.pages @@ -24,6 +24,7 @@ nav: - Rule Blocks: rule-blocks - Workflows: workflows - Variables: variables + - Change Journal: change-journal - Project Access Control: project-access-control - Reference: - Task and Operator Reference: reference diff --git a/docs/build/change-journal/change-journal-project-card.png b/docs/build/change-journal/change-journal-project-card.png new file mode 100644 index 000000000..cfc1850b3 Binary files /dev/null and b/docs/build/change-journal/change-journal-project-card.png differ diff --git a/docs/build/change-journal/change-journal-row-menu.png b/docs/build/change-journal/change-journal-row-menu.png new file mode 100644 index 000000000..76336040f Binary files /dev/null and b/docs/build/change-journal/change-journal-row-menu.png differ diff --git a/docs/build/change-journal/change-journal-view.png b/docs/build/change-journal/change-journal-view.png new file mode 100644 index 000000000..df443e8fd Binary files /dev/null and b/docs/build/change-journal/change-journal-view.png differ diff --git a/docs/build/change-journal/index.md b/docs/build/change-journal/index.md new file mode 100644 index 000000000..80aa1e927 --- /dev/null +++ b/docs/build/change-journal/index.md @@ -0,0 +1,170 @@ +--- +status: new +title: "Change Journal" +icon: material/source-commit +tags: + - Project + - Automate +--- + +# Change journal + +## Introduction + +The change journal records the changes to a Build project in eccenca Corporate Memory. +Each entry states who made the change, when, from which client and what it changed. + +The journal serves two purposes: + +- **Undo changes:** Almost every change can be reverted from the journal, whether it was made in the user interface, through the REST API or by an agent. +- **Review the work of agents:** An agent that works on a project through the [Build MCP server](../../automate/agent-integration/index.md#mcp) changes it without asking for each step. + Its changes stay marked as unreviewed until a person reviews them. + +Everyone who can open a project sees its change journal and can review and revert its changes. + +!!! info "Availability" + + The change journal is available starting with Corporate Memory version 26.3. + +## Recorded changes + +The journal records changes to the content of a project: + +- Tasks that are added, updated or removed, including datasets, transformations and workflows +- Single mapping rules, and the nodes and connections of a workflow +- Project variables +- Tags +- Uploaded and deleted files +- Workflow runs, and workflow runs proposed by an agent + +Project settings such as the label or the prefixes of the project are not recorded. + +## Open the change journal + +The **Changes** card on the project view summarizes the journal. +It states the number of recorded changes and how many changes made by an agent await review. + +![The Changes card of a project with six changes made by an agent awaiting review](change-journal-project-card.png){ class="bordered" width="50%" } + +Open the journal in one of the following ways: + +- Click :eccenca-item-viewdetails: in the header of the **Changes** card. +- Click **Review changes** in the **Changes** card, if changes await review. +- Click :eccenca-item-moremenu: in the header of the project and select **Changes**. + +The **Changes** view of the project opens. + +![The Changes view of a project with unreviewed changes made by an agent](change-journal-view.png){ class="bordered" } + +## Find a change + +The journal lists the changes newest first. +Each row shows: + +- **#:** the sequence number of the change in the project +- **Date:** when the change was made +- **User:** the account that made the change +- **Source:** an icon if the change was made by an agent; hover over the icon to see the client name +- **Change:** a summary of the change, the details of what it changed, and tags for its kind and state + +The icons at the end of a row open what the change refers to: + +- :eccenca-item-viewdetails: opens the task or the project. +- :eccenca-application-mapping: opens a mapping rule in the mapping editor. +- :eccenca-item-download: downloads an added file. +- A workflow run additionally links its execution report. + +To narrow down the list, use the following controls: + +- Enter a term in the search field to search the change summaries. +- Select values in the facets on the left: **KIND** (for example **Added**, **Updated**, **Run**), **STATE** (for example **Unreviewed**, **Proposed run**), **ORIGIN** (**Person** or the client name of an agent), **USER** and **TASK**. +- Click :eccenca-list-sort: to change the sort order. +- Click :eccenca-item-reload: to load changes made since the list was opened. +- Click :eccenca-item-question: to open a description of the change journal. + +## Revert a change + +A revert undoes exactly what a change did. +For example, reverting the addition of a mapping rule removes this rule and nothing else. + +To revert a change, follow these steps: + +1. Click :eccenca-item-moremenu: in the row of the change. +2. Select one of the following entries: + - **Revert this change** reverts the selected change only. + - **Revert this and all newer changes** returns the project to the state before the selected change. + + ![Menu of a change with the review and revert entries](change-journal-row-menu.png){ class="bordered" width="70%" } + +3. Click **Revert** to confirm. + +The revert is recorded as a new change. +Reverting this new change redoes the original change. + +A change can be reverted as long as what it changed is still as the change left it. +Otherwise, the revert entry is disabled, and its tooltip states the reason, for example that a newer change modified the task or that another task references it. +Reverting the newer changes first can make the revert possible again. + +A revert of several changes skips the changes that cannot be reverted and stops at the first conflict. +The result lists the reverted changes. + +!!! warning "Some changes cannot be reverted" + + A workflow run cannot be reverted. + Data that a run wrote, for example to a file, a Knowledge Graph or an external database, remains after the changes that the run used are reverted. + Overwriting or deleting a file cannot be reverted either. + +## Review changes made by an agent + +Changes that an agent makes through the Build MCP server are marked as **Unreviewed**. +The journal highlights them and states their number above the list. +The MCP server offers no tool to mark changes as reviewed, so an agent cannot approve its own work. + +To review the changes, follow these steps: + +1. Click **Review changes** in the **Changes** card of the project. +2. Check each unreviewed change, and revert the changes that are not wanted, as described in [Revert a change](#revert-a-change). +3. Mark the remaining changes as reviewed: + - Click **Mark all as reviewed** to mark all listed changes as reviewed. + Click :eccenca-item-reload: before, to include changes made since the list was opened. + - Alternatively, click :eccenca-item-moremenu: in the row of a change and select **Mark this and older changes as reviewed**. + This marks the selected change and all older changes, and leaves the newer changes for later. +4. Click **Confirm**. + +To reject all unreviewed changes at once, click **Revert unreviewed** and then **Revert**. +The changes are reverted newest first. + +### Workflow runs proposed by an agent + +While the review gate is enabled, which is the default, an agent does not run a workflow while unreviewed changes exist, because a run would use these changes and its results cannot be reverted. +Instead, the request is recorded as a change of the state **Proposed run**, and the agent is asked to wait for the review. + +- After all changes are marked as reviewed, the next request of the agent runs the workflow. + The run is recorded with a link to its execution report. +- Reverting the proposed run rejects it. + +## Configuration and API + +The journal of a project is stored on the file system of Build and survives a restart. +A project export contains the journal by default. +The project export endpoints accept the parameter `exportChangeJournal=false` to leave it out. + +Administrators configure the journal and the review gate for agents in the Build configuration (`dataintegration.conf`): + +`workspace.changes.plugin` + +: Selects where the journal is kept: `fileChangeJournal` on the file system, `inMemoryChangeJournal` in memory for the most recent changes only, or `emptyChangeJournal` to disable the journal together with the review. + +`workspace.changes.fileChangeJournal.dir` + +: The directory of the file-based journal. + +`com.eccenca.di.assistant.McpConfig.reviewGate` + +: Makes workflow runs requested through the MCP server wait for review, as described in [Workflow runs proposed by an agent](#workflow-runs-proposed-by-an-agent). + The default is `true`. + While the review gate is enabled, the MCP server does not offer the `resource_delete` tool, because deleting a file cannot be reverted. + Set it to `false` for automation without a reviewer: workflow runs then execute right away, and `resource_delete` is available. + +The journal of a project is also available through the REST API under `/api/workspace/projects/{project}/changes`. +See the [Build (DataIntegration) API documentation](../../develop/dataintegration-apis/index.md) for details. diff --git a/docs/build/introduction-to-the-user-interface/build-ui-project-import.png b/docs/build/introduction-to-the-user-interface/build-ui-project-import.png new file mode 100644 index 000000000..85568143d Binary files /dev/null and b/docs/build/introduction-to-the-user-interface/build-ui-project-import.png differ diff --git a/docs/build/introduction-to-the-user-interface/build-ui-project.png b/docs/build/introduction-to-the-user-interface/build-ui-project.png new file mode 100644 index 000000000..6448b3e5d Binary files /dev/null and b/docs/build/introduction-to-the-user-interface/build-ui-project.png differ diff --git a/docs/build/introduction-to-the-user-interface/build-ui-workbench.png b/docs/build/introduction-to-the-user-interface/build-ui-workbench.png index 53d464979..ec786f5b0 100644 Binary files a/docs/build/introduction-to-the-user-interface/build-ui-workbench.png and b/docs/build/introduction-to-the-user-interface/build-ui-workbench.png differ diff --git a/docs/build/introduction-to-the-user-interface/index.md b/docs/build/introduction-to-the-user-interface/index.md index 2f1c8ec12..4e1be26ed 100644 --- a/docs/build/introduction-to-the-user-interface/index.md +++ b/docs/build/introduction-to-the-user-interface/index.md @@ -9,7 +9,7 @@ This page provides a short introduction to the BUILD / Data Integration workspac The workbench is the main entry point to the BUILD interface and provides a list view of your work, structured in _projects_. -![](build-ui-workbench.png) +![The Build workbench listing projects, datasets and workflows with the item type facets on the left](build-ui-workbench.png){ class="bordered" } On the left-hand side, a facet list provides an overview of different item types: @@ -21,15 +21,14 @@ On the left-hand side, a facet list provides an overview of different item types The interface can be used to browse from a global level to a project level as well as to search for items globally and project wide. -On the top right side in the header, there is a big blue **Create (+)** button which allows for creation of all possible data integration items: +Click **Create new** in the header to create any kind of Build item: ![](build-ui-datasets.png) ## Projects -Selecting a project from the search results opens the project details view (see above). - -The project is the main concept to structure your work. In the workspace, you can review your projects, create new or delete old ones, and import or export them. +The project is the main concept to structure the work in Build. +In the workbench, projects are created, imported, exported and deleted. Each project is self-contained and holds all relevant parts, such as the following: @@ -42,6 +41,45 @@ Each project is self-contained and holds all relevant parts, such as the followi Resources, datasets, tasks and other parts of a project can be added and configured from within the workspace. By clicking the `Show Details` button on any item in a project, you can ‘look inside’ and define the actual transformation or linking rules, or assemble the actual workflow. +### Project view + +Selecting a project in the workbench opens the project view. + +![The project view of the Product Catalog project with its contents and the cards on the right](build-ui-project.png){ class="bordered" } + +The project view shows the following areas: + +- **Summary:** the label and description of the project +- **Contents:** the items of the project, with a search field and facets +- **Configuration:** the prefixes of the project +- **Project variables:** the [variables](../variables/index.md) of the project +- **Activities:** the activities of the project and their state +- **Changes:** the number of recorded changes and of changes made by an agent that await review, see [Change journal](../change-journal/index.md) +- **Files:** the files uploaded to the project + +### Import a project + +A project exported as a project file can be imported into the workbench. +To import a project, follow these steps: + +1. Click **Create new** in the header. +2. Select **Project** and click **Add**. +3. Click **Import project file**. +4. Upload the project file. + The **Project import** dialog shows the label and the description of the imported project. +5. Select an **Import destination**: + - **Use original project ID** imports the project under the identifier stored in the project file. + - **Generate a new project ID** imports the project under a new identifier. + - **Use a custom project ID** imports the project under the identifier entered in the **Project identifier** field. + + ![The Project import dialog with the import destinations and the warning about an existing project](build-ui-project-import.png){ class="bordered" width="70%" } + +6. Click **Import project**. + +If a project with the selected identifier already exists, the dialog shows a warning. +Select **Replace existing project** and click **Replace existing project** to overwrite the existing project, or select another import destination. +Click **Open existing project page** to inspect the existing project first. + ## Datasets Selecting a dataset in a project or from the search results opens the dataset details view. diff --git a/nav.yml b/nav.yml index ccbd52275..138cc00e8 100644 --- a/nav.yml +++ b/nav.yml @@ -74,6 +74,8 @@ nav: - build/workflows/index.md - Variables: - build/variables/index.md + - Change Journal: + - build/change-journal/index.md - Project Access Control: - build/project-access-control/index.md - Reference: