Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .claude/docs-guidelines/repo-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,11 @@ Do not write `index.html` stubs under `docs/`.
- `class="bordered"` on every product screenshot (436 uses in the tree).
- `width="50%"` or `width="70%"` for dialogs and modals, no width for full-screen views. Keep the width
consistent within one page.
- A screenshot needs a width small enough for its pixels, or it prints coarse: an image of 900 pixels
across the full 16 cm column prints at 143 ppi. `poetry run dec-tool image-widths` lists every raster
image in a page that is not generated that prints below 150 ppi, and `--fix` writes the `width="NN%"`
that reaches it — the pixel width divided by the column and the target, rounded down. The same width
then governs the site.
- `.off-glb` opts an image out of the glightbox lightbox — used for inline icons and decorative images.
- File names: lowercase, hyphen-separated, descriptive (style guide 5.5).
- Delete screenshots that are no longer referenced. Orphan check for one page directory:
Expand Down Expand Up @@ -204,6 +209,24 @@ Known drift to fix when touching a page: 32 `shell-session` blocks in hand-writt
numbering restarts at 1 in the rendered page. Check the rendered result for any step list you touch.
- `sane_lists` is enabled: an unordered item does not continue an ordered list.

## Leaving content out of the print edition

The print edition of the PDF (`task pdf:print`, `tasks/spec.md` §10) can leave out content that only makes sense online.

- A subtree or a single page: an `omit` entry under `sections` in `tools/pdf/print.yml`, keyed by its `docs/` path.
The page itself carries no marker, so this works for generated pages too.
- A part of a page: the class `print-exclude` on the block. The print edition prints a note in its place that points
to the page online; the site and the screen PDF show the part unchanged.

| Block | Markup |
| --- | --- |
| code block | `sql { .print-exclude }` right after the backticks of the opening fence |
| admonition or collapsible block | `??? example print-exclude "INSERT query"` |
| paragraph | `{ .print-exclude }` on the line after the paragraph |
| several blocks | `<div class="print-exclude" markdown>` … `</div>`, with blank lines around the content |

A generated page cannot carry the class: the next generator run removes it.

## Editorial decisions beyond the style guide

`style-guide.md` is a transcription of the Confluence page and is not extended locally.
Expand Down
133 changes: 123 additions & 10 deletions .github/workflows/pdf.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,24 @@
---
name: PDF

# Builds the PDF of the whole documentation after every push to main, merged
# pull requests included, and keeps it as an artifact of the workflow run.
# Builds every PDF of the documentation after a push to main or to the
# print-on-demand branch: the screen edition, the print edition (the book block
# for print on demand), its PDF/X-4 copy in CMYK and the greyscale preview. The
# run summary lists all four with their download links, so they need not be dug
# out of the artifact section.
on:
push:
branches: ["main"]
branches: ["main", "feature/print-on-demand"]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

permissions:
contents: read

# A newer push to main makes a running build obsolete.
# A newer push to the same branch makes a running build obsolete; main and a
# branch do not cancel each other.
concurrency:
group: "pdf"
group: "pdf-${{ github.ref }}"
cancel-in-progress: true

env:
Expand Down Expand Up @@ -62,8 +68,10 @@ jobs:
- name: update packages
run: sudo apt-get update -y

# Poppler carries the pdffonts, pdfimages and pdftotext the preflight report
# reads. Ghostscript comes from a container below, not from apt.
- name: install packages
run: sudo apt-get install -y libffi-dev libz-dev
run: sudo apt-get install -y libffi-dev libz-dev poppler-utils

- name: install pandoc
run: |
Expand All @@ -79,14 +87,119 @@ jobs:
| tar -xJ -C "$RUNNER_TEMP"
sudo install -m 0755 "$RUNNER_TEMP/typst-x86_64-unknown-linux-musl/typst" /usr/local/bin/typst

- name: build pdf
# The ISO Coated v2 profile of the output intent is not in the repository
# (its licence forbids it), so it is fetched once and kept between runs.
- name: cache the output intent profile
uses: actions/cache@v4
with:
path: dist/icc
key: icc-iso-coated-v2-eci

# Ubuntu ships Ghostscript 10.02, where `-dPDFX=4` is still a boolean, so
# the 4 raises `/typecheck in --pdfmark--` and PDF/X-4 cannot be written at
# all. A container carries a newer one, behind a shim that runs it at the
# paths the build uses, so the PDFs, the pdfmark prefix and the profiles
# resolve inside the container exactly as they do outside. The grey output
# intent profile comes out of the same image.
- name: ghostscript from a container
run: |
printf 'FROM alpine:edge\nRUN apk add --no-cache ghostscript\n' | docker build -q -t pdfx-gs -
W="$GITHUB_WORKSPACE"
sudo tee /usr/local/bin/gs-docker >/dev/null <<SHIM
#!/bin/sh
exec docker run --rm -v "$W:$W" -v /tmp:/tmp -w "$W" pdfx-gs gs "\$@"
SHIM
sudo chmod +x /usr/local/bin/gs-docker
gs-docker --version
mkdir -p dist/icc
docker run --rm pdfx-gs cat /usr/share/ghostscript/iccprofiles/default_gray.icc \
> dist/icc/default_gray.icc
{
echo "GHOSTSCRIPT=/usr/local/bin/gs-docker"
echo "PDF_GRAY_PROFILE=$W/dist/icc/default_gray.icc"
} >> "$GITHUB_ENV"

- name: build the screen pdf
run: task pdf

- name: upload pdf
- name: build the print pdfs
run: task pdf:print -- --normalize --gray

- name: upload the screen pdf
id: screen
uses: actions/upload-artifact@v4
with:
name: documentation-eccenca-com-pdf
path: dist/documentation-eccenca-com-*.pdf
name: pdf-screen
path: |
dist/documentation-eccenca-com-*.pdf
!dist/documentation-eccenca-com-*-print*.pdf
if-no-files-found: error
retention-days: 30
# A PDF is compressed already.
compression-level: 0

- name: upload the print pdf
id: print
uses: actions/upload-artifact@v4
with:
name: pdf-print
path: dist/documentation-eccenca-com-*-print.pdf
if-no-files-found: error
retention-days: 30
compression-level: 0

- name: upload the pdf/x-4 pdf
id: pdfx
uses: actions/upload-artifact@v4
with:
name: pdf-print-x4
path: dist/documentation-eccenca-com-*-print-x4.pdf
if-no-files-found: error
retention-days: 30
compression-level: 0

- name: upload the greyscale pdf
id: gray
uses: actions/upload-artifact@v4
with:
name: pdf-print-gray-x4
path: dist/documentation-eccenca-com-*-print-gray-x4.pdf
if-no-files-found: error
retention-days: 30
compression-level: 0

# The table is built in one go and printed to the log as well, so the
# editions and their links are readable even if the summary ever stays
# empty - which is how the first run of this step ended.
- name: list the pdfs in the run summary
run: |
# $1 is a glob; $4, when given, drops what the glob also matches - the
# screen edition is the one file whose name carries no `-print`.
describe() {
file="$(ls $1 2>/dev/null | grep -v -- "${4:-^$}" | head -1)"
if [ -z "$file" ]; then
printf '| %s | - | - | missing |\n' "$2"
return
fi
printf '| %s | %s | %s | [%s](%s) |\n' "$2" \
"$(pdfinfo "$file" 2>/dev/null | awk '/^Pages/ {print $2}')" \
"$(du -h "$file" | cut -f1)" "$(basename "$file")" "$3"
}
{
echo "## PDF editions of ${GITHUB_REF_NAME}"
echo
echo "| Edition | Pages | Size | Download |"
echo "| :------ | ----: | ---: | :------- |"
describe "dist/documentation-eccenca-com-*.pdf" \
"Screen" "${{ steps.screen.outputs.artifact-url }}" "-print"
describe "dist/documentation-eccenca-com-*-print.pdf" \
"Print, book block" "${{ steps.print.outputs.artifact-url }}"
describe "dist/documentation-eccenca-com-*-print-x4.pdf" \
"Print, PDF/X-4 in CMYK" "${{ steps.pdfx.outputs.artifact-url }}"
describe "dist/documentation-eccenca-com-*-print-gray-x4.pdf" \
"Print, greyscale preview" "${{ steps.gray.outputs.artifact-url }}"
echo
echo "The print editions carry the preflight report of \`dec-tool pdf-preflight\`;"
echo "a failed check fails this job."
} | tee -a "$GITHUB_STEP_SUMMARY"
echo "summary file: $GITHUB_STEP_SUMMARY, $(wc -c < "$GITHUB_STEP_SUMMARY") bytes"
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ tmp
.python-version
dist/
.env
*.sw[a-p]
*~
67 changes: 67 additions & 0 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ tasks:
cmds:
- task: check:links
- task: check:rumdl
- task: check:images
- task: check:yamllint
- task: check:navigation
- task: check:output
Expand All @@ -82,6 +83,18 @@ tasks:
# CI run agree: GitHub runners have no user-level yamllint config.
- poetry run yamllint .

check:images:
desc: Fail if a screenshot prints below 150 ppi in the print edition
summary: |
Runs `dec-tool image-widths`: every raster image in a page that is not
generated must carry a width small enough for its pixels, or it prints
coarse (tasks/spec.md, §11, D18). `dec-tool image-widths --fix` writes
the width a listed image needs.
deps:
- install
cmds:
- poetry run dec-tool image-widths

check:navigation:
desc: Fail if nav.yml is out of sync with the docs/**/.pages files
deps:
Expand Down Expand Up @@ -200,6 +213,60 @@ tasks:
cmds:
- poetry run dec-tool build-pdf

pdf:print:
desc: Build the print edition of the PDF - the book block for print on demand
summary: |
Runs `task build`, then `dec-tool build-pdf --edition print`: the same
merge, pandoc run and fonts as `task pdf`, typeset as the book block of
a printed book (BoD, A4, black and white). tasks/spec.md describes it:
mirrored page furniture, right-hand part starts, title page and
imprint, links printed as page references and footnotes, and images
normalized to 300 ppi without transparency.

tools/pdf/print.yml configures the edition: the publisher, the authors'
names and exclusions, and how each navigation section is printed (full,
list or omit). tools/pdf/authors.yml is the imprint's author list;
refresh it with `task pdf:authors`.

Writes dist/documentation-eccenca-com-<ver>-print.pdf; the merged HTML,
the Typst source and the normalized images stay in dist/pdf/print/. The
screen PDF of `task pdf` is not affected.

The build lists images below 150 ppi at their printed size in
dist/pdf/print/low-resolution.tsv and runs the preflight report
(`dec-tool pdf-preflight`), failing on an error. `task pdf:print --
--normalize` (or PDF_NORMALIZE=1) also writes a PDF/X-4 copy in CMYK
with Ghostscript, -print-x4.pdf, and checks that one; the ISO Coated v2
profile is fetched from the ECI into dist/icc/ unless PDF_ICC_PROFILE
names one. `task pdf:print -- --gray` (or PDF_GRAY=1) writes a
greyscale preview, -print-gray-x4.pdf, to check on screen how the book
prints in black and white; it is PDF/X-4 by the grey profile Ghostscript
ships, and the two flags combine.
deps:
- task: build
env:
BUILD_VERSION: '{{.CURRENT_VERSION}}'
cmds:
- poetry run dec-tool build-pdf --edition print {{.CLI_ARGS}}

pdf:authors:
desc: Refresh the author list of the print edition's imprint
summary: |
Writes tools/pdf/authors.yml from the commits to what the print edition
prints - its pages that are not generated, and the images they
reference: most commits first, with the names the GitHub profiles show,
without anonymous commits, bot accounts, software agents and the IDs
excluded in tools/pdf/print.yml. Adds each author that
print.yml does not list yet to its authors.names, without a name, and
warns about each author who would print as a GitHub ID. Commit both
files. Uses GITHUB_TOKEN or GH_TOKEN, else the token of a logged-in
GitHub CLI (`gh auth login`); without a token GitHub allows 60
requests an hour, and a run takes one per author.
deps:
- task: install
cmds:
- poetry run dec-tool pdf-authors

update:icons:
desc: update the used eccenca icons from carbon
cmds:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Click the :eccenca-module-marketplace: **Packages** icon in the main menu under
Search for the required ontologies / vocabularies and click the **Install** button.
Wait for a package installation to complete (the **Install** button will change to **Uninstall**) before installing the next package.

![Search and install ontologies](pdwviw-marketplace-schema-search.png){ class="bordered" }
![Search and install ontologies](pdwviw-marketplace-schema-search.png){ class="bordered" width="90%" }

## 2 Create a new project

Expand Down
2 changes: 1 addition & 1 deletion docs/build/extracting-data-from-a-web-api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ To create a JSON-to-RDF-mapping within Corporate Memory, we have to first regist

1. Click the **Create** button (top right) in the data integration workspace and select the type **JSON**.

![Create JSON dataset](create-new-json-dataset.png)
![Create JSON dataset](create-new-json-dataset.png){ width="71%" }

2. Upload the JSON file [repos.json](repos.json) (API response) as a Dataset into Corporate Memory.

Expand Down
2 changes: 1 addition & 1 deletion docs/build/lift-data-from-json-and-xml-sources/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ The transformation defines how an input dataset (e.g.: JSON or XML) will be tran

=== "JSON"

![Examples of target data JSON](ldfjaxs-json-examples-target-data.png){ class="bordered" }
![Examples of target data JSON](ldfjaxs-json-examples-target-data.png){ class="bordered" width="64%" }

=== "XML"

Expand Down
4 changes: 2 additions & 2 deletions docs/build/loading-jdbc-datasets-incrementally/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@ To extract data from a relational database, you need to first register a **JDBC

1. In Corporate Memory, click Projects under **BUILD** in the navigation on the left side of the page.

![Menu BUILD > Projects](incr-jdbc-menu-build-projects.png){ class="bordered" }
![Menu BUILD > Projects](incr-jdbc-menu-build-projects.png){ class="bordered" width="29%" }

2. Click **Create** at the top of the page.
3. In the Create new item window, select Project and click Add. The Create new item of type Project window appears.
4. In the Create new item window, select Dataset under **Item Type**, search for JDBC endpoint, and click **Add**.

![Create JDBC Dataset](create-dataset-JDBC.png){ class="bordered" }
![Create JDBC Dataset](create-dataset-JDBC.png){ class="bordered" width="71%" }

5. Provide the required configuration details for the JDBC endpoint:
- **Label**: Provide a table name.
Expand Down
2 changes: 1 addition & 1 deletion docs/build/reference/transformer/Replace/excelMap.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Excel map"
description: "Replaces values based on a map of values read from a file in Open XML format (XLSX). The XLSX file may contain several sheets of the form: ```mapFrom,mapTo <source string>,<target string>``` and more. An empty string can be created in Excel and alternatives by inserting `=''` in the input line of a cell. If there are multiple values for a single key, all values will be returned for the given key. Note that the mapping table will be cached in memory. If the Excel file is updated (even while transforming), the map will be reloaded within seconds."
description: "Replaces values based on a map of values read from a file in Open XML format (XLSX). The XLSX file may contain several sheets of the form: `mapFrom,mapTo <source string>,<target string>` and more. An empty string can be created in Excel and alternatives by inserting `=''` in the input line of a cell. If there are multiple values for a single key, all values will be returned for the given key. Note that the mapping table will be cached in memory. If the Excel file is updated (even while transforming), the map will be reloaded within seconds."
icon: octicons/cross-reference-24
tags:
- TransformOperator
Expand Down
3 changes: 1 addition & 2 deletions docs/build/reference/transformer/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,7 @@ Transform operators transform a one or more sequences of string values to a sequ
| [Excel Ln](Excel/Excel_LN.md) | Excel | Excel LN(number): Returns the natural logarithm based on the constant e of the given number. |
| [Excel Log](Excel/Excel_LOG.md) | Excel | Excel LOG(number; base): Returns the logarithm of the given number to the specified base. Base is the base for the logarithm calculation. |
| [Excel Log10](Excel/Excel_LOG10.md) | Excel | Excel LOG10(number): Returns the base-10 logarithm of the given number. |

| [Excel map](Replace/excelMap.md) | Replace | Replaces values based on a map of values read from a file in Open XML format (XLSX). The XLSX file may contain several sheets of the form: ```mapFrom,mapTo <source string>,<target string>``` and more. An empty string can be created in Excel and alternatives by inserting `=""` in the input line of a cell. If there are multiple values for a single key, all values will be returned for the given key. Note that the mapping table will be cached in memory. If the Excel file is updated (even while transforming), the map will be reloaded within seconds. |
| [Excel map](Replace/excelMap.md) | Replace | Replaces values based on a map of values read from a file in Open XML format (XLSX). The XLSX file may contain several sheets of the form: `mapFrom,mapTo <source string>,<target string>` and more. An empty string can be created in Excel and alternatives by inserting `=""` in the input line of a cell. If there are multiple values for a single key, all values will be returned for the given key. Note that the mapping table will be cached in memory. If the Excel file is updated (even while transforming), the map will be reloaded within seconds. |
| [Excel Max](Excel/Excel_MAX.md) | Excel | Excel MAX(number_1; number_2; ... number_30): Returns the maximum value in a list of arguments. Number_1; number_2; ... number_30 are numerical values or ranges. |
| [Excel Maxa](Excel/Excel_MAXA.md) | Excel | Excel MAXA(value_1; value_2; ... value_30): Returns the maximum value in a list of arguments. Unlike MAX, text can be entered. The value of the text is 0. Value_1; value_2; ... value_30 are values or ranges. |
| [Excel Median](Excel/Excel_MEDIAN.md) | Excel | Excel MEDIAN(number_1; number_2; ... number_30): Returns the median of a set of numbers. Number_1; number_2; ... number_30 are values or ranges, which represent a sample. Each number can also be replaced by a reference. |
Expand Down
Loading
Loading