Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions docs/core-internals/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Apache Cloudberry — Core Internals

A developer-oriented handbook for the Cloudberry kernel: storage, indexing,
the catalog, the executor (single-node and distributed), the dual optimizer
(PostgreSQL planner and ORCA), transactions, HA/recovery, and resource
management.

21 chapters, 104 sections, organised into five parts after the structure of
Rogov's *PostgreSQL 14 Internals*.

## What is here

```
index.md landing page and chapter index
part-1-introduction/ ch01
part-2-storage-data-layout/ ch02 .. ch08
part-3-transactions-mvcc-concurrency/ ch09 .. ch12
part-4-query-processing/ ch13 .. ch19
part-5-high-availability-recovery/ ch20
part-6-resource-management/ ch21
figures/ 64 SVG figures
_components/ the one MDX component + its CSS
```

Each part directory carries a `_category_.json` so the sidebar can be
generated from the directory structure.

## How the website consumes this

The directory is written for Docusaurus and is intended to be pulled into
`apache/cloudberry-site` the same way the PXF docs are — as a git submodule
with a sparse checkout limited to this path, served by its own docs plugin
instance with an autogenerated sidebar. Nothing in this directory depends on
site-level configuration except one small MDX component (see below), so it
can also be rendered by any other Docusaurus site or read directly on
GitHub.

This directory is self-contained: a Docusaurus site can point a docs plugin
at it and build, with no site-level registration of components and no
additions to the site's stylesheet. Verified by building it against a site
whose `customCss` is empty and which registers nothing.

- **`_components/`** holds the one MDX component these pages need, together
with its stylesheet. 37 figures are laid out with HTML and CSS rather than
drawn as SVG; their markup carries inline style attributes, void tags and,
in two cases, unbalanced tags, none of which MDX accepts as JSX. Each page
that uses such a figure imports the component itself, so nothing has to be
wired up on the site side. The leading underscore keeps the directory out
of the docs plugin's page routing.
- **Mermaid** is the one optional extra. 113 diagrams are `mermaid` fenced
blocks: a site with `@docusaurus/theme-mermaid` enabled draws them, and
without it they degrade to readable code blocks. GitHub renders them
natively.

## Conventions

- **Code excerpts carry their origin.** Every code block is fenced with
`title="path/to/file.c:LINE"`, so a reader can go straight to the source.
372 excerpts are cited this way.
- **Sessions are real output.** 487 `psql` sessions are captured from a
running cluster, split into an input block and an `output` block. They are
transcripts, not illustrations.
- **Cross-references are links.** Internal `§N.M` references resolve to the
target section's anchor, and are validated at build time.
- **No environment-specific detail.** Host names, addresses and install
paths from the machine the sessions were captured on are replaced with
documentation-safe equivalents of the same width, so column alignment in
captured output is preserved.

## Maintenance

The chapters describe a specific point in the tree's history. File and line
references drift as the code changes; the prose is the durable part and the
line numbers are the perishable part. Two things follow:

- Treat a chapter as documentation of a version, not of `main` forever.
- The `title="file:line"` convention is machine-readable on purpose: a CI
job can check that every cited path still exists, which catches the worst
class of drift (a file that moved or was deleted) without needing to
re-read the prose.

Corrections are welcome as ordinary pull requests against the chapter file.
60 changes: 60 additions & 0 deletions docs/core-internals/_components/RawFigure.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

/* Layout for the HTML-and-CSS figures, lifted from the handbook's own
stylesheet so the figures render identically here. */

.book-figure { margin: 1.4rem 0; }
.book-figure figcaption {
margin-top: .5rem;
font-size: .9em;
color: var(--ifm-color-emphasis-600);
}

/* Selective highlight: the handbook wraps a key token so it stands out.
Same three rules as the standalone edition. */
.hl { border-radius: 2px; font-weight: 700; }
figure.code .hl { background: #fff8c5; color: #1f2328; }
figure.session pre code .hl { background: #bb800940; color: #ffea7f; }
.book-figure .hl { background: #fff8c5; color: #1f2328; padding: 0 2px; }

/* Layered module map and the banded brick figures. */
.bricks { border:1px solid var(--border);border-radius:12px;overflow:hidden;background:var(--soft) }
.bk-band { display:flex;align-items:stretch;border-top:1px solid var(--border) }
.bk-band:first-child { border-top:none }
.bk-lbl { flex:0 0 116px;display:flex;flex-direction:column;justify-content:center;gap:2px;padding:10px 13px }
.bk-lbl b { font-size:12.5px;font-weight:800;color:var(--fg);line-height:1.2 }
.bk-lbl span { font-size:9.5px;font-weight:700;color:var(--muted);text-transform:uppercase;letter-spacing:.5px }
.bk-row { flex:1 1 auto;display:flex;flex-wrap:wrap;gap:7px;padding:9px 10px }
.bk { flex:1 1 118px;min-width:104px;display:flex;flex-direction:column;justify-content:center;gap:2px; padding:8px 11px;border-radius:7px;border:1px solid;background:#fff;text-decoration:none;transition:.13s }
.bk-t2 { font-size:12.5px;font-weight:700;line-height:1.25 }
.bk-ch { font-size:10.5px;font-weight:700;font-variant-numeric:tabular-nums;opacity:.8 }
.bk:hover { transform:translateY(-2px);box-shadow:0 4px 12px #0000001f;text-decoration:none }
.bk-q { background:#eef6ff }
.bk-q .bk { border-color:#54aeff66;color:#0a3069;background:#ddf4ff }
.bk-s { background:#eefcf2 }
.bk-s .bk { border-color:#4ac26b80;color:#04260f;background:#dafbe1 }
.bk-t { background:#fff6ef }
.bk-t .bk { border-color:#fb8f4480;color:#5e2700;background:#fff1e5 }
.bk-h { background:#fff4f3 }
.bk-h .bk { border-color:#ff818280;color:#5c0011;background:#ffebe9 }
.bk-r { background:#f3f5f8 }
.bk-r .bk { border-color:#afb8c1;color:#1f2328;background:#eaeef2 }
.bk-band { flex-direction:column }
.bk-lbl { flex-basis:auto;border-bottom:1px solid var(--border) }
40 changes: 40 additions & 0 deletions docs/core-internals/_components/RawFigure.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import React from 'react';
import './RawFigure.css';

/**
* Renders a figure that is laid out with HTML and CSS rather than drawn as SVG.
*
* The markup arrives as a string and is injected, instead of being written as
* JSX in the page. That is deliberate: these figures carry inline style
* attributes, void tags and — in two cases — unbalanced tags, none of which
* MDX will accept as JSX. Passing the markup as a string keeps MDX out of it
* and leaves the rendering to the browser, so the figure looks the way it
* looks in the standalone edition of the handbook.
*/
export default function RawFigure({html, caption}) {
return (
<figure className="book-figure">
<div dangerouslySetInnerHTML={{__html: html}} />
{caption ? <figcaption>{caption}</figcaption> : null}
</figure>
);
}
68 changes: 68 additions & 0 deletions docs/core-internals/figures/fig-001.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading