Skip to content

Optional hierarchical documentation output and OKF-compatible export for agents #125

Closed

Description

@Soveilove

CodeWiki produces high-quality documentation. Its coverage of business flows and implementation details is much more useful than a simple code summary.

Currently, the generated Markdown files are mostly stored in a flat directory. For large repositories, this makes it harder for an agent to navigate by business area and find the relevant documents. Would you consider an optional hierarchical output structure, with an index at each level?

Would an Open Knowledge Format (OKF)–compatible export also be worth exploring? Structured metadata such as document type, related modules, and source-code references might help agents select the right documents without loading unrelated content. The potential improvements in token usage and context quality are a hypothesis that could be evaluated against real retrieval tasks.

I’d suggest keeping the current output as the default, with hierarchical output or OKF export as optional features. Any new structure would need to keep file paths, internal links, module_tree.json, incremental updates, and the existing viewer consistent. I’d also be interested to know whether CodeWiki’s current MCP capabilities already address part of this use case.

Activity

  1. added a commit that references this issue on Oct 2, 2026
    0815b54
  2. anhnh2002 commented on Oct 2, 2026

    @anhnh2002
    Collaborator

    Thanks for the detailed suggestion, and for the kind words about the docs!

    Some background on why the output was flat: when we first built CodeWiki, smaller models had trouble with a nested folder layout. They kept writing wrong relative links between pages, so we kept every page in one folder. Current models handle the hierarchy well, so instead of adding hierarchy as an option, we're making it the default. #128 does this:

    • Pages follow the module tree. A module's page sits next to the folder of its sub-modules (auth.md, auth/login.md, auth/login/tokens.md), so each parent page also works as the index for its folder. overview.md stays at the root as the top-level index.
    • Links between pages are relative. After each run, CodeWiki moves any page an agent saved in the wrong place and repairs links that point to the wrong path.
    • --flat keeps the current layout for small models.
    • Your consistency points are covered:
    • module_tree.json is unchanged; module names stay unique and are still the page filenames.
    • The layout is recorded in metadata.json, and --update keeps it. Existing flat docs stay flat.
    • The GitHub Pages viewer and the web app follow the layout.
    • For MCP, processing_order.json now gives each module's doc_path, so an agent driving CodeWiki through MCP writes pages in the same structure.

    OKF export is out of scope for this PR. The idea of letting agents pick documents by metadata (type, related modules, source references) instead of loading everything is interesting, and we'd want to measure it on real retrieval tasks, as you suggest. Let's keep it as a follow-up; if you have an OKF schema or example consumers in mind, please share them.

    Feedback on #128 is welcome, especially if you try it on a large repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions