MkDocs Plugin: Evaluation Function Documentation Loader

Mkdocs plugin for fetching additional .md files registered in a db before render. Specifically from a web request which returns all the available evaluation functions endpoints.

This plugin was specifially developed for the LambdaFeedback platform.

NOTE: There is currently no safety checking to make sure downloaded markdown files are valid and able to be rendered, they are simply copied over directly from the evaluation function endpoint

Configuration

Enable plugin in the mkdocs.yml file:

plugins:
  - evaldocsloader:
    functions_announce_endpoint: "http://127.0.0.1:5050/testingfunctions"
    api_key: !ENV [API_KEY, "disable"]
    dev_section: ["Developers", "Evaluation Functions"]
    user_section: ["Teachers, "Evaluation Functions"]

functions_announce_endpoint: Endpoint from which a list of evaluation functions be fetched

api_key Key to be passed onto the headers of the request ade to the functions announcing endpoint, used to authenticate the request.

dev_section and user_section: Paths under which the fetched documentation files should be included, for the developer and teacher-facing files respectively. Thes can be arbirarily long. In this example, developer documentation would be appended to content under the "Developers" section in the "Evaluation Functions" subsection.

Behaviour

This plugin hooks into three events:

on_config: After the config is loaded, a list of evaluation functions is fetched the endpoint specified in functions_announce_endpoint. Documentation files are fetched from each of the urls returned, and saved to a temporary directory. Successfully downloaded files are then registered to the nav config, under the sections specified in dev_section and user_section.

on_files: Downloaded files are appended onto the end of the main mkdocs.structure.files.Files object

on_post_build: The created temporary directory is cleaned up

For all events, if a plugin-breaking error occurs, it will be caught and evaluation function documentation fetching is aborted.

Installing

This package is distributed via tagged GitHub Releases, not PyPI. The PyPI project (evaldocsloader, last released 0.1.5 in 2023) is deprecated and no longer updated — do not install from it.

Pin a released tag in the consuming project:

• Poetry (pyproject.toml):

  evaldocsloader = { git = "https://github.com/lambda-feedback/EvalDocsLoader.git", tag = "v0.3.0" }

• pip / requirements.txt:

  evaldocsloader @ git+https://github.com/lambda-feedback/EvalDocsLoader.git@v0.3.0
  

Tracking branch = "main" / @main instead of a tag pulls in unreleased code and should only be done alongside a committed lock file.

Local development

Preferred:

poetry install

or, with plain pip:

pip install -e .

A small Flask API for testing lives in testing_api/. It is not part of the plugin — it just provides an endpoint to develop against.

Cutting a release (maintainers)

Releases are cut from the GitHub UI:

1. Actions → Release → Run workflow, on main.
2. Pick the version bump: patch, minor, major, or a pre* variant.
3. The Release workflow runs poetry version <bump>, commits the new version to main, creates the matching vX.Y.Z tag, runs poetry build, and publishes a GitHub Release with the sdist/wheel attached (pre* versions are marked as pre-releases). Confirm it under the Releases tab.
4. Bump the pinned tag in consuming repos (e.g. user-documentation/pyproject.toml) in a follow-up PR.

The workflow also runs on a manually pushed v* tag: it checks the tag matches pyproject.toml, then builds and releases the same way. Use this only to release outside the UI.

The "Run workflow" button pushes the version-bump commit and tag directly to the branch it runs from, so GitHub Actions must be allowed to write to the default branch: repo Settings → Actions → General → Workflow permissions → Read and write, with no branch-protection rule blocking the github-actions bot.

Sources/References

Plugin for loading external markdown files: https://github.com/fire1ce/mkdocs-embed-external-markdown

Template for plugins: https://github.com/byrnereese/mkdocs-plugin-template

File Selection: https://github.com/supcik/mkdocs-select-files

Dealing with new files: https://github.com/oprypin/mkdocs-gen-files/blob/master/mkdocs_gen_files/plugin.py