Skip to content

docs: document unified memory env configuration - #1051

Open
amarkdotdev wants to merge 3 commits into
docker:mainfrom
amarkdotdev:docs/uma-env-config-994
Open

docs: document unified memory env configuration#1051
amarkdotdev wants to merge 3 commits into
docker:mainfrom
amarkdotdev:docs/uma-env-config-994

Conversation

@amarkdotdev

Copy link
Copy Markdown

Summary

  • Document GGML_CUDA_ENABLE_UNIFIED_MEMORY for Docker Compose, docker model run, and dmr

Fixes #994

Test plan

  • Verified docs/unified-memory.md and README link

Fixes docker#994

Signed-off-by: Aaron <amark@g.jct.ac.il>
Fixes docker#994

Signed-off-by: Aaron <amark@g.jct.ac.il>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="docs/unified-memory.md" line_range="20" />
<code_context>
+## docker model run
+
+```shell
+docker model run -e GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 ai/gemma3 "Hello"
+```
+
</code_context>
<issue_to_address>
**issue (bug_risk):** The documented `docker model run -e GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 ...` command fails because this repository's `run` command does not define an `-e`/`--env` flag, so Cobra rejects the option instead of configuring the runner.

**Triggers:** When users follow the Docker Model Runner CLI example.

**Suggested fix:** Set the variable in the environment before invoking `docker model run`, or document a supported runner/container configuration mechanism instead of passing `-e` to `docker model run`.

```suggestion
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 docker model run ai/gemma3 "Hello"
```
</issue_to_address>

### Comment 2
<location path="docs/unified-memory.md" line_range="1" />
<code_context>
+# Unified memory configuration for integrated GPUs
+
+Docker Model Runner uses llama.cpp under the hood. On systems with integrated GPUs (for example AMD APUs), available shared memory may be reported incorrectly unless unified memory is enabled.
</code_context>
<issue_to_address>
**nitpick:** The new unified-memory document is not linked from `README.md` or any existing documentation index, so the repository's stated README-link verification is false and users browsing the README cannot discover this configuration guide.

**Suggested fix:** Add a README link to `docs/unified-memory.md`, preferably alongside the existing documentation resources.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread docs/unified-memory.md Outdated
Comment thread docs/unified-memory.md
Use env-prefix syntax for docker model run instead of invalid -e flag.
Add unified-memory.md to README Additional Resources.

Signed-off-by: Aaron <amark@g.jct.ac.il>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Available unified memory in llama.cpp

1 participant