[Lushiyao5]

Motivation & Context

FileMemoryProvider derives a memory's description sidecar by replacing the file's extension: both notes.md and notes map to notes_description.md. Two distinct memories therefore share one sidecar, so writing a description for one silently rewrites the other's, rewriting one file without a description deletes the other's description, and deleting either file removes both.

Description & Review Guide

• What are the major changes? _description_file_name now appends _description.md to the whole memory file name (notes.md → notes.md_description.md), which makes the derivation injective: distinct memory files always get distinct sidecars. The module docstring, AGENTS.md, and the existing tests are updated to the new name, and tests cover names that share a stem (notes.md / notes), rewriting one file without a description, and deleting one file.
• What is the impact of these changes? Sidecar names for memories that have an extension now include that extension. Sidecars written by earlier versions (extension-less stem) are not migrated; AGENTS.md documents this. Reading and rewriting a memory recreates its sidecar under the new name, and the old orphaned sidecar is simply unused.
• What do you want reviewers to focus on? The naming choice (append vs. separator) and whether leaving pre-existing sidecars unmigrated is acceptable for an experimental harness feature.

Related Issue

Fixes #9871

[Copilot]

🟡 Changes recommended

Sidecar names can exceed filesystem filename limits for valid memory filenames, leaving partially written data and potentially breaking later index rebuilds.

1 open issue

• Bound sidecar filenames to filesystem limits · New

What changed in this PR

This PR addresses #9871 by changing Python FileMemoryProvider sidecar names so memories such as notes.md and notes no longer share a description.

Changes:

• Append _description.md to the complete memory filename.
• Add regression coverage for independent descriptions, rewrites, and deletion.
• Update documentation and clarify that legacy sidecars are not migrated.

File	Description
python/​packages/​core/​tests/​core/​test_harness_file_memory.py	Updates sidecar expectations and tests isolation between same-stem memories.
python/​packages/​core/​AGENTS.md	Documents the naming scheme and lack of legacy migration.
python/​packages/​core/​agent_framework/​_harness/​_file_memory.py	Preserves the full filename when deriving sidecar names and updates docstrings.

_{Review effort: Balanced}

[Lushiyao5]

@microsoft-github-policy-service agree

[Lushiyao5]

Addressed the Copilot review comment about sidecar filenames exceeding filesystem name limits: _description_file_name now checks the appended sidecar against the common 255-byte filename cap. Names that would overflow (e.g. 'a' * 238 + '.md' → 256 bytes) fall back to a SHA-256 digest sidecar (~memory-desc-<digest>_description.md, 96 bytes) that keeps the _description.md suffix, so file_memory_ls still hides it and reads/deletes resolve to the same sidecar. The two namespaces cannot collide: any name whose verbatim sidecar would look like a digest sidecar is far below the digest branch's length threshold.

Covered by test_description_file_name_stays_within_filesystem_name_limits (fails without the fix; full test_harness_file_memory.py suite: 41 passed).

[BlueRaddish]

The digest fallback still collides with a valid short memory filename. At head 85c87ce4974b5aa2b95c495a9185639136f32f55, take:

long_name = "a" * 238 + ".md"
short_name = "~memory-desc-" + hashlib.sha256(long_name.encode()).hexdigest()

Both names pass the provider's filename validation. The long name takes the digest branch, while the short name takes the verbatim branch, and both resolve to the same ~memory-desc-ffbd1fac2db7b2978c6e59c1424fa35068080eecd1510570b0d8223907c241bc_description.md sidecar. This requires no SHA-256 collision.

I reproduced the effect through FileMemoryProvider.before_run and its actual file_memory_write, file_memory_ls, and file_memory_delete tools with InMemoryAgentFileStore: writing the short memory changes the long memory's description from long-description to short-description; deleting the short memory then removes the long memory's description. The long memory's body and an unrelated memory's description remain intact.

Please make the digest and verbatim namespaces disjoint, including valid short names beginning with ~memory-desc-, and add a cross-branch lifecycle regression. The current test compares a long name with ~memory-desc-x, which does not select the matching digest and misses this case.

The 41 existing test_harness_file_memory.py tests pass on the same exact source. My separate public-tool consumer has two failing isolation checks and two passing controls. Environment: Windows 11, Python 3.11.17, existing development dependencies; this is an in-process store/tool check, with no live model, service, or disk-backed filesystem claim. No product source was modified. Analysis and reproduction preparation were AI-assisted.

[Eduard van Valkenburg (eavanvalkenburg)]

The digest and verbatim branches still share an output namespace. For any long filename L, a valid short memory named ~memory-desc-<sha256(L)> produces the same description sidecar, so writing, rewriting, or deleting either memory can overwrite or remove the other memory’s description without any SHA-256 collision. Please make the two namespaces disjoint—for example, also encode or hash input names that start with the reserved digest prefix—and add a regression covering a long filename and its matching digest-stem short filename.

[Lushiyao5]

Addressed in b3ec135. Thanks BlueRaddish for the concrete reproduction and Eduard van Valkenburg (@eavanvalkenburg) for the suggested fix.

Namespaces are now disjoint by construction. _description_file_name routes every name that starts with the reserved ~memory-desc- prefix through the digest branch, even when the verbatim sidecar would fit. The verbatim branch can therefore never emit a name with that prefix, while the digest branch only emits names with it — so no input pair can produce the same sidecar without a SHA-256 collision. Your exact example ("a" * 238 + ".md" vs. ~memory-desc-<sha256(...)>) now derives two different sidecars.

Regressions added:

• test_long_name_and_matching_digest_stem_short_name_have_independent_lifecycles drives the real public tools through FileMemoryProvider.before_run with InMemoryAgentFileStore: write both names, assert independent descriptions via file_memory_ls, read both bodies, delete the short memory (long description survives), and rewrite the long one. It fails on the previous derivation at the first assertion and reproduces your cross-branch overwrite/delete scenario.
• The namespace assertions in test_description_file_name_stays_within_filesystem_name_limits now cover reserved-prefix short names explicitly (including ~memory-desc-x, which the old comparison used but did not make take the digest branch).
• test_long_name_sidecar_is_writable_on_disk uses a real FileSystemAgentFileStore: over-limit names get a writable digest sidecar on disk, listings stay correct, and deleting the digest-stem short name leaves the long memory's description intact. Windows path limits reject the longest memory names themselves, so on Windows the test uses a shorter name with a proportionally lowered sidecar budget; Linux CI exercises the real 255-byte boundary.

Verification: the file-memory suite passes 43 tests; ruff check and format are clean. The only failure in the neighboring harness suites (test_filesystem_store_matches_in_memory_complexity_bound) also fails on the unmodified head and is unrelated to this change.

[Eduard van Valkenburg (eavanvalkenburg)]

Lushiyao5 The reserved digest-prefix check is case-sensitive, so uppercase or mixed-case variants can bypass it on default case-insensitive APFS and NTFS. A valid short topic named with ~MEMORY-DESC-... can then alias a long topic’s digest sidecar; writing the short topic overwrites the long topic’s description/index entry, and deleting it removes that description. Please reserve the prefix case-insensitively and add uppercase and mixed-case lifecycle regressions covering write, read, and delete behavior.

[Lushiyao5]

Addressed in 1d77069. Thanks Eduard van Valkenburg (@eavanvalkenburg) — good catch on the case-insensitive filesystems.

The reserved prefix is now matched case-insensitively. _description_file_name lowercases the input before the startswith(_DIGEST_DESCRIPTION_PREFIX) check, so ~MEMORY-DESC-... and ~Memory-Desc-... variants also take the digest branch. The verbatim branch can therefore never emit any case of the reserved prefix, keeping the two namespaces disjoint on case-insensitive APFS/NTFS.

Regressions added:

• test_case_variant_reserved_prefix_short_name_does_not_alias_long_name, parametrized over ~MEMORY-DESC- and ~Memory-Desc-: drives the real provider tools through write → list → read → delete, asserting the case-variant short memory never overwrites or removes the long memory's body/description. Both variants fail on the previous head (they alias the long name's digest sidecar) and pass now.
• Case-insensitive assertions in test_description_file_name_stays_within_filesystem_name_limits covering ~MEMORY-DESC-x and ~Memory-Desc-x.

Verification: the file-memory suite passes 45 tests; ruff check and format are clean.