Skip to content

Re-enable once pure_nnx config issue for DeepseekV4 - #4808

Open
dipakg-lang wants to merge 1 commit into
AI-Hypercomputer:mainfrom
dipakg-lang:fix_tests_for_dsv4_bias
Open

Re-enable once pure_nnx config issue for DeepseekV4#4808
dipakg-lang wants to merge 1 commit into
AI-Hypercomputer:mainfrom
dipakg-lang:fix_tests_for_dsv4_bias

Conversation

@dipakg-lang

Copy link
Copy Markdown
Collaborator

Description

Start with a short description of what the PR does and how this is a change from
the past.

https://b.corp.google.com/issues/544531325

If the change fixes a bug or a Github issue, please include a link, e.g.,:
FIXES: b/544531325

Notice 1: Once all tests pass, the "pull ready" label will automatically be assigned.
This label is used for administrative purposes. Please do not add it manually.

Notice 2: For external contributions, our settings currently require an approval from a MaxText maintainer to trigger CI tests.

Tests

Please describe how you tested this change, and include any instructions and/or
commands to reproduce.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request re-enables the deepseek4-284b model integration test by setting the pure_nnx flag for this model and updating the _DEEPSEEK4_MLP_SCANNED dictionary. Feedback suggests consolidating _DEEPSEEK4_MLP_SCANNED and _DEEPSEEK4_MLP since they are now identical, and using a set instead of a direct string comparison for checking is_pure_nnx to improve extensibility.

"bias": None,
"kernel": mdn(reduction_axis=(0,), output_axis=(-1,)),
},
"gate": {"kernel": mdn(reduction_axis=(0,), output_axis=(-1,))},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

With the removal of "bias": None from "gate", _DEEPSEEK4_MLP_SCANNED is now completely identical to _DEEPSEEK4_MLP (defined on line 249). To improve maintainability and reduce code duplication, consider consolidating them in a future refactor (e.g., by pointing _DEEPSEEK4_MLP_SCANNED = _DEEPSEEK4_MLP or removing one of them entirely).

Comment on lines +414 to +415
is_pure_nnx = model_name == "deepseek4-284b"
actual_output = muon_utils.get_model_mdn(model_name, scan_layers=True, pure_nnx=is_pure_nnx)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Hardcoding the model name check model_name == "deepseek4-284b" is fragile and less extensible if other models are migrated to pure_nnx in the future. Using a set of models or a prefix check makes it cleaner and easier to maintain.

Suggested change
is_pure_nnx = model_name == "deepseek4-284b"
actual_output = muon_utils.get_model_mdn(model_name, scan_layers=True, pure_nnx=is_pure_nnx)
is_pure_nnx = model_name in {"deepseek4-284b"}
actual_output = muon_utils.get_model_mdn(model_name, scan_layers=True, pure_nnx=is_pure_nnx)

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@dipakg-lang
dipakg-lang force-pushed the fix_tests_for_dsv4_bias branch from 517ff37 to 67dbeaf Compare August 10, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant