[test] reorganize zero/gemini tests by ver217 · Pull Request #3445 · hpcaitech/ColossalAI

ver217 · 2023-04-04T09:58:43Z

📌 Checklist before creating the PR

I have created an issue for this PR for traceability
The title follows the standard format: [doc/gemini/tensor/...]: A concise description
I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

Closes #3437

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

Reorganize folder structure of tests related to zero/gemini.

💥 Checklist before requesting a review

I have linked my PR to an issue (instruction)
My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
I have performed a self-review of my code
I have added thorough tests.
I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

🌝 Yes, I do.
🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

github-actions · 2023-04-04T10:28:32Z

The code coverage for the changed files is 96%.

Click me to view the complete report

Name                                                             Stmts   Miss  Cover
------------------------------------------------------------------------------------
tests/test_moe/test_moe_checkpoint.py                               41      1    98%
tests/test_moe/test_moe_colo_init.py                                43      2    95%
tests/test_moe/test_moe_zero_init.py                                72      2    97%
tests/test_moe/test_moe_zero_model.py                               55      1    98%
tests/test_moe/test_moe_zero_optim.py                               85      3    96%
tests/test_zero/test_gemini/test_chunk_mgrv2.py                     52      1    98%
tests/test_zero/test_gemini/test_chunkv2.py                         90      1    99%
tests/test_zero/test_gemini/test_fwd_bwd.py                         81      1    99%
tests/test_zero/test_gemini/test_gemini_use_rmt.py                  77      1    99%
tests/test_zero/test_gemini/test_get_torch_model.py                 43      1    98%
tests/test_zero/test_gemini/test_grad_clip.py                       86      1    99%
tests/test_zero/test_gemini/test_inference.py                      103      1    99%
tests/test_zero/test_gemini/test_optim.py                          123      1    99%
tests/test_zero/test_gemini/test_runtime_mem_tracer.py              38      1    97%
tests/test_zero/test_gemini/test_search.py                          74      1    99%
tests/test_zero/test_gemini/test_zeroddp_state_dict.py              87      5    94%
tests/test_zero/test_gemini/test_zerooptim_state_dict.py            71      1    99%
tests/test_zero/test_legacy/common.py                               83     24    71%
tests/test_zero/test_legacy/test_found_inf.py                       50      1    98%
tests/test_zero/test_legacy/test_gemini_manager.py                  54      1    98%
tests/test_zero/test_legacy/test_init_context.py                    53      2    96%
tests/test_zero/test_legacy/test_param_op.py                        52      5    90%
tests/test_zero/test_legacy/test_shard_model_v2.py                  48      1    98%
tests/test_zero/test_legacy/test_shard_param.py                     67      1    99%
tests/test_zero/test_legacy/test_sharded_optim_state_dict.py        68      1    99%
tests/test_zero/test_legacy/test_sharded_optim_v2.py                82      1    99%
tests/test_zero/test_legacy/test_sharded_optim_with_sync_bn.py      46      1    98%
tests/test_zero/test_legacy/test_state_dict.py                      40      1    98%
tests/test_zero/test_legacy/test_tensor_utils.py                    67      7    90%
tests/test_zero/test_legacy/test_zero_engine.py                     73      5    93%
tests/test_zero/test_low_level/test_grad_acc.py                     94      1    99%
tests/test_zero/test_low_level/test_zero1_2.py                      93      1    99%
tests/test_zero/test_low_level/test_zero_init.py                    44      4    91%
tests/test_zero/test_low_level/test_zero_tp.py                      70      1    99%
------------------------------------------------------------------------------------
TOTAL                                                             2305     83    96%

github-actions · 2023-04-04T10:56:03Z

The code coverage for the changed files is 96%.

Click me to view the complete report

Name                                                             Stmts   Miss  Cover
------------------------------------------------------------------------------------
tests/test_moe/test_moe_checkpoint.py                               41      1    98%
tests/test_moe/test_moe_colo_init.py                                43      2    95%
tests/test_moe/test_moe_zero_init.py                                72      2    97%
tests/test_moe/test_moe_zero_model.py                               55      1    98%
tests/test_moe/test_moe_zero_optim.py                               85      3    96%
tests/test_zero/test_gemini/test_chunk_mgrv2.py                     52      1    98%
tests/test_zero/test_gemini/test_chunkv2.py                         90      1    99%
tests/test_zero/test_gemini/test_fwd_bwd.py                         81      1    99%
tests/test_zero/test_gemini/test_gemini_use_rmt.py                  77      1    99%
tests/test_zero/test_gemini/test_get_torch_model.py                 43      1    98%
tests/test_zero/test_gemini/test_grad_clip.py                       86      1    99%
tests/test_zero/test_gemini/test_inference.py                      103      1    99%
tests/test_zero/test_gemini/test_optim.py                          123      1    99%
tests/test_zero/test_gemini/test_runtime_mem_tracer.py              38      1    97%
tests/test_zero/test_gemini/test_search.py                          74      1    99%
tests/test_zero/test_gemini/test_zeroddp_state_dict.py              87      5    94%
tests/test_zero/test_gemini/test_zerooptim_state_dict.py            71      1    99%
tests/test_zero/test_legacy/common.py                               83     24    71%
tests/test_zero/test_legacy/test_found_inf.py                       50      1    98%
tests/test_zero/test_legacy/test_gemini_manager.py                  54      1    98%
tests/test_zero/test_legacy/test_init_context.py                    53      2    96%
tests/test_zero/test_legacy/test_param_op.py                        52      5    90%
tests/test_zero/test_legacy/test_shard_model_v2.py                  48      1    98%
tests/test_zero/test_legacy/test_shard_param.py                     67      1    99%
tests/test_zero/test_legacy/test_sharded_optim_state_dict.py        68      1    99%
tests/test_zero/test_legacy/test_sharded_optim_v2.py                82      1    99%
tests/test_zero/test_legacy/test_sharded_optim_with_sync_bn.py      46      1    98%
tests/test_zero/test_legacy/test_state_dict.py                      40      1    98%
tests/test_zero/test_legacy/test_tensor_utils.py                    67      7    90%
tests/test_zero/test_legacy/test_zero_engine.py                     73      5    93%
tests/test_zero/test_low_level/test_grad_acc.py                     94      1    99%
tests/test_zero/test_low_level/test_zero1_2.py                      93      1    99%
tests/test_zero/test_low_level/test_zero_init.py                    44      4    91%
tests/test_zero/test_low_level/test_zero_tp.py                      70      1    99%
------------------------------------------------------------------------------------
TOTAL                                                             2305     83    96%

github-actions · 2023-04-04T11:23:32Z

The code coverage for the changed files is 96%.

Click me to view the complete report

Name                                                             Stmts   Miss  Cover
------------------------------------------------------------------------------------
tests/test_moe/test_moe_checkpoint.py                               41      1    98%
tests/test_moe/test_moe_colo_init.py                                43      2    95%
tests/test_moe/test_moe_zero_init.py                                72      2    97%
tests/test_moe/test_moe_zero_model.py                               55      1    98%
tests/test_moe/test_moe_zero_optim.py                               85      3    96%
tests/test_zero/test_gemini/test_chunk_mgrv2.py                     52      1    98%
tests/test_zero/test_gemini/test_chunkv2.py                         90      1    99%
tests/test_zero/test_gemini/test_fwd_bwd.py                         81      1    99%
tests/test_zero/test_gemini/test_gemini_use_rmt.py                  77      1    99%
tests/test_zero/test_gemini/test_get_torch_model.py                 43      1    98%
tests/test_zero/test_gemini/test_grad_clip.py                       86      1    99%
tests/test_zero/test_gemini/test_inference.py                      103      1    99%
tests/test_zero/test_gemini/test_optim.py                          123      1    99%
tests/test_zero/test_gemini/test_runtime_mem_tracer.py              38      1    97%
tests/test_zero/test_gemini/test_search.py                          74      1    99%
tests/test_zero/test_gemini/test_zeroddp_state_dict.py              87      5    94%
tests/test_zero/test_gemini/test_zerooptim_state_dict.py            71      1    99%
tests/test_zero/test_legacy/common.py                               83     24    71%
tests/test_zero/test_legacy/test_found_inf.py                       50      1    98%
tests/test_zero/test_legacy/test_gemini_manager.py                  54      1    98%
tests/test_zero/test_legacy/test_init_context.py                    53      2    96%
tests/test_zero/test_legacy/test_param_op.py                        52      5    90%
tests/test_zero/test_legacy/test_shard_model_v2.py                  48      1    98%
tests/test_zero/test_legacy/test_shard_param.py                     67      1    99%
tests/test_zero/test_legacy/test_sharded_optim_state_dict.py        68      1    99%
tests/test_zero/test_legacy/test_sharded_optim_v2.py                82      1    99%
tests/test_zero/test_legacy/test_sharded_optim_with_sync_bn.py      46      1    98%
tests/test_zero/test_legacy/test_state_dict.py                      40      1    98%
tests/test_zero/test_legacy/test_tensor_utils.py                    67      7    90%
tests/test_zero/test_legacy/test_zero_engine.py                     73      5    93%
tests/test_zero/test_low_level/test_grad_acc.py                     94      1    99%
tests/test_zero/test_low_level/test_zero1_2.py                      93      1    99%
tests/test_zero/test_low_level/test_zero_init.py                    44      4    91%
tests/test_zero/test_low_level/test_zero_tp.py                      70      1    99%
------------------------------------------------------------------------------------
TOTAL                                                             2305     83    96%

[test] reorganize zero/gemini tests

1f396ee

ver217 added Run Build and Test module: zero/gemini ZeRO (LowLevelZero) and Gemini: sharding, offload, chunk management labels Apr 4, 2023

ver217 requested a review from FrankLeeeee April 4, 2023 09:58

ver217 self-assigned this Apr 4, 2023

FrankLeeeee approved these changes Apr 4, 2023

View reviewed changes

FrankLeeeee merged commit 933048a into hpcaitech:main Apr 6, 2023

ver217 deleted the refactor/zero-test branch April 10, 2023 07:35

JazyJiang added the module: ci CI, GitHub Actions, test infrastructure and the tests themselves label Oct 26, 2026

Navigation Menu

Repository navigation

[test] reorganize zero/gemini tests - #3445

[test] reorganize zero/gemini tests#3445
FrankLeeeee merged 1 commit into
hpcaitech:mainfrom
ver217:refactor/zero-test

ver217 commented Apr 4, 2023

Uh oh!

github-actions Bot commented Apr 4, 2023

Uh oh!

github-actions Bot commented Apr 4, 2023

Uh oh!

github-actions Bot commented Apr 4, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ver217 commented Apr 4, 2023

📌 Checklist before creating the PR

🚨 Issue number

📝 What does this PR do?

💥 Checklist before requesting a review

⭐️ Do you enjoy contributing to Colossal-AI?

Uh oh!

github-actions Bot commented Apr 4, 2023

Uh oh!

github-actions Bot commented Apr 4, 2023

Uh oh!

github-actions Bot commented Apr 4, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants