Fix DMRL save() failing when log_metrics=True - #724
Merged
qtuantruong merged 2 commits intoSep 13, 2026
Merged
Conversation
DMRL created a tensorboard SummaryWriter in __init__ and kept it as self.tb_writer. Recommender.save() deep-copies the model, and the writer holds thread locks and an open event file, so saving raised "TypeError: cannot pickle '_thread.lock' object" (PreferredAI#656). Constructing the model also created temp/tb_data/run_1 even if it was never trained. Create the writer locally in _fit_dmrl, where all logging happens, and close it when training finishes. Add a save/load test that is skipped when the DMRL requirements or tensorboard are missing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
qtuantruong
requested changes
Sep 13, 2026
qtuantruong
left a comment
Member
There was a problem hiding this comment.
The fix looks good to me. Though we should remove the test file from this PR. Thanks
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
Thanks for the review! Done. I pushed a commit that removes I updated the description. The fix is still verified by a local script (fit with |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
DMRL(log_metrics=True)cannot be saved, as reported in #656.DMRL.__init__created a tensorboardSummaryWriterand stored it asself.tb_writer.Recommender.save()callscopy.deepcopy(self), andRecommender.__deepcopy__deep-copies every attribute exceptignored_attrs. The writer holds thread locks and an open event file, so saving fails:TypeError: cannot pickle '_thread.lock' object(plain PyTorch/tensorboard)cannot pickle 'tensorflow.python.lib.io._pywrap_file_io.WritableFile', the variant reported in [BUG] i am not able to do save dmrl.save() in Disentangled Multimodal Representation Learning for Recommendation (DMRL) #656 (on Google Colab, where TensorFlow is installed; I only reproduced the plain PyTorch/tensorboard error above)A side effect: just constructing
DMRL(log_metrics=True)createdtemp/tb_data/run_1in the working directory, even if the model was never trained.The writer is only used inside
_fit_dmrl. This PR creates it there as a local variable, right before logging the hyperparameters, and closes it when training finishes. The log directory and the logged scalars/histograms are unchanged.I considered adding
tb_writertoignored_attrs. It would fixsave(), but a loaded model trained again withlog_metrics=Truewould then have no writer, and__init__would still create the directory.How I tested
Per review, the save/load test that was in the first version of this PR has been removed, so the PR now only changes
cornac/models/dmrl/recom_dmrl.py.I verified the fix with a local script (Windows, Python 3.12, torch 2.14.0+cpu, sentence-transformers 6.0.1, cornac built from source). It trains a tiny DMRL on random feedback plus pre-encoded image features, with
log_metrics=Trueand no model download, then callssave()andRecommender.load():mastertemp/created byDMRL(log_metrics=True)beforefitfit)save()afterfitTypeError: cannot pickle '_thread.lock' objectRecommender.load()pytest tests/cornac/modelsThe 3 failures are the
tests/cornac/models/dmrl/test_transformertext.pytests. They need to download a Hugging Face model and I ran offline, so they fail the same way onmaster.masterflake8 . --select=E9,F63,F7,F82(as in CI)Related Issues
Fixes #656
Checklist:
README.md(if you are adding a new model). (N/A)examples/README.md(if you are adding a new example). (N/A)datasets/README.md(if you are adding a new dataset). (N/A)🤖 Generated with Claude Code