Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add warning when the model has eval mode parameters #20573

Open
h-munakata opened this issue Feb 4, 2025 · 0 comments
Open

Add warning when the model has eval mode parameters #20573

h-munakata opened this issue Feb 4, 2025 · 0 comments
Labels
needs triage Waiting to be triaged by maintainers refactor

Comments

@h-munakata
Copy link

h-munakata commented Feb 4, 2025

Outline & Motivation

After the update for 2.2.0, parameters that are in eval mode before the training are not changed to train mode in the training loop.

https://github.com/Lightning-AI/pytorch-lightning/releases/tag/2.2.0
In some libraries, for example HuggingFace, models are created in evaluation mode by default (e.g. HFModel.from_pretrained(...)). Starting from 2.2, you will have to set .train() on these models if you intend to train them.
This update is effective for unfortunate side effects.

However, I think this feature can often be overlooked in many cases because users (including me) would expect the train/eval mode to be automatically changed in the loop.

Pitch

To avoid this problem, it would be effective to display a warning before training that the model parameters are in eval mode.

Additional context

No response

cc @lantiga @justusschock

@h-munakata h-munakata added needs triage Waiting to be triaged by maintainers refactor labels Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs triage Waiting to be triaged by maintainers refactor
Projects
None yet
Development

No branches or pull requests

1 participant