You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After the update for 2.2.0, parameters that are in eval mode before the training are not changed to train mode in the training loop.
https://github.com/Lightning-AI/pytorch-lightning/releases/tag/2.2.0
In some libraries, for example HuggingFace, models are created in evaluation mode by default (e.g. HFModel.from_pretrained(...)). Starting from 2.2, you will have to set .train() on these models if you intend to train them.
This update is effective for unfortunate side effects.
However, I think this feature can often be overlooked in many cases because users (including me) would expect the train/eval mode to be automatically changed in the loop.
Pitch
To avoid this problem, it would be effective to display a warning before training that the model parameters are in eval mode.
Outline & Motivation
After the update for 2.2.0, parameters that are in eval mode before the training are not changed to train mode in the training loop.
However, I think this feature can often be overlooked in many cases because users (including me) would expect the train/eval mode to be automatically changed in the loop.
Pitch
To avoid this problem, it would be effective to display a warning before training that the model parameters are in eval mode.
Additional context
No response
cc @lantiga @justusschock
The text was updated successfully, but these errors were encountered: