Add utility for Reload Transformers imports cache for development workflow #35508

[sambhavnoobcoder]

Problem Statement

When developing or modifying Transformers models, cached imports can prevent code changes from being picked up without manually restarting Python. This makes the development workflow more cumbersome, especially when making iterative changes to model implementations.

Fixes #35508

Root Cause Analysis

The issue stems from Python's module caching behavior in sys.modules. Additionally, Transformers uses a custom _LazyModule system for imports that maintains its own object cache. Both these caches need to be cleared to fully reload modified code.

Implementation

Added a clear_import_cache() utility function to transformers.utils.import_utils that:

1. Identifies all Transformers modules in sys.modules
2. Clears internal caches of _LazyModule instances
3. Removes modules from sys.modules
4. Forces reload of the main Transformers module

The implementation is minimally invasive and works with Transformers' existing module system.

Testing

Added a test case that verifies:

1. Initial module imports work correctly
2. Cache clearing removes modules from sys.modules
3. Modules can be reimported after clearing
4. The lazy loading system continues to function properly

The test ensures the cache clearing is thorough while maintaining backward compatibility.

Screenshots

cc : @ArthurZucker @SunMarc @Rocketknight1 and anyone interested in development workflow improvements.