Restormer Implementation

[phisanti]

Fixes # .

Description

This PR implements the Restormer architecture for high-resolution image restoration in MONAI following the discussion in issue #8261. The implementation supports both 2D and 3D images using MONAI's convolution as the base. Key additions include:

• Downsample class for efficient downsampling operations
• pixel_unshuffle operation complementing existing pixel_shuffle
• Channel Attention Block (CABlock) with FeedForward layer
• Multi-DConv Head Transposed Self-Attention (MDTA)
• OverlapPatchEmbed class
• Comprehensive unit tests for all new components

The implementation follows MONAI's coding patterns and includes performance validations against native PyTorch operations where applicable.

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[ericspod]

Looks good overall but I had a few inline comments, and we should have full docstrings everywhere appropriate. For any classes meant for general purpose use (ie. not just by Restormer) please ensure they have docstring descriptions for the arguments (at the very least for constructor args). Thanks!

[ericspod]

Hi @phisanti thanks for bearing with us here but we've updated the tests directory so there's a conflict you'll need to resolve. We've introduced a subdirectory structure within tests so please move your test files to somewhere appropriate within that. Also it appears that a number of changes you've made to address comments have been reverted back, I think somewhere in your commits there's been some mixup of versions.

[phisanti]

Hi @ericspod, no probs. Thanks for the feedback. I will move the test to the appropriate new folder this Friday. Also, I would like to ask for a bit of help if possible. It seems some of the tests cannot be running due to the optional import lineups. For example, it happens in premerge-min/min-dep-py3 (3.12) but not in premerge/quick-py3 (ubuntu-latest). I tried to place the skip test on top, but maybe you can give me some indication of where it should go.

Regarding the mixup on the changes, I think it happened when I did a merge back on Feb 7th as the branch got behind, I will try to look on it.

[ericspod]

Hi @phisanti We can look at the issues with the tests, no problem. I think I should let you figure out the mixup with branches and update your tests first then we can have a look at what the issue is. We should avoid just ignoring tests as much as possible, but if you need einops for example you can use @skipUnless(has_einops, "Requires einops") as a test method decorator. If that isn't working for you we can figure it out together.

[phisanti]

Hi @phisanti We can look at the issues with the tests, no problem. I think I should let you figure out the mixup with branches and update your tests first then we can have a look at what the issue is. We should avoid just ignoring tests as much as possible, but if you need einops for example you can use @skipUnless(has_einops, "Requires einops") as a test method decorator. If that isn't working for you we can figure it out together.

Hi @ericspod, I finally manage to find some time to do the merge and be up-to-date with main. Now, in regards with the unit test, I placed them all in their appropriate folders mirroring the structure within source:

/tests/networks/utils/text_pixelshuffle.py
/tests/networks/blocks/text_CABlock.py
/tests/networks/blocks/text_downsample.py
/tests/networks/nets/text_restormer.py

I also noticed that I have to move the import of SkipIfBeforePyTorchVersion in test_CABlock.py from tests.utils to tests.test_utils. I have placed the statement einops, has_einops = optional_import("einops") plus @skipUnless(has_einops, "Requires einops"), however, it seems some of the checks still fail. Thus, any help will be welcome, I believe the required changes are quite minimal.