Help with adding support for pytorch 2.2.0

[yit-b]

Comment:

Version 0.7.6 of pytorch3d supports pytorch version 2.2.0
https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.6

But the feedstock is still producing builds depending on 2.1. Seems like there's still a build-time requirement of pytorch 2.1, as seen in the conda-smithy-generated build configuration files, e.g.:
https://github.com/conda-forge/pytorch3d-feedstock/blob/main/.ci_support/linux_64_c_compiler_version12cuda_compilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12numpy1.23python3.11.____cpython.yaml#L35

The README in .ci_support says to go change recipe/conda_build_config.yaml, which I assume is just an outdated reference to recipe/meta.yaml, but I'm not sure where conda-smithy is picking up the version 2.1 from.

Is it just solving for 2.1 on this line here?
https://github.com/conda-forge/pytorch3d-feedstock/blob/main/recipe/meta.yaml#L55

What changes need to be made to build for pytorch 2.2? Do additional build variations need to be added for different versions of pytorch, a la python?

For reference, here's the output of a conda search for a build produced today:

pytorch3d 0.7.6 cuda120py311h2bb5123_2
--------------------------------------
file name   : pytorch3d-0.7.6-cuda120py311h2bb5123_2.conda
name        : pytorch3d
version     : 0.7.6
build       : cuda120py311h2bb5123_2
build number: 2
size        : 14.4 MB
license     : BSD-3-Clause
subdir      : linux-64
url         : https://conda.anaconda.org/conda-forge/linux-64/pytorch3d-0.7.6-cuda120py311h2bb5123_2.conda
md5         : a9da06b730689ad918d0f40a042bd92b
timestamp   : 2024-05-01 05:20:14 UTC
constraints :
  - pytorch =*=*cuda*
dependencies:
  - __glibc >=2.17,<3.0.a0
  - cuda-cudart
  - cuda-version >=12.0,<13
  - cudnn >=8.9.7.29,<9.0a0
  - fvcore
  - iopath
  - libcublas >=12.0.1.189,<13.0a0
  - libcusolver >=11.4.2.57,<12.0a0
  - libcusparse >=12.0.0.76,<13.0a0
  - libgcc-ng >=12
  - libstdcxx-ng >=12
  - numpy >=1.23.5,<2.0a0
  - python >=3.11,<3.12.0a0
  - python_abi 3.11.* *_cp311
  - pytorch >=2.1.2,<2.2.0a0
  - torchvision

The <2.2.0a0 constraint on pytorch is unnecessary as of version 0.7.6. Seems this can be bumped to <2.3.0a0?

Thank you!

[Tobias-Fischer]

We need to bump the global pin for PyTorch - see https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/2afb0c7d4a59c885bfb68cf0c13c93675090e960/recipe/conda_build_config.yaml#L609-L611 and conda-forge/conda-forge-pinning-feedstock#5412

Once that's bumped, the bot will automatically migrate pytorch3d to the new PyTorch version.

/cc @hmaarrfk

[hmaarrfk]

There are no 2.2.0 build of pytorch on conda-forge

There are a few PRs aimed at improving the pytorch recipe along different axes. None have made a substantial enough improvement to warrant merging, but perhaps you can combine a few together into one.

[yit-b]

Maybe noob question: Is there a specific reason to rely on conda-forge::pytorch and not the seemingly more widely used pytorch::pytorch?

[weiji14]

Recommendation is to not mix depedencies from different channels, e.g. installing pytorch::pytorch with conda-forge::pytorch3d can lead to incompatibilities. You'll likely run into weird undefined symbol errors when doing channel mixing. For more info, see e.g.:

• https://conda-forge.org/docs/user/tipsandtricks/#using-multiple-channels
• Packaging difference conda-forge's and pytorch's distribution of pytorch pytorch-cpu-feedstock#218
• https://stackoverflow.com/questions/62526717/error-package-or-namespace-load-failed-for-rjags/62566835#62566835
• Fix compatibility with pytorch from pytorch channel mmcv-recipe-feedstock#16 (comment)

And yes, it can be frustrating to not be able to use the latest Pytorch 2.2 version. It usually takes maybe 3-4 months (if not longer) after the official upstream release for conda-forge to package things properly (see ongoing effort at conda-forge/pytorch-cpu-feedstock#221), depending on maintainer availability. There's just a lot of moving pieces to get all the packages to line up properly and be compatible with each other.

[yit-b]

@weiji14 Ah, I understand. Thank you for the in-depth explanation!

[weiji14]

FYI, pytorch3d on conda-forge has a Pytorch 2.3 build now (#12). I'll close this issue, but feel free to reopen or continue the discussion if needed.

[hmaarrfk]

Thanks for your patience all