Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken pytorch version constraint #31

Closed
wants to merge 2 commits into from
Closed

Conversation

yit-b
Copy link

@yit-b yit-b commented May 6, 2024

When trying to install mmcv from conda-forge alongside a precise version of pytorch, the constraint on pytorch's build string fails erroneously due to a missing leading asterisk in the run requirements/constraints. Installation combinations that should be allowed are blocked.

Example repro using micromamba (also repros with bare conda, but it's so slow):

micromamba env create -n foobar
conda activate /home/$USER/micromamba/envs/foobar
CONDA_OVERRIDE_CUDA=12.1 \
    micromamba/bin/micromamba install \
    -c conda-forge \
    -c pytorch  \
    -c nvidia \
    pytorch=2.1.0=py3.10_cuda12.1_cudnn8.9.2_0 \
    mmcv=*=*cuda*

(Side note: CONDA_OVERRIDE_CUDA is required since the meta.yaml places a requirement that the host has a system cuda installation available, which is not necessary, but that's a different issue.)

Result:

error    libmamba Could not solve for environment specs
    The following packages are incompatible
    ├─ mmcv * *cuda* is installable and it requires
    │  └─ pytorch * cuda*, which can be installed;
    └─ pytorch 2.1.0 py3.10_cuda12.1_cudnn8.9.2_0 is not installable because it conflicts with any installable versions previously reported.

There is a constraint that the build begins with the string literal "cuda" (with nothing before it). However, pytorch builds are prefixed with the python version, followed by the cuda version. So the constraint should be pytorch * *cuda*.

I encountered this exact issue with pytorch3d's feedstock repo and applied the same fix in this PR:
conda-forge/pytorch3d-feedstock#9

This issue may be rather common among other feedstock repos.

@yit-b yit-b requested review from apatsekin and weiji14 as code owners May 6, 2024 14:10
@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@weiji14
Copy link
Member

weiji14 commented May 6, 2024

@yit-b, can you provide the output of mamba info (want to see your channel_urls)? I think you're trying to install pytorch from the pytorch channel at https://anaconda.org/pytorch/pytorch/files which has build numbers like py3.10_cuda12.1_cudnn8.9.2_0, whereas the conda-forge channel at https://anaconda.org/conda-forge/pytorch/files uses build numbers like cuda120_py310h327d3bc_303.

For context, I actually did add the asterisk before at #16, but decided to revert it in 0f05997/#17 because of an ImportError (see #16 (comment)). The gist is that we shouldn't be installing mmcv from conda-forge with pytorch from the pytorch channel, as it can result in binary incompatibilities (channel mixing is discouraged in conda-forge), so the missing asterisk in the pytorch ={{ pytorch }}={{ torch_proc_type }}* version constraint is actually deliberate to only allow conda-forge's pytorch.

@weiji14
Copy link
Member

weiji14 commented May 22, 2024

Heads up that I'll close this PR once the Pytorch 2.3 build PR is merged at #33, as the original motivation of this PR was probably to get mmcv working with a newer version of pytorch>=2.2 (conda-forge/pytorch3d-feedstock#10) if I'm not mistaken.

@weiji14 weiji14 closed this in #33 May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants