Release 20.8b1 Missing Binary

[midnightconman]

Describe the bug A clear and concise description of what the bug is.

The most recent release only contains source code packages, no binaries. https://github.com/psf/black/releases/tag/20.8b1
To Reproduce Steps to reproduce the behavior:

1. Visit (https://github.com/psf/black/releases/tag/20.8b1) and notice there is no binary available for download

Expected behavior A clear and concise description of what you expected to happen.
I expected a binary to be available, like previous releases.

Environment (please complete the following information):

N/A

Does this bug also happen on master? To answer this, you have two options:

N/A

Additional context Add any other context about the problem here.

[midnightconman]

20.8b0 is also missing a release binary: https://github.com/psf/black/releases/tag/20.8b0

[hugovk]

You can find sources and wheels on PyPI, for at least both of those releases:

• https://pypi.org/project/black/20.8b1/#files
• https://pypi.org/project/black/20.8b0/#files

PyPI is the usual place to distribute Python packages, rather than GitHub Releases.

[midnightconman]

It was really nice to have the self enclosed binaries... not requiring a python environment really helped using this in CI.

[hugovk]

Ah, do you mean like the black file at https://github.com/psf/black/releases/tag/19.10b0?

[JelleZijlstra]

I'm not sure where that file comes from, @ambv did you do anything to generate it? It's an ELF file that works on Linux but I wasn't aware that we had anything in this repo that would build such a file.

[johnthagen]

Perhaps built with Pyinstaller or similar?

[matthewfeickert]

You can find sources and wheels on PyPI, for at least both of those releases:

• https://pypi.org/project/black/20.8b1/#files
• https://pypi.org/project/black/20.8b0/#files

PyPI is the usual place to distribute Python packages, rather than GitHub Releases.

@hugovk Maybe they've been removed (for reasons that I don't know about) since you posted this, but neither v20.8b0 or v20.8b1 have wheels on PyPI. Though v19.10b0 does.

cc @kratsg

[cooperlees]

They were removed due to a build error causing them to be out of date. We're hoping to have that fixed for 20.9 release coming the end of this month.

[matthewfeickert]

They were removed due to a build error causing them to be out of date. We're hoping to have that fixed for 20.9 release coming the end of this month.

Great! Thanks very much for the helpful and fast reply. 🙇

[ichard26]

@JelleZijlstra, #362 asked for binary releases so Black could be run conveniently and without needing CPython 3.6+ set up. Linux and Windows binaries started to be automatically created and uploaded after their respective PRs were merged, #410 and #422.

20.8b0 and 20.8b1 don't have the Linux binary because the refactor to packages broke the Travis config. I have no idea whether any Windows binaries were successfully created and uploaded but regardless, we stopped used AppVeyor some time back.

I personally want to see platform specific binaries continue to be available as it enables more people to use Black without much cost. They were already created and uploaded automatically. Maybe this time we should use GitHub Actions instead of Travis CI + AppVeyor. For one, we could start providing binaries for MacOS! (er, Travis CI has support for Mac OS)

[matthewfeickert]

I personally want to see platform specific binaries continue to be available as it enables more people to use Black without much cost. They were already created and uploaded automatically. Maybe this time we should use GitHub Actions instead of Travis CI + AppVeyor. For one, we could start providing binaries for MacOS!

@ichard26 I might point you to the very nice work that @henryiii has done for Scikit-HEP to get this working very nicely with cibuildwheel. He even has developer instructions on the Scikit-HEP website for doing this all with GitHub Actions. You can see this working super nicely for boost-histogram.

Please let us know if you have any questions!

[henryiii]

I think this could easily be a "universal" wheel (name may be slightly out of date now that it's Python 3 only), I don't think it needs any "true" binaries. There is a MyPyC extension, which could probably be built using cibuildwheel, though you could also provide a pure Python wheel too, as a fallback. I'm not sure how important / useful that is (or really what it is).

(Unless there really are compiled components that I didn't notice on quick glance)

[hugovk]

I'd also recommend deploying to TestPyPI on (say) merges to master, that way the deploy mechanism is also regularly tested so there's no surprises on release day.

[henryiii]

@matthewfeickert, I think @ichard26 is referring to self contained, binary packages, while you and several of us are referring to wheels on PyPI.

The developer instructions discuss downloading artifacts from any CI build, including PRs, so release day shouldn't be a surprise. :)

[ichard26]

@matthewfeickert thanks for the information! I'm a bit lost though as cibuildwheel seems to builds .whl files (which are not binary files, just a ZIP archive that may contain compiled extensions) not actual OS native binary executables.

edit: oh thanks @henryiii, we posted at the same time :)

[henryiii]

Level 0: SDist
Level 1: Wheels
Level 10: Binary self contained executables ;)

We are currently missing level 1 (and also level 10, I gather)

[ichard26]

We already provide Pure Python wheels, but a mistake was made during build so the wheels were yanked. @ cooperlees already addressed this:

They were removed due to a build error causing them to be out of date. We're hoping to have that fixed for 20.9 release coming the end of this month.

As said already, the next release should have PEP 427 binaries unless another mistake is made :)

[hugovk]

The developer instructions discuss downloading artifacts from any CI build, including PRs, so release day shouldn't be a surprise. :)

Ah yes, that's really handy for testing that the artifacts are generated properly. I still suggest uploading to TestPyPI to keep the upload mechanism itself well oiled :)

[pradyunsg]

https://pypi.org/project/black/#files -- there still isn't a binary for the latest release.

[ichard26]

@pradyunsg thanks for the reminder but this issue is for self-contained binaries that don't need a Python installation to run.

[pradyunsg]

Right, I understand that; except that isn't what the OP is about here.

Expected behavior A clear and concise description of what you expected to happen.
I expected a binary to be available, like previous releases.

[johnthagen]

Given how popular black is, it would make a lot of CI jobs more efficient if a wheel was published so it wasn't rebuilt every time.

For details see: https://pythonwheels.com/

[pradyunsg]

While having independent binaries is great, not having a wheel uploaded on PyPI is not.

My understanding was that this issue is tracking the re-upload of a wheel, or a new release with a wheel. I'm not sure why this was closed because independent binaries are built in CI now.

[ichard26]

Honestly I don't even understand, I'm confused at this point :/

• The OP (midnightconman) said "I expected a binary to be available, like previous releases" as noted in their initial report
• Then hugovk mentioned how 20.8b0 and 20.8b1 had wheels on PyPI (no longer true but I digress, that's another rant for another time)
• After that, the OP added this comment: "It was really nice to have the self enclosed binaries... not requiring a python environment really helped using this in CI."
• Hugo realized that the binary the OP might have been talking about is the one from like here so they asked for clarification...
  • the OP +1ed that guess

Yes, there are comments about wheels here, like #1669 (comment), but those ones aren't from the OP.

I'm very sorry if I'm the one being dumb here and completely misreading the situation (totally possible!) but I don't believe wheels are what the OP was talking about.

Also, regardless of that, I'd like to share my appreciation for the consistent reminders for us to publish a release with wheels. I share and understand your frustration.

Finally, I'm just going to open a separate issue asking for a release with wheels.

[pradyunsg]

FWIW, the wheels were uploaded and deleted subsequently.

I'd post a link, but I'm on my mobile and Github search isn't surfacing the discussion that led to the deletion of the wheel. That said, I think making a new black release would be a good idea overall, and adopting a regular release cadence (like pip, especially given that you also follow calver) would be great!

[ichard26]

Finally, I'm just going to open a separate issue asking for a release with wheels.

done see GH-2125

discussion that led to the deletion of the wheel

The discussion was via private channels among the core team but I can post the snippets relevant (although I should probably ask for permission from them, but I'm ambivalent about doing so due to some mixed feelings :/)

IIRC it boiled down to this:

• I was doing some unrelated debugging and discovered the wheels had outdated files (both a black module and black package were present in the distribution)
• I let the core team know and after confirmation, it was decided that deleting the wheel would be the best course of option

Although I'm not entirely sure what the deletion was all about since apparently Python favours packages over modules in case of a conflict (which is good since only the module version of black in the wheels were outdated).