Add a release engine for npm

[ezekg]

This is part of our overarching goal of supporting more package managers in Keygen: #493.

Implement similar to #478. I think I’d prefer a node or npm subdomain. See #679 for implementation of PyPI, and #753 for
Tauri. The Tauri one will likely be the most helpful for reference, since the PyPI included a lot of noisy ground work.

Helpful resources:

• https://keygen.sh/blog/how-to-license-and-distribute-commercial-node-modules/
• https://github.com/keygen-sh/example-private-npm-package

This engine is meant to replace the workflow in that blog post — it should be a turn-key way to distribute modules.

In the end, I want a customer to be able to upload a module (e.g. via npm pack) to the distribution API we already have, and allow their end-users to npm install that module like they would from https://npmjs.com.

Implementation should include integration tests. lmk of any questions.

[ezekg]

What makes this hard is that npm's abbreviated package manifest requires dependency data, and we don't have artifact blobs to pull that data from (blobs are uploaded to the storage provider after the redirect). And I don't want to add one-off functionality to the API to treat package.json artifacts as "special."

The simplest route here would be to require additional fields like dependencies, devDependencies, peerDependencies, bin, etc. be supplied in the package.json artifact's metadata. We can add a baseline per the required properties like name, version, dist, while pulling the rest from metadata. The time field also looks to be used by Unity's package manager (which seems based on the npm spec), so we should include that too.

This approach aligns with what we do for PyPI artifact metadata as well, e.g. data-requires-python.

Publishing via the CLI (see also: keygen-sh/keygen-cli#13):

keygen upload package.json --release 1.0.0 --metadata "$(cat package.json |  jq 'del(.name, .version)')"

Every registry has their own quirks. 🤪

[ezekg]

/bounty $500

[algora-pbc]

💎 $500 bounty • Keygen

Steps to solve:

1. Start working: Comment /attempt #489 with your implementation plan
2. Submit work: Create a pull request including /claim #489 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to keygen-sh/keygen-api!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @varshith257	Sep 12, 2024, 3:32:48 PM	WIP

[varshith257]

An interested issue to tackle. I will go ahead with it
/attempt #489

Algora profile	Completed bounties	Tech	Active attempts	Options
@varshith257	11 bounties from 5 projects	TypeScript, Go		Cancel attempt

[rajesh-jonnalagadda]

@varshith257 I would like to collaborate with you. If you're interested, please let me know how I can assist.

[varshith257]

I started working on it and all set to go of initial work. I will connect you on discord if any

[ezekg]

Thought it may be helpful to see how others do it: Gitea and GitLab.

[varshith257]

Will draft PR soon with taking help of references attached

[ezekg]

What makes this hard is that npm's abbreviated package manifest requires dependency data, and we don't have artifact blobs to pull that data from (blobs are uploaded to the storage provider after the redirect). And I don't want to add one-off functionality to the API to treat package.json artifacts as "special."

The simplest route here would be to require additional fields like dependencies, devDependencies, peerDependencies, bin, etc. be supplied in the package.json artifact's metadata. We can add a baseline per the required properties like name, version, dist, while pulling the rest from metadata. The time field also looks to be used by Unity's package manager (which seems based on the npm spec), so we should include that too.

This approach aligns with what we do for PyPI artifact metadata as well, e.g. data-requires-python.

Publishing via the CLI (see also: keygen-sh/keygen-cli#13):

keygen upload package.json --release 1.0.0 --metadata "$(cat package.json |  jq 'del(.name, .version)')"

Every registry has their own quirks. 🤪

I actually think I want to go a different route here, since Rubygems looks like it similarly needs dependency 'metadata' too, and we can't just throw that into release metadata like we can with npm: #896 (review)

The idea is that we set up a processing job pipeline for release 'metadata' artifacts, such as an npm package.json or a Rubygems gemspec. Ultimately, this would allow us to store release metadata locally in a release_metadata table, allowing deeper integrations with package managers, while reducing reliance on S3/R2 for obtaining said metadata.

This approach also avoids weird quirks which come from shoehorning resource metadata to fit this purpose like I originally proposed above (and like we currently do with PyPI via the data- attributes ). A proper pipeline will allow us to expand into other package managers, regardless of metadata format.