p5.js 2.0 Compatibility

[limzykenneth]

Hi @ziyuan-linn @shiffman @lindapaiste, we had briefly gone through the potential changes to preload to use promises in p5.js 2.0 before (@shiffman you have reviewed it in the proposal list as well) and we have now released p5.js 2.0 betas that uses an async setup function instead of the preload function.

I went ahead and tested an example from every model and find they generally don't need major changes as your existing implementation already uses promises anyway and preload is more of a patch for you. I've created a collection for you to checkout what the updated examples look like (except for SoundClassifier which I couldn't get to work in the regular version), but here is the ImageClassifier one as an example:

// Initialize the Image Classifier method with MobileNet. A callback needs to be passed.
let classifier;

// A variable to hold the image we want to classify
let img;

// Variables for displaying the results on the canvas
let label = "";
let confidence = "";

async function setup() {
  img = await loadImage("images/bird.jpg");
  classifier = ml5.imageClassifier("MobileNet");
  createCanvas(400, 400);
  let results = await classifier.classify(img);
  
  // The results are in an array ordered by confidence
  console.log(results);

  image(img, 0, 0, width, height);
  // Display the results on the canvas
  fill(255);
  stroke(0);
  textSize(18);
  label = "Label: " + results[0].label;
  confidence = "Confidence: " + nf(results[0].confidence, 0, 2);
  text(label, 10, 360);
  text(confidence, 10, 380);
}

There are a few considerations for ml5 and I'm very happy to help work through them however you wish.

---

In terms of transition, we are happy to work with whatever plan you'd like and based on p5.js' timeline we have a few suggestion that we are putting forward for addon libraries. The rough timeline is described here: processing/p5.js#7488 but essentially the p5.js web editor will still default to using 1.x, after the planned end of March 2025 release of p5.js 2.0, until August 2026 to provide ample time for transition and to reduce mid academic year disruption. Your release timeline don't need to match ours so this is just for reference.

For addon libraries, I see three options.

1. Release new version that only support p5.js 2.0 async/await while user needing preload function use the previous released version. This is potentially quite disruptive for your user.
2. Release parallel versions until August 2026, one supporting preload function, another supporting just promises.
3. Release version that is compatible with both promises and preload. In a way this is where ml5.js is at so I would recommend this, however there are a few things I noticed while working on the examples that I'll explain below.

For documentation and examples, we are planning to follow the same timeline roughly while working out a good way to make 1.x and 2.0 reference available for whoever needs either versions.

---

I noticed that the factory functions used to create the model object returns the created object directly instead of a promise, which I guess is to enable usage in preload since it doesn't work with returning promises. For most model it still works with async setup without issue but there are a couple model that require an async operation to complete before it can start working, the async operation is guarded behind the model.ready promise. While async setup can await the model.ready promise, which is what I did in those cases, it is likely not fully intended to be used this way. It adds an extra line to the user code and not all model needs it. I think this affect ml5.js usage outside of p5.js as well.

The ideal suggestion from me would be to have the factory function return a function that resolved to the model object that has all asynchronous setup task completed, similar to the pattern often seen in WASM backed libraries. This technically changes the API of your factory functions and may be a breaking change that you don't want to make.

The alternative is to propogate that model.ready promise down the chain to where it is async already and always await it before any operation. For example in BodyPose:

async detectLoop() {
  await this.ready;
  // Do everything else
}

This is a pattern I used in some obscure projects I did before and the downside is mainly additional boilerplate.

Let me know if my analysis above is incorrect and/or there's a better way to resolve this. I'm happy to go with whatever plan ml5 wants to go with in terms of implementing compatibility with p5.js 2.0, I can file PR towards them once there's a decision towards any particular solution as well.