feat: implement fetchable runner

[sheremet-va]

Description

Rought but working concept of a fetchable runner (related to #18191)

Note that this is not a fetchable environment (a bit confusing, is it?).

const server = await createServer()
await server.listen()

// can be initiated in a another process
const runner = createFetchableModuleRunner({
  root: process.env.VITE_CONFIG_ROOT,
  serverURL: process.env.VITE_SERVER_URL,
  environmentName: process.env.VITE_ENVIRONMENT_NAME,
})

[stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[sheremet-va]

I need help with this one 🤔 @patak-dev

[hi-ogawa]

Related to my comment #18338 (comment), exposing fetchModule from a bridge server would essentially have the same issue, but what I did is to protect the endpoint by requiring key, which is only available to child process hi-ogawa/vite-environment-examples#132

[sheremet-va]

If you have a bridge, it's not a fetchable environment anymore, is it? This seems like completely unrelated approach

[hi-ogawa]

Yeah, it's probably right to say it's a completely independent concept from "fetchable environment" concept.

Bridge server is only to help a few http rpc between environment implementation and runtime side communication, which can include fetchModule. Then, independently from such logic, devEnv.dispatchFetch(request) can be implemented as proxing the server spawned inside runtime.

For child process specifically, rpc can be implemented with stdio (or if nodejs child process, onmessage/postMessage works too). And if it's miniflare, then it already supports fairly arbitrary communication between main process and runtime, so that's used.

My impression is that even if we provide this middleware to expose fetchModule, I would imagine environment implementation would require other communication primitive for other need, so this middleware alone doesn't seem useful in practice.

[sheremet-va]

I would imagine environment implementation would require other communication primitive for other need

For what needs? How would that achieve those needs in production then if fetch is not enough?

[hi-ogawa]

Well, this is the only need I had, but runner side can tell its randomly allocated server port to the environment, so they can proxy dispatchFetch later https://github.com/netlify/netlify-vite-environment/blob/f4c2e77d832b3f33df57d2a30bdc8de0a4ab5ca7/packages/vite-environment-provider-netlify/src/runtime-bridge.ts#L54-L55

This is only about "dev" environment implementation specifics, which is hidden from production framework/app implementations.

[hi-ogawa]

This looks very natural to have, but one thing I worry is that requiring server.listen before creating a runner seems not so intuitive. For example, during DevEnvironment.init, server is not started, so environment integration would require delaying runner initialization further (though when init runs might change for #18264).

To mitigate this limitation, I saw Netlify made a "bridge" server for their environment's own purpose, which allows communicating between runner and environment in more arbitrary manner. https://github.com/netlify/netlify-vite-environment/blob/main/packages/vite-environment-provider-netlify/src/runtime-bridge.ts

Though it feels pretty odd that for Vite to spawn yet another server, Netlify's approach seems simpler to reason about to implement environment, so I did something similar in my child process poc https://github.com/hi-ogawa/vite-environment-examples/tree/main/examples/child-process.

[sheremet-va]

This looks very natural to have, but one thing I worry is that requiring server.listen before creating a runner seems not so intuitive. For example, during DevEnvironment.init, server is not started, so environment integration would require delaying runner initialization further (though when init runs might change for #18264).

This is not the case though. listen needs to be called before you import the module, not before you create a runner.

[sheremet-va]

@Rich-Harris would love to hear your thoughts on the implementation and @hi-ogawa feedback

[hi-ogawa]

This is not the case though. listen needs to be called before you import the module, not before you create a runner.

For child process case I have in mind, spawning child process needs to be delayed to be able to pass VITE_SERVER_URL during spawn time. Or alternatively, it requires communication from server to runtime to pass VITE_SERVER_URL after spawned.

[hi-ogawa]

Ah, forgot to add that this middleware should work for web worker environment/runner https://github.com/hi-ogawa/vite-environment-examples/tree/main/examples/web-worker. serverUrl can be taken from self.location.origin.