Provide default JVM version to be used by Mill server process, when `.mill-jvm-version` and system `java` are not provided #4597

lihaoyi · 2025-02-20T08:26:28Z

This PR makes Mill's server use a fixed JVM version by default in the case where no other Java version is available:

No .mill-jvm-version file present
No java command available environmentally
The Mill client has no java.home set (i.e. it is a graal native image)

This makes running Mill on a clean environment without a JVM installed work by default. Previously, it either needed you to define a .mill-jvm-version or install a JVM, either of which is a manual step we now can now avoid. Now it instead uses the version of Java configured in Mill's own .mill-jvm-version file

We still allow any system java command to take precedence over the default, since that's what most people would expect when using the JVM.

Added an integration test in CI no-jvm-bootstrap that runs Mill with the native launcher with no .mill-jvm-version file and no system java installed to verify that it falls back to the Mill default JVM version. Needed to add some hacky test hooks to make sure we exercise the code path in question

alexarchambault · 2025-02-20T10:25:31Z

runner/client/src/mill/runner/client/MillProcessLauncher.java

    Path millJvmVersionFile = millJvmVersionFile();

    String javaHome = null;
    if (Files.exists(millJvmVersionFile)) {
      jvmId = Files.readString(millJvmVersionFile).trim();
+      if (jvmId.equals("global")) jvmId = null;


FYI in coursier, system stands for "use the already installed / available JVM". Don't know which is better, if global feels natural to enough people, I could accept it in coursier too.

system works too, I was unaware of Coursier's default. I can change this over

alexarchambault · 2025-02-20T10:27:21Z

runner/client/src/mill/runner/client/MillProcessLauncher.java

    Path millJvmVersionFile = millJvmVersionFile();

    String javaHome = null;
    if (Files.exists(millJvmVersionFile)) {
      jvmId = Files.readString(millJvmVersionFile).trim();
+      if (jvmId.equals("global")) jvmId = null;
+    } else {
+      jvmId = "zulu:17.0.3";


Is there a reason to use zulu rather than temurin? I recall defaulting to zulu sometimes, because it has JDK 8 builds for macOS ARM64 while temurin doesn't, but that shouldn't be a problem here.

IIRC there were some JDKs that Zulu had published for Windows-ARM that Temurin hadn't for these versions, which is why we use Zulu in Mill's own config. But probably can find a version that works on Termurin

Here is the issue: Temurin doesn't publish JDKs for versions <21 on Windows-ARM (e.g. my Surface Laptop 7). The only options for JDK 17 on Windows-ARM are Liberica and Zulu

https://github.com/coursier/jvm-index/blob/master/indices/windows-arm64.json

lefou

Looks good to me.

lihaoyi · 2025-02-21T13:20:28Z

Seems like CI is hitting No space left on device errors with this PR. I assume it has to be due to us leaking downloaded JVMs somewhere in Coursier, and now that we're calling Coursier every integration/example test it's adding up enough to be a problem. @alexarchambault probably will need your help to look into coursier see if we're leaking any temporary download files or something

alexarchambault · 2025-02-21T13:44:56Z

Not 100% sure, but I suspect the way the coursier archive cache is handled on CI might cause that. IIUC, things like this then this make the COURSIER_ARCHIVE_CACHE env var have an empty value. I think that must make it use the current directory as cache. And with sandbox stuff, that directory must change quite often, so JDKs must be unpacked more than they need to.

alexarchambault · 2025-02-21T13:45:45Z

Going to try to tweak how COURSIER_ARCHIVE_CACHE is handled in workflows in a separate PR

lihaoyi · 2025-02-21T14:05:29Z

Good catch with the COURSIER_ARCHIVE_CACHE, maybe the empty value is making it live in the cwd rather than just having the value unset.

lihaoyi · 2025-02-21T14:24:08Z

Let me see if setting the default in github actions to null instead of "" helps

lihaoyi · 2025-02-21T14:46:52Z

Looks like passing in null doesn't seem to help https://github.com/lihaoyi/mill-1/actions/runs/13458684002/job/37608878277

lihaoyi · 2025-02-21T16:17:31Z

Seems like setting coursierarchive to /tmp seems to have worked! A few other errors to sort out, but at least it seems we're unblocked now

lihaoyi · 2025-02-22T02:33:59Z

One possible point of confusion with this PR if you use Mill without configuring a java version on your JavaModules, and without configuring a .mill-jvm-version file in your build, you will get the Mill default zulu:17.0.3 JVM rather than your environmentally-installed java JVM.

This would be pretty confusing, and maybe in the absence of explicit configuration it's worth making application code build/run/tested with mill default to the system java rather than the Mill default.

In general it seems reasonable that the default JVM for user-code and for .mill build files would differ: user code may require a variety of JVM versions, while .mill build files are more under Mill's control and can demand a more fixed JVM version. But currently the user-code default JVM is also the Mill JVM, with not way to configure them separately.

I think the right thing to do here may be to separate the two:

.mill-jvm-version that applies to application code, and defaults to system if not set
.mill-build-jvm-version that applies to Mill itself, and defaults to zulu:17.0.3 if not set

I expect most people who want to tweak the Java version would care about .mill-jvm-version, and only very rarely would someone want to tweak .mill-build-jvm-version for doing fancy things in their .mill files.

However, the downside here is that someone would need to download two separate JVMs just to build their code. While the download is automatic, the user would still need to wait for it to complete, which might be slow since JVMs are big

lihaoyi · 2025-02-22T04:40:32Z

Another more lightweight change would be to only fall back to the hardcoded zulu:17.0.3 if there is no existing JVM installed. That would preserve the ability to use whatever JDK people have installed, while also keeping the Mill JDK consistent with the JavaModule default JDK, and avoid downloading two separate JDKs.

I'm not sure how to check whether there is already a JVM available, apart from shelling out to java -help which is pretty expensive (50-100ms). @alexarchambault do you know if there's any easy way to do such a check?

alexarchambault · 2025-02-22T13:29:15Z

Both coursier and Scala CLI do such things. I can't have a look right now, but IIRC, they might just look at JAVA_HOME or for a java binary in PATH (which needs quite some care on Windows, with PATHEXT and all), and maybe read the content of a file under JAVA_HOME 🤔

alexarchambault · 2025-02-22T13:30:45Z

coursier and Scala CLI don't require a minimum Java version (they do, like >= 8, but don't really check it IIRC), but Scala CLI does look at the Java version when it spawns a Bloop server (it needs >= 17).

lihaoyi · 2025-02-25T00:43:04Z

Decided to go with the more conservative approach of just providing a fallback default if there is no JVM available.

fix

6f68a42

lihaoyi requested review from lefou and alexarchambault February 20, 2025 08:26

alexarchambault reviewed Feb 20, 2025

View reviewed changes

fix

0541e36

lihaoyi changed the title ~~Pin JVM version used by Mill server process, unless override by .mill-jvm-version containing global~~ Pin JVM version used by Mill server process, unless override by .mill-jvm-version containing system Feb 21, 2025

lefou approved these changes Feb 21, 2025

View reviewed changes

lihaoyi changed the title ~~Pin JVM version used by Mill server process, unless override by .mill-jvm-version containing system~~ Pin JVM version used by Mill server process, unless overriden by .mill-jvm-version containing system Feb 21, 2025

try null coursierarchive

7882e6d

lihaoyi force-pushed the default-pinned-java branch from 645065b to 7882e6d Compare February 21, 2025 14:18

lihaoyi changed the title ~~Pin JVM version used by Mill server process, unless overriden by .mill-jvm-version containing system~~ Pin JVM version used by Mill server process, unless overridden by .mill-jvm-version containing system Feb 21, 2025

.

daa6ddf

lihaoyi added 4 commits February 22, 2025 08:52

.

5b20756

.

6b65476

.

b876b0f

.

ccf7fb8

lihaoyi added 3 commits February 24, 2025 11:47

.

8d5da59

.

d74164a

.

24fa5bc

lihaoyi added 6 commits February 24, 2025 16:51

debug

da3da83

debug

f7d7917

debug

cbb0491

debug

dd35537

debug

5849024

debug

a286d19

lihaoyi force-pushed the default-pinned-java branch from 05a229f to a286d19 Compare February 24, 2025 09:17

lihaoyi added 3 commits February 24, 2025 17:17

debug

c9e4e9a

debug

997f43d

debug

de7c1d7

lihaoyi force-pushed the default-pinned-java branch from 90e5a23 to de7c1d7 Compare February 24, 2025 11:22

lihaoyi added 10 commits February 24, 2025 19:25

debug

5e43808

debug

30157dd

debug

b5d4517

debug

b57c36d

debug

60bc49c

.

230be4d

.

e471d7d

.

b35629a

.ush

f61d422

update-patch

de8a343

lihaoyi force-pushed the default-pinned-java branch from f7a0d41 to de8a343 Compare February 24, 2025 16:31

lihaoyi added 4 commits February 25, 2025 00:40

update-patch

e1ee7a9

merge-patch

4308d71

patch

50aeaab

.

14eedaa

lihaoyi merged commit 5281a01 into com-lihaoyi:main Feb 25, 2025
33 checks passed

lihaoyi mentioned this pull request Feb 25, 2025

Don't change COURSIER_ARCHIVE_CACHE at all on CI #4606

Closed

lefou added this to the 0.13.0 milestone Feb 25, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide default JVM version to be used by Mill server process, when `.mill-jvm-version` and system `java` are not provided #4597

Provide default JVM version to be used by Mill server process, when `.mill-jvm-version` and system `java` are not provided #4597

lihaoyi commented Feb 20, 2025 •

edited

Loading

alexarchambault Feb 20, 2025

lihaoyi Feb 21, 2025

alexarchambault Feb 20, 2025

lihaoyi Feb 21, 2025

lihaoyi Feb 21, 2025 •

edited

Loading

lefou left a comment

lihaoyi commented Feb 21, 2025

alexarchambault commented Feb 21, 2025

alexarchambault commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 22, 2025 •

edited

Loading

lihaoyi commented Feb 22, 2025

alexarchambault commented Feb 22, 2025

alexarchambault commented Feb 22, 2025

lihaoyi commented Feb 25, 2025

Provide default JVM version to be used by Mill server process, when .mill-jvm-version and system java are not provided #4597

Provide default JVM version to be used by Mill server process, when .mill-jvm-version and system java are not provided #4597

Conversation

lihaoyi commented Feb 20, 2025 • edited Loading

alexarchambault Feb 20, 2025

Choose a reason for hiding this comment

lihaoyi Feb 21, 2025

Choose a reason for hiding this comment

alexarchambault Feb 20, 2025

Choose a reason for hiding this comment

lihaoyi Feb 21, 2025

Choose a reason for hiding this comment

lihaoyi Feb 21, 2025 • edited Loading

Choose a reason for hiding this comment

lefou left a comment

Choose a reason for hiding this comment

lihaoyi commented Feb 21, 2025

alexarchambault commented Feb 21, 2025

alexarchambault commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 21, 2025

lihaoyi commented Feb 22, 2025 • edited Loading

lihaoyi commented Feb 22, 2025

alexarchambault commented Feb 22, 2025

alexarchambault commented Feb 22, 2025

lihaoyi commented Feb 25, 2025

Provide default JVM version to be used by Mill server process, when `.mill-jvm-version` and system `java` are not provided #4597

Provide default JVM version to be used by Mill server process, when `.mill-jvm-version` and system `java` are not provided #4597

lihaoyi commented Feb 20, 2025 •

edited

Loading

lihaoyi Feb 21, 2025 •

edited

Loading

lihaoyi commented Feb 22, 2025 •

edited

Loading