Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mmtk-julia] Allow building Julia with variations of the binding #57176

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

udesou
Copy link
Contributor

@udesou udesou commented Jan 28, 2025

This PR enables building the mmtk-julia binding with different configurations other than non-moving Immix. While it was already possible to do so when building from source, this should enable using the binaries generated via the latest version from BinaryBuilder (JuliaPackaging/Yggdrasil#10357).
I have also decoupled MMTk from the code, such that it should be an option for a third-party heap. This should be set using WITH_THIRD_PARTY_GC=mmtk (in Make.user or as an environment variable).

The different configurations of MMTk can be achieved by setting the variables below (set in Make.user, for example):

Variable
MMTK_PLAN Immix StickyImmix
MMTK_MOVING 0 1
MMTK_BUILD release debug

Note that the actual code to support building with moving and sticky immix will be added in future PRs.
I've also added some preliminary documentation in doc/gc.md about building Julia with MMTk, including a link to the binding and an small FAQ on what to do if you break the build of Julia+MMTk.

@udesou udesou added the GC Garbage collector label Jan 28, 2025
@udesou udesou force-pushed the feature/adding-build-flags branch 2 times, most recently from 5ec92d1 to 12e2954 Compare January 28, 2025 05:10
@udesou udesou requested review from d-netto and qinsoon January 28, 2025 05:12
@udesou udesou force-pushed the feature/adding-build-flags branch from 12e2954 to 47cf1c5 Compare January 28, 2025 05:18
@udesou udesou changed the title Allow building Julia with variations of the binding [mmtk-julia] Allow building Julia with variations of the binding Jan 28, 2025
@udesou udesou added the GC: MMTK MMTK GC integration label Jan 28, 2025
@udesou udesou force-pushed the feature/adding-build-flags branch 3 times, most recently from cab6d06 to 9f53c2c Compare January 28, 2025 07:14
@udesou udesou force-pushed the feature/adding-build-flags branch 11 times, most recently from 45f710e to a0f5c24 Compare February 4, 2025 01:22
@d-netto d-netto force-pushed the feature/adding-build-flags branch 2 times, most recently from 7e309a4 to d7bc72c Compare February 4, 2025 18:56
@udesou udesou force-pushed the feature/adding-build-flags branch 2 times, most recently from d5b9db2 to f890322 Compare February 9, 2025 22:46
@udesou udesou force-pushed the feature/adding-build-flags branch 2 times, most recently from 1870113 to 51a7abc Compare February 12, 2025 00:18
@udesou udesou force-pushed the feature/adding-build-flags branch 5 times, most recently from 9aac610 to 20d4373 Compare February 18, 2025 23:22
@@ -0,0 +1,43 @@
# Julia + MMTk

There has been quite a lot of effort to refactor the GC code inside Julia to support external GCs. The first step to enable using different GC algorithms for Julia was the design and implementation of a [GC interface](https://docs.google.com/document/d/1v0jtSrIpdEDNOxj5S9g1jPqSpuAkNWhr_T8ToFC9RLI/edit?usp=sharing). To drive that interface, we added support for building Julia with [MMTk](https://www.mmtk.io). MMTk is a memory management toolkit providing language implementers with a framework to implement flexible and performant GCs. The flexibility comes from the fact that it is possible to switch implementations fairly easily. MMTk supports state-of-the-art high-performance implementations that are continuously added and maintained in the core part of the framework. MMTk is under active development and has been used by other programming languages such as [Java](https://github.com/mmtk/mmtk-openjdk) and [Ruby](https://github.com/ruby/mmtk). To support a language, it is necessary to implement an *MMTk binding*, which contains the code that connects the language to [mmtk-core](https://github.com/mmtk/mmtk-core). The mmtk-julia binding can be found in [this repository](https://github.com/mmtk/mmtk-julia).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some reservations about this paragraph. In particular:

  • We could make the explanation about what the MMTk acronym stands for a lot more explicit. You implicitly explain it in the sentence MMTk is a memory management toolkit, but I think we should probably put this in a parentheses right after the first time the acronym is mentioned. I.e. MMTk (Memory Management Toolkit).

  • The sentences The flexibility comes from the fact that it is possible to switch implementations fairly easily. MMTk supports state-of-the-art high-performance implementations that are continuously added and maintained in the core part of the framework. MMTk is under active development and has been used by other programming languages such as [Java](https://github.com/mmtk/mmtk-openjdk) and [Ruby](https://github.com/ruby/mmtk).

Don't seem like a good fit for a developer documentation.

As a core developer, I'd be a lot more interested in knowing that MMTk will allow me to experiment with a range of algorithms ranging from a simple serial mark-sweep GC to a sophisticated moving and concurrent GC (e.g. LXR), and having a brief explanation about why the design/architecture of MMTk allows me to that easily -- using a bit of technical terminology would be fine, since it's a developer documentation.

The sentences describing that (implementations) are continuously added and maintained in the core part of the framework and MMTk is under active development seem unnecessary: these seem like minimum expectations for any high-quality software project.

Citing other languages that adopted MMTk doesn't seem necessary in this documentation either.

Comment on lines 10 to 18
There are 3 different ways of building Julia with MMTk: building from source using a fixed release of the binding, checking out a custom version in the mmtk-julia [repository](https://github.com/mmtk/mmtk-julia) or using a precompiled binary from Julia's BinaryBuilder. The easiest way is to use the BinaryBuilder binary. First, to enable MMTk as a third-party GC, set the variable `WITH_THIRD_PARTY_GC` to `mmtk`. Then, simply set the variable `MMTK_PLAN` to one of the supported plans below and build Julia as usual.

There are different configurations supported by the following variables, which can be set in a `Make.user` file or as an environment variable.

| Variable | | |
|---------------|--------------|---------------|
| `MMTK_PLAN` | Immix | StickyImmix |
| `MMTK_MOVING` | 0 | 1 |
| `MMTK_BUILD` | release | debug |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're only instructing users to set WITH_THIRD_PARTY_GC and MMTK_PLAN in the Make.user, but then mention that there are other environment variables available (MMTK_MOVING, MMTK_BUILD).

This is a bit confusing, and it's possibly coming from the fact that we don't support moving yet.

I think we should mention that these variables can also be set, but at the time we're writing this, they will be no-ops, throw an error or something else because we don't support moving yet.

Comment on lines 1 to 12
// This file is a part of Julia. License is MIT: https://julialang.org/license

// Pick the appropriate third-party implementation
#ifdef WITH_THIRD_PARTY_HEAP
#if WITH_THIRD_PARTY_HEAP == 1
#include "gc-tls-mmtk.h"
#endif
// To be extended by other GC implementations
// #if WITH_THIRD_PARTY_HEAP == X
// #include gc-tls-XXXX.h
// #endif
#endif
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having this file seems a bit too much at the moment.

We don't plan to support any other third-party GC in the foreseeable future, so I'd just directly inline this logic:

#ifdef WITH_THIRD_PARTY_HEAP
#if WITH_THIRD_PARTY_HEAP == 1
#include "gc-tls-mmtk.h"
#endif

into julia_threads.h.

@udesou udesou force-pushed the feature/adding-build-flags branch 2 times, most recently from 3816d0c to 1673f57 Compare February 20, 2025 05:37
@udesou udesou force-pushed the feature/adding-build-flags branch from 1673f57 to 2ece67b Compare February 20, 2025 05:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GC: MMTK MMTK GC integration GC Garbage collector
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants