Implement runtime with dynamic dispatch #1051

JulianGCalderon · 2025-01-14T17:10:36Z

Implements runtime with dynamic dispatch.

Depends on #1052

Previously, runtime function were defined in a different crate cairo_native_runtime, compiled as a static library (.a), and linked to the starknet contract.

This PR gets rid of the runtime static library completely, by defining a global for each runtime function. On execution, it will access the global to find the runtime function. This global is originally empty, but when loading the executor, the caller sets the globals, making them point to the correct functions (located in the callers binary).

This PR is a POC and should be tested thoroughly. As this new approach adds an additional level of indirection when calling the runtime, we should check that it doesn't introduce benchmark regressions.

Implementation Details

The build_function method declares the global if not yet declared, accesses it, and calls the function.

The setup_runtime function sets the value of all defined globals with their corresponding location. It is called when loading the executors.

Since we are using rust's runtime, instead of contract's runtime, we can call set_costs_builtin directly, instead of looking for the symbol.

I removed the hyperfine standalone AOT benches, as adapting it to the new dynamic runtime is difficult, and probably not worth it.

Roadmap

Adapt all runtime bindings
Remove original runtime link (AOT)
Remove original runtime register (JIT)
Compare execution with VM on many blocks
- From 900000 to 900484 without diffs
- From 1000000 to 1000500 without diffs
Measure performance: 8% to 11% improvement
- From 900000 to 900010: 8%
  - With static Runtime: 1.78s
  - With dynamic Runtime: 1.64s
- From 900010 to 900020: 9%
  - With static Runtime: 2.81s
  - With dynamic Runtime: 2.57s
- From 900020 to 900030: 8%
  - With static Runtime: 3.89s
  - With dynamic Runtime: 3.59s
- From 900030 to 900040: 11%
  - With static Runtime: 5.47s
  - With dynamic Runtime: 4.91s
Measure compiled programs size: 20X improvement
- Average complied library with static runtime: 13MiB
- Average compiled library with dynamic runtime: 450KiB

Why is it faster? We should take a closer look at it.

Checklist

Linked to Github Issue
Unit tests added
Integration tests added.
This change requires new documentation.
- Documentation has been added/updated.

github-actions · 2025-01-14T18:25:08Z

Benchmark results Main vs HEAD.

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
`base dict_insert.cairo (JIT)`	3.935 ± 0.048	3.860	4.001	1.00
`base dict_insert.cairo (AOT)`	4.020 ± 0.011	3.996	4.032	1.02 ± 0.01

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
`head dict_insert.cairo (JIT)`	3.946 ± 0.115	3.842	4.246	1.03 ± 0.03
`head dict_insert.cairo (AOT)`	3.816 ± 0.029	3.775	3.858	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
`base dict_snapshot.cairo (JIT)`	3.834 ± 0.046	3.773	3.910	1.00
`base dict_snapshot.cairo (AOT)`	3.905 ± 0.029	3.860	3.940	1.02 ± 0.01

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
`head dict_snapshot.cairo (JIT)`	3.794 ± 0.050	3.738	3.905	1.03 ± 0.02
`head dict_snapshot.cairo (AOT)`	3.693 ± 0.032	3.623	3.749	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
`base factorial_2M.cairo (JIT)`	4.170 ± 0.023	4.138	4.214	1.00
`base factorial_2M.cairo (AOT)`	4.397 ± 0.051	4.307	4.456	1.05 ± 0.01

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
`head factorial_2M.cairo (JIT)`	4.157 ± 0.026	4.130	4.204	1.02 ± 0.01
`head factorial_2M.cairo (AOT)`	4.090 ± 0.032	4.038	4.137	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
`base fib_2M.cairo (JIT)`	3.725 ± 0.067	3.644	3.837	1.00
`base fib_2M.cairo (AOT)`	3.874 ± 0.048	3.802	3.948	1.04 ± 0.02

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
`head fib_2M.cairo (JIT)`	3.680 ± 0.028	3.626	3.723	1.01 ± 0.01
`head fib_2M.cairo (AOT)`	3.630 ± 0.028	3.584	3.696	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
`base linear_search.cairo (JIT)`	3.873 ± 0.036	3.837	3.959	1.00
`base linear_search.cairo (AOT)`	3.970 ± 0.033	3.915	4.019	1.03 ± 0.01

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
`head linear_search.cairo (JIT)`	3.878 ± 0.051	3.804	3.998	1.04 ± 0.01
`head linear_search.cairo (AOT)`	3.726 ± 0.021	3.696	3.755	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
`base logistic_map.cairo (JIT)`	4.004 ± 0.024	3.964	4.037	1.00
`base logistic_map.cairo (AOT)`	4.053 ± 0.038	3.998	4.110	1.01 ± 0.01

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
`head logistic_map.cairo (JIT)`	3.980 ± 0.036	3.944	4.060	1.05 ± 0.01
`head logistic_map.cairo (AOT)`	3.785 ± 0.022	3.749	3.828	1.00

codecov-commenter · 2025-01-14T18:27:00Z

Codecov Report

Attention: Patch coverage is 98.13433% with 5 lines in your changes missing coverage. Please review.

Project coverage is 80.80%. Comparing base (ecb0c82) to head (505ee41).

Files with missing lines	Patch %	Lines
src/executor/contract.rs	71.42%	4 Missing ⚠️
src/metadata/runtime_bindings.rs	99.57%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1051      +/-   ##
==========================================
- Coverage   80.94%   80.80%   -0.14%     
==========================================
  Files         108      108              
  Lines       29978    29557     -421     
==========================================
- Hits        24266    23885     -381     
+ Misses       5712     5672      -40

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2025-01-14T18:39:07Z

Benchmarking results

Benchmark for program `dict_insert`

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
`Cairo-vm (Rust, Cairo 1)`	21.192 ± 0.208	20.947	21.569	5.31 ± 0.08
`cairo-native (embedded AOT)`	3.994 ± 0.044	3.918	4.057	1.00
`cairo-native (embedded JIT using LLVM's ORC Engine)`	4.037 ± 0.026	4.013	4.084	1.01 ± 0.01

Benchmark for program `dict_snapshot`

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
`Cairo-vm (Rust, Cairo 1)`	6.015 ± 0.043	5.955	6.098	1.57 ± 0.02
`cairo-native (embedded AOT)`	3.822 ± 0.032	3.772	3.875	1.00
`cairo-native (embedded JIT using LLVM's ORC Engine)`	3.893 ± 0.030	3.844	3.936	1.02 ± 0.01

Benchmark for program `factorial_2M`

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
`Cairo-vm (Rust, Cairo 1)`	14.661 ± 0.109	14.433	14.813	3.58 ± 0.04
`cairo-native (embedded AOT)`	4.093 ± 0.029	4.055	4.145	1.00
`cairo-native (embedded JIT using LLVM's ORC Engine)`	4.184 ± 0.032	4.134	4.242	1.02 ± 0.01

Benchmark for program `fib_2M`

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
`Cairo-vm (Rust, Cairo 1)`	14.561 ± 0.104	14.334	14.735	3.98 ± 0.05
`cairo-native (embedded AOT)`	3.662 ± 0.033	3.615	3.726	1.00
`cairo-native (embedded JIT using LLVM's ORC Engine)`	3.703 ± 0.033	3.653	3.748	1.01 ± 0.01

Benchmark for program `linear_search`

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
`Cairo-vm (Rust, Cairo 1)`	6.027 ± 0.060	5.917	6.113	1.58 ± 0.02
`cairo-native (embedded AOT)`	3.810 ± 0.027	3.771	3.857	1.00
`cairo-native (embedded JIT using LLVM's ORC Engine)`	3.993 ± 0.044	3.936	4.099	1.05 ± 0.01

Benchmark for program `logistic_map`

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
`Cairo-vm (Rust, Cairo 1)`	5.838 ± 0.078	5.722	5.979	1.51 ± 0.04
`cairo-native (embedded AOT)`	3.856 ± 0.074	3.778	3.970	1.00
`cairo-native (embedded JIT using LLVM's ORC Engine)`	3.994 ± 0.066	3.924	4.088	1.04 ± 0.03

src/executor/contract.rs

* Load dynamic runtime globals * Implement dynamic runtime bindings * Use dynamic runtime set_costs_builtin * Remove phantom data from RuntimeBindingsMeta * Remove symbol register in JIT executor * Remove runtime linking in AOT executor * Remove cairo_native__get_version check * Use original name for runtime globals * Remove no_mangle attribute * Remove AOT standalone benchmarks --------- Co-authored-by: Edgar <[email protected]>

JulianGCalderon force-pushed the dynamic-runtime branch from 76e49df to a468e23 Compare January 14, 2025 17:30

JulianGCalderon force-pushed the dynamic-runtime branch 11 times, most recently from 94ca074 to be15f36 Compare January 15, 2025 18:49

JulianGCalderon changed the base branch from main to fix-ci January 15, 2025 18:49

JulianGCalderon force-pushed the dynamic-runtime branch 5 times, most recently from 23138a4 to 5005809 Compare January 15, 2025 19:33

JulianGCalderon changed the base branch from fix-ci to main January 15, 2025 19:35

JulianGCalderon force-pushed the dynamic-runtime branch 3 times, most recently from f62b6d7 to 46872ae Compare January 15, 2025 21:11

JulianGCalderon added 5 commits January 16, 2025 12:16

Load dynamic runtime globals

528d06e

Implement dynamic runtime bindings

973d668

Use dynamic runtime set_costs_builtin

3135554

Remove phantom data from RuntimeBindingsMeta

9f05df2

Remove symbol register in JIT executor

d28fad9

JulianGCalderon added 3 commits January 16, 2025 12:16

Use original name for runtime globals

a6f134c

Remove no_mangle attribute

69950cd

Remove AOT standalone benchmarks

1880fbe

JulianGCalderon force-pushed the dynamic-runtime branch from 8d6bac7 to 1880fbe Compare January 16, 2025 15:40

JulianGCalderon marked this pull request as ready for review January 16, 2025 15:52

JulianGCalderon requested review from edg-l, igaray, azteca1998, jrchatruc, entropidelic, fmoletta, Oppen, pefontana and gabrielbosio as code owners January 16, 2025 15:52

azteca1998 previously approved these changes Jan 16, 2025

View reviewed changes

src/executor/contract.rs Show resolved Hide resolved

edg-l added the review-ready A PR that is ready for review label Jan 20, 2025

edg-l previously approved these changes Jan 20, 2025

View reviewed changes

edg-l and others added 2 commits January 21, 2025 09:04

Merge branch 'main' into dynamic-runtime

981093a

Merge branch 'main' into dynamic-runtime

505ee41

JulianGCalderon dismissed stale reviews from azteca1998 and edg-l via 505ee41 January 22, 2025 13:56

gabrielbosio approved these changes Jan 22, 2025

View reviewed changes

edg-l approved these changes Jan 23, 2025

View reviewed changes

azteca1998 approved these changes Jan 24, 2025

View reviewed changes

gabrielbosio added this pull request to the merge queue Jan 24, 2025

Merged via the queue into main with commit 7d8871c Jan 24, 2025
28 checks passed

gabrielbosio deleted the dynamic-runtime branch January 24, 2025 19:56

gabrielbosio mentioned this pull request Jan 31, 2025

Set Runtime as artifact dependency #985

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement runtime with dynamic dispatch #1051

Implement runtime with dynamic dispatch #1051

JulianGCalderon commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 14, 2025 •

edited

Loading

codecov-commenter commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 14, 2025 •

edited

Loading

Implement runtime with dynamic dispatch #1051

Implement runtime with dynamic dispatch #1051

Conversation

JulianGCalderon commented Jan 14, 2025 • edited Loading

Implementation Details

Roadmap

Checklist

github-actions bot commented Jan 14, 2025 • edited Loading

Benchmark results Main vs HEAD.

Base

Head

Base

Head

Base

Head

Base

Head

Base

Head

Base

Head

codecov-commenter commented Jan 14, 2025 • edited Loading

Codecov Report

github-actions bot commented Jan 14, 2025 • edited Loading

Benchmarking results

Benchmark for program dict_insert

Benchmark for program dict_snapshot

Benchmark for program factorial_2M

Benchmark for program fib_2M

Benchmark for program linear_search

Benchmark for program logistic_map

JulianGCalderon commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 14, 2025 •

edited

Loading

codecov-commenter commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 14, 2025 •

edited

Loading

Benchmark for program `dict_insert`

Benchmark for program `dict_snapshot`

Benchmark for program `factorial_2M`

Benchmark for program `fib_2M`

Benchmark for program `linear_search`

Benchmark for program `logistic_map`