Feature request: Caching

[jmcvetta]

It would be very useful if the Github Actions cache system could be integrated with this action. It should be able to save significant time when installing a large list of tools.

[smorimoto]

Does it mean caching the plugins themselves? ​Or does it mean caching the tools installed by asdf?

[jmcvetta]

We want to cache anything that is downloaded when running asdf install against a .tool-versions file.

With the cache action, it would look something like this:

      - uses: actions/cache@v2
        with:
          path: |
            ~/.asdf/installs
            ~/.asdf/plugins
          key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}

Currently we can't use the cache action with the asdf action. Because the asdf action runs git checkout to install asdf. If the cache action has already populated ~/.asdf/installs, then the git checkout fails because the folder ~/.asdf/ already exists.

[0x6a74]

In my use case caching works as expected:

... checkout
... asdf setup

- name: asdf cache
  uses: actions/cache@v2
  with:
    path: ${{ env.ASDF_DIR }}
    key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}
    restore-keys: |
      ${{ runner.os }}-asdf-

... asdf add plugin 
... asdf install

[smorimoto]

OK. We will make some major improvements in the near future, so let's do it together.

[thbar]

Many thanks @0x6a68, your snippet proved very useful (I'm caching the Erlang build which takes ages get currently, as described in asdf-vm/asdf-erlang#165).

For those interested, the full script is available at thbar/ex-portmidi#4

[jthegedus]

Currently we can't use the cache action with the asdf action. Because the asdf action runs git checkout to install asdf. If the cache action has already populated ~/.asdf/installs, then the git checkout fails because the folder ~/.asdf/ already exists.

@jmcvetta I have seen people using continue-on-error: true with the Cache action to avoid this issue

Additionally, you can skip the asdf install step if you get a cache hit with: https://github.com/actions/cache#skipping-steps-based-on-cache-hit

[afirth]

I think you could close this @jthegedus, the cache action works well.

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Install asdf
        uses: asdf-vm/actions/setup@v1

      - name: asdf cache
        id: asdf-cache
        uses: actions/cache@v2
        with:
          path: ~/.asdf/
          key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}

      - name: Install CD tools from .tool-versions
        if: steps.asdf-cache.outputs.cache-hit != 'true'
        uses: asdf-vm/actions/install@v1

Thanks for all the hints above

[ai]

Some benchmark from my abstract project:

• asdf-vm/actions: 50 sec
• actions/node: 40 sec
• asdf-vm/actions with caching: 30 sec

Looks like cache will be very useful (since @afirth snippet is long).

[Kurt-von-Laven]

Thanks for the great snippet, @afirth; I expect this will speed up our workflows considerably! I agree that caching would be an excellent improvement to bake into the action itself. Even if the snippet exists, only a small percentage of users of the action will find it, and it downloads asdf itself twice in the event of a cache hit. In case this helps others, you only need to specify asdf_branch to the setup action if you don't want to use master since the setup step will be skipped by the install action. Also, I expect that users of this snippet will encounter issues where outdated versions of files in asdf core from the cache overwrite an updated asdf core. I suggest both pinning a release using the asdf_branch parameter to the setup action and including that release in the cache key so that updates to asdf itself invalidate the cache.

[snowe2010]

When trying to restore the cache currently, we get issues because it seems like node isn't restored properly. No clue why though.

[Kurt-von-Laven]

@snowe2010, can you offer more details? We are using the approach I described successfully, and we do use the node plugin. See ScribeMD/pre-commit-action for an example.

[andyduong1920]

I think you could close this @jthegedus, the cache action works well.

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Install asdf
        uses: asdf-vm/actions/setup@v1

      - name: asdf cache
        id: asdf-cache
        uses: actions/cache@v2
        with:
          path: ~/.asdf/
          key: ${{ runner.os }}-${{ hashFiles('**/.tool-versions') }}

      - name: Install CD tools from .tool-versions
        if: steps.asdf-cache.outputs.cache-hit != 'true'
        uses: asdf-vm/actions/install@v1

Thanks for all the hints above

Thanks @afirth, this solution works on our project as well, should we add this into the README file so other developer can follow it 😄

[andyduong1920]

It can works without the if: steps.asdf-cache.outputs.cache-hit != 'true' #445 (comment)

[Kurt-von-Laven]

It would be great to document the recipe more visibly, but it would need to be corrected first as mentioned previously:

Thanks for the great snippet, @afirth; I expect this will speed up our workflows considerably! I agree that caching would be an excellent improvement to bake into the action itself. Even if the snippet exists, only a small percentage of users of the action will find it, and it downloads asdf itself twice in the event of a cache hit. In case this helps others, you only need to specify asdf_branch to the setup action if you don't want to use master since the setup step will be skipped by the install action. Also, I expect that users of this snippet will encounter issues where outdated versions of files in asdf core from the cache overwrite an updated asdf core. I suggest both pinning a release using the asdf_branch parameter to the setup action and including that release in the cache key so that updates to asdf itself invalidate the cache.

[snowe2010]

@Kurt-von-Laven I think it is due to using a custom build machine that is shared with projects that have different node versions. The node version somehow gets mangled or not updated correctly.

[Kurt-von-Laven]

@snowe2010, what “it” are you referring to? Are you asking a question, and if so, what question? What are you trying to achieve, and how are you currently going about it?

[snowe2010]

#235 (comment)

[snowe2010]

To give you more information might take a while. I pretty much just trashed the asdf caching for the projects I was having issues on because I couldn't get node to work correctly.

[Kurt-von-Laven]

Makes sense. To try to help you I would need to see the relevant excerpt of your workflow and any pertinent errors, warnings, logs, etc., but obviously your call whether it's worth investigating.

[pvinis]

Just coming here to say that I also used that and it works great. https://github.com/react-native-community/rn-diff-purge/blob/master/.github/workflows/new_release.yml

[MPV]

Idea: could this make use of the same RUNNER_TOOL_CACHE mechanism that the Python ASDF action uses? See:

• https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#hosted-tool-cache

I realize it's adjacent to the discussion here, crosslinking it:

• Use Existing Setup Actions When Available #442

[TPXP]

The action seems to pick existing data properly now, adding this to your pipeline should work

    - name: Cache asdf
      uses: actions/cache@v3
      with:
        path: |
          ~/.asdf
        key: ${{ runner.os }}-asdf-${{ hashFiles('**/.tool-versions') }}

[jaymecd]

works like a charm:

• install asdf itself
• restore cache:
  • cache only 2 directories: ~/.asdf/plugins, ~/.asdf/installs
• reshim installed tools:
  • required to recreate adsf shims used to run tools

  steps:
    - name: Setup ASDF itself
      uses: asdf-vm/actions/setup@v2

    - name: Restore ASDF tools from cache
      id: asdf-tools-cache
      uses: actions/cache@v3
      with:
        key: asdf-tools-${{ hashFiles('.tool-versions') }}
        restore-keys: |
          asdf-tools-
        path: |
          ${{ env.ASDF_DIR }}/plugins
          ${{ env.ASDF_DIR }}/installs

    - name: Install ASDF tools on cache-miss
      if: ${{ steps.asdf-tools-cache.outputs.cache-hit != 'true' }}
      uses: asdf-vm/actions/install@v2

    - name: Reshim installed ASDF tools
      shell: bash
      run: asdf reshim

[Kurt-von-Laven]

Any reason not to simply use ${{ env.ASDF_DIR }} as the cache path? Not sure why, but we don't reshim, and it still works at least for the plugins we are using.

[jaymecd]

Any reason not to simply use ${{ env.ASDF_DIR }} as the cache path? Not sure why, but we don't reshim, and it still works at least for the plugins we are using.

Definitely, according to the documentation ${{ env.ASDF_DIR }} depending on the installation either could be a ASDF git repo clone, or it might contain other artifacts not needed to be cached.

That's why install asdf first, restore previously installed plugins/tools, and reshim to match current versions from .tool-versions file.

[vorburger]

actions/cache#1328 adds documentation re. this to the cache action.