Code

Fused ROPE and reshape cache kernel#229

Code

Open

maleksan85 wants to merge 21 commits intomainROCm/vllm:mainfrom fused_re_and_reshape_cache_kernelROCm/vllm:fused_re_and_reshape_cache_kernelCopy head branch name to clipboard

+758-33

Commits

Commits on Sep 19, 2024

• Initial Frame, without kernel itself

  

  Aleksandr Malyshev

  committed
  eee3c78

  Copy full SHA for eee3c78

Commits on Sep 20, 2024

• fused kernel

  

  Aleksandr Malyshev

  committed
  2f19c01

  Copy full SHA for 2f19c01

Commits on Oct 7, 2024

• Merge branch 'main' into fused_re_and_reshape_cache_kernel

  

  Aleksandr Malyshev

  committed
  beba692

  Copy full SHA for beba692

Commits on Oct 9, 2024

• compilation passes, begging query

  

  Aleksandr Malyshev

  committed
  d7eaac6

  Copy full SHA for d7eaac6

• query correctly ROPEd

  

  Aleksandr Malyshev

  committed
  deaf6aa

  Copy full SHA for deaf6aa

Commits on Oct 11, 2024

• teduced test scope is working

  

  Aleksandr Malyshev

  committed
  108fb65

  Copy full SHA for 108fb65

• ordered per slot mapping test passed

  

  Aleksandr Malyshev

  committed
  c5dca86

  Copy full SHA for c5dca86

• kernel ready except fp8

  

  Aleksandr Malyshev

  committed
  58a93c1

  Copy full SHA for 58a93c1

• + fp8 for KV cache support

  

  Aleksandr Malyshev

  committed
  ef9a664

  Copy full SHA for ef9a664

Commits on Oct 14, 2024

• vllm and test runs, vllm has corretness issue

  

  Aleksandr Malyshev

  committed
  b30a7f7

  Copy full SHA for b30a7f7

Commits on Oct 15, 2024

• correctness is fixed

  

  Aleksandr Malyshev

  committed
  f0648c3

  Copy full SHA for f0648c3

• Merge branch 'main' into fused_re_and_reshape_cache_kernel

  

  Aleksandr Malyshev

  committed
  afa466b

  Copy full SHA for afa466b

• honest switch from fused ROPE and original

  

  Aleksandr Malyshev

  committed
  2548322

  Copy full SHA for 2548322

Commits on Oct 16, 2024

• timing test

  

  Aleksandr Malyshev

  committed
  3af41ef

  Copy full SHA for 3af41ef

Commits on Nov 13, 2024

• Merge branch 'main' into fused_re_and_reshape_cache_kernel

  

  Aleksandr Malyshev

  committed
  3806814

  Copy full SHA for 3806814

Commits on Nov 19, 2024

• small refactoring of kernel

  

  Aleksandr Malyshev

  committed
  52a70b3

  Copy full SHA for 52a70b3

Commits on Nov 21, 2024

• moving load cos/sin to common code

  maleksan85

  committed
  97582cc

  Copy full SHA for 97582cc

Commits on Dec 3, 2024

• everything except values to cache

  

  Aleksandr Malyshev

  committed
  47ff040

  Copy full SHA for 47ff040

• + value cache filling

  

  Aleksandr Malyshev

  committed
  81bcf3e

  Copy full SHA for 81bcf3e

Commits on Dec 4, 2024

• union with float4

  

  Aleksandr Malyshev

  committed
  d14c054

  Copy full SHA for d14c054

Commits on Dec 5, 2024

• dwordx4 seen

  

  Aleksandr Malyshev

  committed
  f066592

  Copy full SHA for f066592