Code

fp8 backward#119

Code

Draft

micmelesse wants to merge 19 commits intomain_perfROCm/flash-attention:main_perffrom micmelesse/fp8_bwdROCm/flash-attention:micmelesse/fp8_bwdCopy head branch name to clipboard

+719-398

Commits

Commits on Feb 6, 2025

• Enable BWD fp8 with split kernel

  Show description for 156a9bc
  micmelesse

  committed
  156a9bc

  Copy full SHA for 156a9bc

• add type info for backward

  micmelesse

  committed
  e6a67b3

  Copy full SHA for e6a67b3

• fix DEBUG flag bug

  micmelesse

  committed
  c13c0f0

  Copy full SHA for c13c0f0

• fix bug with backward. Normal forward works with dropout. Segfault with causal. Varlen has some issues. Might be related to strides.

  micmelesse

  committed
  acb05ad

  Copy full SHA for acb05ad

• pass descale strides

  micmelesse

  committed
  ca267ed

  Copy full SHA for ca267ed

• test causal

  micmelesse

  committed
  00d1c6f

  Copy full SHA for 00d1c6f

• fix causal compiler assert. min head should be 32

  micmelesse

  committed
  3ba93db

  Copy full SHA for 3ba93db

• remove descale_p

  micmelesse

  committed
  3694224

  Copy full SHA for 3694224

Commits on Feb 7, 2025

• save

  micmelesse

  committed
  7908150

  Copy full SHA for 7908150

• explict name as causal

  micmelesse

  committed
  86fd7e6

  Copy full SHA for 86fd7e6

• isolate bad case

  micmelesse

  committed
  01b370a

  Copy full SHA for 01b370a

• just run fp8 tests

  micmelesse

  committed
  290d594

  Copy full SHA for 290d594

• bench with autotune

  micmelesse

  committed
  e9e4d6e

  Copy full SHA for e9e4d6e

• min changes

  micmelesse

  committed
  736a990

  Copy full SHA for 736a990

Commits on Feb 10, 2025

• cast_fp8 helper

  micmelesse

  committed
  9fc0d0a

  Copy full SHA for 9fc0d0a

• cast_varlen_to_fp8

  micmelesse

  committed
  db15c3d

  Copy full SHA for db15c3d

• save

  micmelesse

  committed
  32d552e

  Copy full SHA for 32d552e

• minor

  micmelesse

  committed
  fbb00d6

  Copy full SHA for fbb00d6

Commits on Feb 11, 2025

• highlight failing configs

  micmelesse

  committed
  9615417

  Copy full SHA for 9615417