Code

[WIP][AMDGPU] try rocm POC#491

Code

Draft

yiakwy-xpu-ml-framework-team wants to merge 15 commits intoflashinfer-ai:mainflashinfer-ai/flashinfer:mainfrom yiakwy-xpu-ml-framework-team:porting_kernels_amd_ck_hip_versionyiakwy-xpu-ml-framework-team/AMD-flashinfer-benchmark-fork:porting_kernels_amd_ck_hip_versionCopy head branch name to clipboard

+2,590-144

Commits

Commits on Sep 9, 2024

• add rocm support:

  Show description for 553037f
  yiakwy-xpu-ml-framework-team

  committed
  553037f

  Copy full SHA for 553037f

Commits on Sep 22, 2024

• add support decode

  yiakwy-xpu-ml-framework-team

  committed
  5dfa16a

  Copy full SHA for 5dfa16a

• reproduce decode test

  yiakwy-xpu-ml-framework-team

  committed
  76ca1ca

  Copy full SHA for 76ca1ca

Commits on Sep 23, 2024

• add support of fast_div

  yiakwy-xpu-ml-framework-team

  committed
  512a45f

  Copy full SHA for 512a45f

• add support of test_fast_dequant (surpress fp8 tests in this PR)

  yiakwy-xpu-ml-framework-team

  committed
  d053266

  Copy full SHA for d053266

• add support of fast sampling

  yiakwy-xpu-ml-framework-team

  committed
  7583155

  Copy full SHA for 7583155

• fix wrong wavesize for DeterministicInclusiveSum

  yiakwy-xpu-ml-framework-team

  committed
  f172fc0

  Copy full SHA for f172fc0

• add support of test_page

  yiakwy-xpu-ml-framework-team

  committed
  c1144e4

  Copy full SHA for c1144e4

Commits on Oct 9, 2024

• add support of prefill (part 1). AMD mfma inst

  yiakwy-xpu-ml-framework-team

  committed
  1c8ed61

  Copy full SHA for 1c8ed61

Commits on Oct 11, 2024

• update compute_qk fragment layout

  yiakwy-xpu-ml-framework-team

  committed
  3d2d75d

  Copy full SHA for 3d2d75d

• update qk_compute layout and comments

  yiakwy-xpu-ml-framework-team

  committed
  6d583d8

  Copy full SHA for 6d583d8

Commits on Oct 14, 2024

• update kv fragment thread mapping

  yiakwy-xpu-ml-framework-team

  committed
  4a16a90

  Copy full SHA for 4a16a90

Commits on Oct 15, 2024

• update compute_qk logging info

  yiakwy-xpu-ml-framework-team

  committed
  61dfb1b

  Copy full SHA for 61dfb1b

Commits on Oct 21, 2024

• update kv fragment warps mapping (add boundary checking, e.g. qkv shape=(1,16,64))

  yiakwy-xpu-ml-framework-team

  committed
  4856dcf

  Copy full SHA for 4856dcf

Commits on Oct 25, 2024

• add support of prefill (part 2). add support update_mdo_states for output product registers

  yiakwy-xpu-ml-framework-team

  committed
  addf8b5

  Copy full SHA for addf8b5