[BugFix] Add value.contiguous in attention to avoid some accuracy pro…

…blems. (#95) fix an accuracy problem caused by missing of value contiguous Signed-off-by: hw_whx <[email protected]> Co-authored-by: hw_whx <[email protected]>
vllm-project · Feb 19, 2025 · 36e9d6e · 36e9d6e
1 parent 65c2795
commit 36e9d6e
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/vllm_ascend/attention.py b/vllm_ascend/attention.py
@@ -538,6 +538,8 @@ def forward(
         query = query.view(-1, self.num_heads, self.head_size)
         key = key.view(-1, self.num_kv_heads, self.head_size)
         value = value.view(-1, self.num_kv_heads, self.head_size)
+        # TODO: Remove this contiguous in the future.
+        value = value.contiguous()
 
         output = torch.empty(num_tokens,
                              self.num_heads,