You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently for encoder-decoder models we don't support cuda graph during the decode phase. This fr tracks adding support for cuda graph during decode phase. Adding this support will help speed up the decode phase.
This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!
🚀 The feature, motivation and pitch
Currently for encoder-decoder models we don't support cuda graph during the decode phase. This fr tracks adding support for cuda graph during decode phase. Adding this support will help speed up the decode phase.
#7366
cc: @afeldman-nm
Alternatives
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: