Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Encoder-Decoder]: Add support for cuda graph during decoding in encoder-decoder models #7447

Closed
sroy745 opened this issue Aug 13, 2024 · 1 comment

Comments

@sroy745
Copy link
Collaborator

sroy745 commented Aug 13, 2024

🚀 The feature, motivation and pitch

Currently for encoder-decoder models we don't support cuda graph during the decode phase. This fr tracks adding support for cuda graph during decode phase. Adding this support will help speed up the decode phase.

#7366

cc: @afeldman-nm

Alternatives

No response

Additional context

No response

Copy link

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant