Modify tacticSources

[austinmw]

With trtexec I need to add --tacticSources=-cublasLt,+cublas to compile my model, or else I get:

[TensorRT] INTERNAL ERROR: Assertion failed: cublasStatus == CUBLAS_STATUS_SUCCESS

How can I do this in torch2trt?

[jaybdub]

Hi @austinmw ,

Thanks for reaching out!

Currently this isn’t exposed, but it seems like a relatively straightforward feature for us to add. Particularly, after the following PR for DLA support is merged, torch2trt will use a builder config, so I can just pass an extra argument.

#507

I will try to notify you here of any updates, or if I discover anything along way.

Best,
John

[austinmw]

Thanks!

[JonasKonrad]

I have been asking for TRT/PyTorch version recommendations for torch2trt in Discussions ("version (mis)match overview"). However, there is none so far. This issue seems to be linked to such a recommendation, as the default tactics has been changed from cuBLAS to cuBLASLt as of TRT 7.2.1 (see here).
Can you confirm @austinmw ?

Edit: It's stated here, that this already applies for TRT 7.2(.0).

Best,
Jonas

[azuryl]

straightforward feature for us to add

@jaybdub hello jay can you tell me how to add it

Thank you

[azuryl]

With trtexec I need to add --tacticSources=-cublasLt,+cublas to compile my model, or else I get:

[TensorRT] INTERNAL ERROR: Assertion failed: cublasStatus == CUBLAS_STATUS_SUCCESS

How can I do this in torch2trt?
@austinmw
config = builder.create_builder_config()
config.set_tactic_sources(1 << int(trt.TacticSource.CUBLAS))