How can I use the TRT engine that has interpolation layers in C++?

[bitwangdan]

@jaybdub hi， Can you give me some advice?

[jaybdub]

Hi bitwangdan,

Thanks for reaching out!

First, you'll need to export the engine

with open('my_model.engine', 'wb') as f:
    f.write(my_model.engine.serialize())

Next, you would need to load the model in C++. For this, I'd refer to the TensorRT documentation.

For plugins specifically, you'll need to link against the compiled library and register the plugins in C++. I haven't tested this personally.

Are you able to use TensorRT 7.1+? If so, interpolation is natively supported in TensorRT, and you should not need to link against any special plugin library.

Best,
John

[monsterlyg]

Hi bitwangdan,

Thanks for reaching out!

First, you'll need to export the engine

with open('my_model.engine', 'wb') as f:
    f.write(my_model.engine.serialize())

Next, you would need to load the model in C++. For this, I'd refer to the TensorRT documentation.

For plugins specifically, you'll need to link against the compiled library and register the plugins in C++. I haven't tested this personally.

Are you able to use TensorRT 7.1+? If so, interpolation is natively supported in TensorRT, and you should not need to link against any special plugin library.

Best,
John

I try to use the engine converted with torch2trt with plugins in c++. But actually, as you said, thr plugin should be registered in c++, how to figure out this problem? Do you have any instruction?