'Tensor' object has no attribute '_trt' #15

gavrin-s · 2019-07-11T08:54:30Z

I get error AttributeError: 'Tensor' object has no attribute '_trt' when I meet MaxPool2d

/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py in wrapper(*args, **kwargs)
95
96 #print('%s : %s' % (method.qualname, converter.name))
---> 97 converter(ctx)
98
99 # convert to None so conversion will fail for unsupported layers

/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/converters/MaxPool2d.py in convert_MaxPool2d(ctx)
21
22 layer = ctx.network.add_pooling(
---> 23 input=input._trt, type=trt.PoolingType.MAX, window_size=kernel_size)
24 layer.stride = stride
25 layer.padding = padding

AttributeError: 'Tensor' object has no attribute '_trt'

The text was updated successfully, but these errors were encountered:

mtheise8 · 2019-07-11T13:16:54Z

I'm also getting this error when I meet AvgPool2d trying to convert the ESPNetV2 model here: https://github.com/sacmehta/EdgeNets

jaybdub · 2019-07-11T19:58:23Z

Hi All,

This is most likely because there Is a layer converter that has not been implemented. It is likely the layer before the one that threw the error (because the _trt was not set by the previous converter).

We’re focused on supporting the models in the README, but the coverage may increase over time.

@gavrin-s are you able to share the model you’re attempting to convert?

FYI, You can see the list of registered converters by typing

‘’’python
import torch2trt

print(torch2trt.CONVERTERS)
‘’’

It may be possible to add support for the unsupported layers by using the ‘’@tensorrt_converter’’ method described in the README.md.

Please let me know if this helps or you have any other questions.

Best,
John

vfdev-5 · 2019-07-25T17:03:18Z

HI @jaybdub ,

I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic.

Click to see the log

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-8-7f1400d9157d> in <module>
----> 1 model_trt = torch2trt.torch2trt(model_w, [data], fp16_mode=True)

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/torch2trt.py in torch2trt(module, inputs, input_names, output_names, log_level, max_batch_size, fp16_mode, max_workspace_size)
    250         ctx.add_inputs(inputs, input_names)
    251 
--> 252         outputs = module(*inputs)
    253 
    254         if not isinstance(outputs, tuple):

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

<ipython-input-5-405ea0677ccc> in forward(self, x)
      4         self.model = model
      5     def forward(self, x):
----> 6         return model(x)['out']

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/miniconda3/envs/py35/lib/python3.5/site-packages/torchvision/models/segmentation/_utils.py in forward(self, x)
     20         result = OrderedDict()
     21         x = features["out"]
---> 22         x = self.classifier(x)
     23         x = F.interpolate(x, size=input_shape, mode='bilinear', align_corners=False)
     24         result["out"] = x

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/container.py in forward(self, input)
     90     def forward(self, input):
     91         for module in self._modules.values():
---> 92             input = module(input)
     93         return input
     94 

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    491             result = self._slow_forward(*input, **kwargs)
    492         else:
--> 493             result = self.forward(*input, **kwargs)
    494         for hook in self._forward_hooks.values():
    495             hook_result = hook(self, input, result)

~/miniconda3/envs/py35/lib/python3.5/site-packages/torchvision/models/segmentation/deeplabv3.py in forward(self, x)
     90         for conv in self.convs:
     91             res.append(conv(x))
---> 92         res = torch.cat(res, dim=1)
     93         return self.project(res)

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/torch2trt.py in wrapper(*args, **kwargs)
     95 
     96             #print('%s : %s' % (method.__qualname__, converter.__name__))
---> 97             converter(ctx)
     98 
     99             # convert to None so conversion will fail for unsupported layers

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/converters/cat.py in convert_cat(ctx)
     12 
     13     output = ctx.method_return
---> 14     trt_inputs = [i._trt for i in inputs]
     15 
     16     layer = ctx.network.add_concatenation(inputs=trt_inputs)

~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/converters/cat.py in <listcomp>(.0)
     12 
     13     output = ctx.method_return
---> 14     trt_inputs = [i._trt for i in inputs]
     15 
     16     layer = ctx.network.add_concatenation(inputs=trt_inputs)

AttributeError: 'Tensor' object has no attribute '_trt'

Any ideas why it does not work ?

Packages Versions

``` # packages in environment at /root/miniconda3/envs/py35: # # Name Version Build Channel _libgcc_mutex 0.1 main attrs 19.1.0 pypi_0 pypi backcall 0.1.0 pypi_0 pypi blas 1.0 mkl bleach 3.1.0 pypi_0 pypi ca-certificates 2019.5.15 0 certifi 2018.8.24 py35_1 cffi 1.11.5 py35he75722e_1 cudatoolkit 10.0.130 0 decorator 4.4.0 pypi_0 pypi defusedxml 0.6.0 pypi_0 pypi entrypoints 0.3 pypi_0 pypi freetype 2.9.1 h8a8886c_1 intel-openmp 2019.4 243 ipykernel 5.1.1 pypi_0 pypi ipython 7.6.1 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi ipywidgets 7.5.0 pypi_0 pypi jedi 0.14.1 pypi_0 pypi jinja2 2.10.1 pypi_0 pypi jpeg 9b h024ee3a_2 jsonschema 3.0.1 pypi_0 pypi jupyter 1.0.0 pypi_0 pypi jupyter-client 5.3.1 pypi_0 pypi jupyter-console 6.0.0 pypi_0 pypi jupyter-core 4.5.0 pypi_0 pypi libedit 3.1.20181209 hc058e9b_0 libffi 3.2.1 hd88cf55_4 libgcc-ng 9.1.0 hdf63c60_0 libgfortran-ng 7.3.0 hdf63c60_0 libpng 1.6.37 hbc83047_0 libstdcxx-ng 9.1.0 hdf63c60_0 libtiff 4.0.10 h2733197_2 markupsafe 1.1.1 pypi_0 pypi mistune 0.8.4 pypi_0 pypi mkl 2019.4 243 nbconvert 5.5.0 pypi_0 pypi nbformat 4.4.0 pypi_0 pypi ncurses 6.1 he6710b0_1 ninja 1.8.2 py35h6bb024c_1 notebook 6.0.0 pypi_0 pypi numpy 1.14.2 py35hdbf6ddf_0 olefile 0.46 py35_0 openssl 1.0.2s h7b6447c_0 pandocfilters 1.4.2 pypi_0 pypi parso 0.5.1 pypi_0 pypi pexpect 4.7.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 5.2.0 py35heded4f4_0 pip 10.0.1 py35_0 prometheus-client 0.7.1 pypi_0 pypi prompt-toolkit 2.0.9 pypi_0 pypi ptyprocess 0.6.0 pypi_0 pypi pycparser 2.19 py35_0 pygments 2.4.2 pypi_0 pypi pyrsistent 0.15.3 pypi_0 pypi python 3.5.6 hc3d631a_0 python-dateutil 2.8.0 pypi_0 pypi pytorch 1.1.0 py3.5_cuda10.0.130_cudnn7.5.1_0 pytorch pyzmq 18.0.2 pypi_0 pypi qtconsole 4.5.2 pypi_0 pypi readline 7.0 h7b6447c_5 send2trash 1.5.0 pypi_0 pypi setuptools 40.2.0 py35_0 six 1.11.0 py35_1 sqlite 3.29.0 h7b6447c_0 terminado 0.8.2 pypi_0 pypi testpath 0.4.2 pypi_0 pypi tk 8.6.8 hbc83047_0 torch2trt 0.0.0 pypi_0 pypi torchvision 0.3.0 py35_cu10.0.130_1 pytorch tornado 6.0.3 pypi_0 pypi traitlets 4.3.2 pypi_0 pypi wcwidth 0.1.7 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi wheel 0.31.1 py35_0 widgetsnbextension 3.5.0 pypi_0 pypi xz 5.2.4 h14c3975_4 zlib 1.2.11 h7b6447c_3 zstd 1.3.7 h0b5b093_0 ```

lawrencekiba · 2019-08-05T01:11:06Z

HI @jaybdub ,

I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic.

Click to see the log

Any ideas why it does not work ?

Packages Versions

I have a similar error when working to replicate the image classification example

Traceback (most recent call last): File "conversionog.py", line 14, in <module> model_trt = torch2trt(model, [data]) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 252, in torch2trt outputs = module(*inputs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.5/dist-packages/torchvision-0.3.0a0+6a834e9-py3.5-linux-x86_64.egg/torchvision/models/resnet.py", line 208, in forward File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 97, in wrapper converter(ctx) File "Desktop/torch2trt/torch2trt/converters/Linear.py", line 12, in convert_Linear layer = ctx.network.add_shuffle(input._trt) AttributeError: 'Tensor' object has no attribute '_trt'

lawrencekiba · 2019-08-05T01:20:46Z

HI @jaybdub ,
I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic.
Click to see the log
Any ideas why it does not work ?
Packages Versions

I have a similar error when working to replicate the image classification example

Traceback (most recent call last): File "conversionog.py", line 14, in <module> model_trt = torch2trt(model, [data]) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 252, in torch2trt outputs = module(*inputs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.5/dist-packages/torchvision-0.3.0a0+6a834e9-py3.5-linux-x86_64.egg/torchvision/models/resnet.py", line 208, in forward File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 97, in wrapper converter(ctx) File "Desktop/torch2trt/torch2trt/converters/Linear.py", line 12, in convert_Linear layer = ctx.network.add_shuffle(input._trt) AttributeError: 'Tensor' object has no attribute '_trt'

Tried running commands in https://github.com/NVIDIA-AI-IOT/torch2trt#convert and encountered similar errors too.

kawa23 · 2019-08-13T03:11:16Z

Hi All,

This is most likely because there Is a layer converter that has not been implemented. It is likely the layer before the one that threw the error (because the _trt was not set by the previous converter).

We’re focused on supporting the models in the README, but the coverage may increase over time.

@gavrin-s are you able to share the model you’re attempting to convert?

FYI, You can see the list of registered converters by typing

‘’’python
import torch2trt

print(torch2trt.CONVERTERS)
‘’’

It may be possible to add support for the unsupported layers by using the ‘’@tensorrt_converter’’ method described in the README.md.

Please let me know if this helps or you have any other questions.

Best,
John

Hi, @jaybdub
I can't find the Upsample and LeakyReLU at torch2trt.CONVERTERS, is it still not supported yet?

As you said, it can be solved by using the @tensorrt_converter, but I don't know how to start, could you show a demo, thanks. BTW, I installed torch2trt without plugins.

jaybdub · 2019-08-13T04:58:11Z

Hi kawa23,

Thanks for reaching out.

We have implemented the interpolate plugin, which can accomplish Upsampling. You will need to install torch2trt with plugins as described in the README.

For LeakyReLU, the following converter should work

import numpy as np
import tensorrt
import torch2trt

logger = trt.Logger(trt.Logger.INFO)
trt.init_libnvinfer_plugins(logger, '')

@torch2trt.tensorrt_converter('torch.nn.functional.leaky_relu')
def convert_leaky_relu(ctx):
    input = ctx.method_args[0]
    output = ctx.method_return
    
    if len(ctx.method_args) > 1:
        negative_slope = ctx.method_args[1]
    elif 'negative_slope' in ctx.method_kwargs:
        negative_slope = ctx.method_kwargs['negative_slope']
    
    registry = trt.get_plugin_registry()
    
    creator = [c for c in registry.plugin_creator_list if c.name == 'LReLU_TRT'][0]
    lrelu_slope_field = trt.PluginField("neg_slope", np.array([negative_slope], dtype=np.float32), trt.PluginFieldType.FLOAT32)
    field_collection = trt.PluginFieldCollection([lrelu_slope_field])
    plugin = creator.create_plugin(name='LReLU_TRT', field_collection=field_collection)
            
    layer = ctx.network.add_plugin_v2(inputs=[input._trt], plugin=plugin)
                                  
    output._trt = layer.get_output(0)

After executing the above code, you should be able to convert the model as described in the README.

Please let me know if this works for you or if you run into any further issues. I'd be happy to take a look at your model if you continue to run into problems.

Best,
John

cloudhan · 2019-09-27T07:12:06Z

OK, after reading the code, I finally figure out what happens under the hood.
There is a ConversionContext, when enter it, some pytorch's methods are replaced with wrapped methods, the wrapped methods do:

run the original method (to driven the conversion process)
do the conversion

problem happens when you encounter some unhooked method, they will silently fail the latter conversion, some part is unconnected. Main issue is not the failure in @gavrin-s 's case, but is, you don't know what missing hook trigger the failure, only looking into the original pytorch code will give you the insight...

jaybdub · 2019-09-29T20:35:00Z

@cloudhan Your understanding is correct. Admittedly this makes it a bit harder to debug a missing converter, and currently in some cases reading the code may be necessary.

One tip that may help, is to check the grad_fn of the tensor which is missing the _trt attribute.
This is set for any non-leaf tensor requiring gradient. I believe you can check this by

Attempt conversion (should throw error)
```
model_trt = torch2trt(model, [data])
```
Launch debugger post-mortem
```
import pdb
pdb.pm()
```
Print grad_fn of tensor without _trt attribute.
```
p input.grad_fn
```
Manually search for torch method which corresponds to printed backward method

Perhaps we could log this automatically upon missing _trt error, to provide an extra hint when conversion fails.

Please let me know if this helps or you have any other questions or feedback.

Best,
John

wangguojun2018 · 2019-10-20T08:33:01Z

HI @jaybdub ,
I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic.
Click to see the log
Any ideas why it does not work ?
Packages Versions

I have a similar error when working to replicate the image classification example

Traceback (most recent call last): File "conversionog.py", line 14, in <module> model_trt = torch2trt(model, [data]) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 252, in torch2trt outputs = module(*inputs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.5/dist-packages/torchvision-0.3.0a0+6a834e9-py3.5-linux-x86_64.egg/torchvision/models/resnet.py", line 208, in forward File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 97, in wrapper converter(ctx) File "Desktop/torch2trt/torch2trt/converters/Linear.py", line 12, in convert_Linear layer = ctx.network.add_shuffle(input._trt) AttributeError: 'Tensor' object has no attribute '_trt'

I have the same problem, I open a issue in 124

kausainshah · 2020-07-01T08:50:45Z

AttributeError: 'Tensor' object has no attribute '_trt'
Even using the Convert script provided in README.md for alexnet

azuryl · 2021-12-24T17:45:28Z

@jaybdub

does torch2trt support nn.Parameter

when I covert efficientnet in https://github.com/shariqfarooq123/AdaBins/blob/2fb686a66a304f0a719bc53d77412460af97fd61/models/layers.py#L19
positional_encodings1 = self.positional_encodings[:embeddings1_shape, :]#.T.unsqueeze(0)
File "/home/delight-gpu/project/torch2trt/torch2trt/torch2trt.py", line 300, in wrapper
converter"converter"
File "/home/delight-gpu/project/torch2trt/torch2trt/converters/getitem.py", line 30, in convert_tensor_getitem
input_trt = input._trt
AttributeError: 'Parameter' object has no attribute '_trt'

the self.positional_encodings = nn.Parameter(torch.rand(500, embedding_dim), requires_grad=True)

prathik-naidu mentioned this issue Jun 22, 2020

check.py script not working DocF/YOLOv3-Torch2TRT#6

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

'Tensor' object has no attribute '_trt' #15

'Tensor' object has no attribute '_trt' #15

gavrin-s commented Jul 11, 2019

mtheise8 commented Jul 11, 2019

jaybdub commented Jul 11, 2019

vfdev-5 commented Jul 25, 2019

lawrencekiba commented Aug 5, 2019 •

edited

Loading

lawrencekiba commented Aug 5, 2019

kawa23 commented Aug 13, 2019 •

edited

Loading

jaybdub commented Aug 13, 2019 •

edited

Loading

cloudhan commented Sep 27, 2019 •

edited

Loading

jaybdub commented Sep 29, 2019 •

edited

Loading

wangguojun2018 commented Oct 20, 2019

kausainshah commented Jul 1, 2020

azuryl commented Dec 24, 2021 •

edited

Loading

'Tensor' object has no attribute '_trt' #15

'Tensor' object has no attribute '_trt' #15

Comments

gavrin-s commented Jul 11, 2019

mtheise8 commented Jul 11, 2019

jaybdub commented Jul 11, 2019

vfdev-5 commented Jul 25, 2019

lawrencekiba commented Aug 5, 2019 • edited Loading

lawrencekiba commented Aug 5, 2019

kawa23 commented Aug 13, 2019 • edited Loading

jaybdub commented Aug 13, 2019 • edited Loading

cloudhan commented Sep 27, 2019 • edited Loading

jaybdub commented Sep 29, 2019 • edited Loading

wangguojun2018 commented Oct 20, 2019

kausainshah commented Jul 1, 2020

azuryl commented Dec 24, 2021 • edited Loading

lawrencekiba commented Aug 5, 2019 •

edited

Loading

kawa23 commented Aug 13, 2019 •

edited

Loading

jaybdub commented Aug 13, 2019 •

edited

Loading

cloudhan commented Sep 27, 2019 •

edited

Loading

jaybdub commented Sep 29, 2019 •

edited

Loading

azuryl commented Dec 24, 2021 •

edited

Loading