torch2trt will impact the normal pytorch?

[dancingpipi]

in pytorch, I can do the below code right:

a = torch.ones((4, 3, 13, 13), dtype=torch.uint8)
b = torch.rand((4, 3, 13, 13))

c = b[a]

but if I use torch2trt to convert model, then execute the code. error whill occur:
IndexError: too many indices for tensor of dimension 4

Will any one reproduce this error.

Wait for your help~

[jaybdub]

Hi z13974509906,

Thanks for reaching out! I would like to try and reproduce this issue.

Do you mind sharing the model that you were attempting to convert?

Best,
John

[dancingpipi]

Hi z13974509906,

Thanks for reaching out! I would like to try and reproduce this issue.

Do you mind sharing the model that you were attempting to convert?

Best,
John

I have follow pytorch yolov3 . I use torch2trt convert the backbone.
After convert, pytorch became abnormal as I mention: b[a] will cause an indexerror, so I have to change it to b[a.tolist()].

If I don't convert the backbone by torch2trt, b[a] work properly

[dancingpipi]

Hi z13974509906,

Thanks for reaching out! I would like to try and reproduce this issue.

Do you mind sharing the model that you were attempting to convert?

Best,
John

you can also test torchvison.models.resnet.resnet50(), it can reproduce this issue

[dancingpipi]

any update?

[QZ-cmd]

IndexError: too many indices for tensor of dimension 2
Hello i also found the same problem, may i have any solution

[dancingpipi]

IndexError: too many indices for tensor of dimension 2
Hello i also found the same problem, may i have any solution

Uss tensor.tolist() may solve it. But it won't work everywhere

[QZ-cmd]

Hello where is it placed?for example x=randn（1,3,244,244）.cuda().tolist()，
Can you tell me exactly where to change? thanks

[dancingpipi]

Hello where is it placed?for example x=randn（1,3,244,244）.cuda().tolist()，
Can you tell me exactly where to change? thanks

no, I found it only impact index. for example : b[a] should change to b[a.tolist()]

[QZ-cmd]

Hello, do you have an email? I have some questions for you. Thank you

[zhengjiawen]

I have the same problem, add tolist() can solve it, But I don't know why?

[QZ-cmd]

the same problem, add tolist() can solve it, But I don't kno

Can you tell me where to add tolist (), I didn't solve the problem，thanks！

[zhengjiawen]

the same problem, add tolist() can solve it, But I don't kno

Can you tell me where to add tolist (), I didn't solve the problem，thanks！

add tolist() in your index tensor array

[dancingpipi]

the same problem, add tolist() can solve it, But I don't kno

Can you tell me where to add tolist (), I didn't solve the problem，thanks！

I may be powerless. The only way to solve the problem is debugging step by step.

I recommend pdb or code

[chebbyChefNEQ]

I think this is the same as #270

[PressEtoRace]

I have the same problem. I debug the code on Nvidia jetson AGX， after debugging, I think torch2trt will really impact the original pytorch. Can anyone solve it? Or tell me the reason for the problem. Thanks!

[baheytharwat]

@dancingpipi @PressEtoRace Did you succeed in optimizing inference time of YOLOv3? if yes, Do you detect all objects well with a good accuracy? because the model I am working on can detect only one object!

[PressEtoRace]

@dancingpipi @PressEtoRace Did you succeed in optimizing inference time of YOLOv3? if yes, Do you detect all objects well with a good accuracy? because the model I am working on can detect only one object!
I've had problems like yours. I think this is also caused by the influence of torch2trt on pytorch. My solution is to modify the NMS code in yolov3. You can print the size of the data after each operation in NMS, and you should find the problem.

[twmht]

same problem

[twmht]

I have also found out when running mask rcnn with mmdetection, it would throw cuda error at torch.tensor.any()

RuntimeError: CUDA error: an illegal memory access was encountered
terminate called after throwing an instance of 'c10::Error'
  what():  CUDA error: an illegal memory access was encountered
Exception raised from create_event_internal at ../c10/cuda/CUDACachingAllocator.cpp:687 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7f24d0f949ab in /home/acer/nfs-share/pytorch/torch/lib/libc10.so)
frame #1: c10::cuda::CUDACachingAllocator::raw_delete(void*) + 0xc10 (0x7f24d11d7280 in /home/acer/nfs-share/pytorch/torch/lib/libc10_cuda.so)
frame #2: c10::TensorImpl::release_resources() + 0x4d (0x7f24d0f7ce0d in /home/acer/nfs-share/pytorch/torch/lib/libc10.so)
frame #3: <unknown function> + 0x559d82 (0x7f24ed5eed82 in /home/acer/nfs-share/pytorch/torch/lib/libtorch_python.so)
<omitting python frames>
frame #19: __libc_start_main + 0xe7 (0x7f2538809b97 in /lib/x86_64-linux-gnu/libc.so.6)
frame #20: _start + 0x2a (0x56250f80577a in /home/acer/.pyenv/versions/pytorch_build/bin/python)

but after commenting out the converter of getitem (https://github.com/NVIDIA-AI-IOT/torch2trt/blob/master/torch2trt/converters/getitem.py#L32), it would work fine.

I think overriding torch.getitem is dangerous, since other torch api would use that. why not convert torch.narrow (https://pytorch.org/docs/stable/tensors.html)

[1309123499]

I have also found out when running mask rcnn with mmdetection, it would throw cuda error at torch.tensor.any()

RuntimeError: CUDA error: an illegal memory access was encountered
terminate called after throwing an instance of 'c10::Error'
  what():  CUDA error: an illegal memory access was encountered
Exception raised from create_event_internal at ../c10/cuda/CUDACachingAllocator.cpp:687 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7f24d0f949ab in /home/acer/nfs-share/pytorch/torch/lib/libc10.so)
frame #1: c10::cuda::CUDACachingAllocator::raw_delete(void*) + 0xc10 (0x7f24d11d7280 in /home/acer/nfs-share/pytorch/torch/lib/libc10_cuda.so)
frame #2: c10::TensorImpl::release_resources() + 0x4d (0x7f24d0f7ce0d in /home/acer/nfs-share/pytorch/torch/lib/libc10.so)
frame #3: <unknown function> + 0x559d82 (0x7f24ed5eed82 in /home/acer/nfs-share/pytorch/torch/lib/libtorch_python.so)
<omitting python frames>
frame #19: __libc_start_main + 0xe7 (0x7f2538809b97 in /lib/x86_64-linux-gnu/libc.so.6)
frame #20: _start + 0x2a (0x56250f80577a in /home/acer/.pyenv/versions/pytorch_build/bin/python)

but after commenting out the converter of getitem (https://github.com/NVIDIA-AI-IOT/torch2trt/blob/master/torch2trt/converters/getitem.py#L32), it would work fine.

I think overriding torch.getitem is dangerous, since other torch api would use that. why not convert torch.narrow (https://pytorch.org/docs/stable/tensors.html)

would you please show how to solve this problem more clearly? thanks!

[SrivastavaKshitij]

Tried this example with PR #691 and there is no error.