Gather_xxx requires foreign node optimizations incompatible with data-dependent shapes, so treating as error.

[kfiring]

Description

Description

convert onnx model to tensorrt engine error in tensorrt 8.5.3.1, no such error in 8.6(but has other errors in 8.6, one of those issued here and got no replies for days, that's why i turn back to tensorrt 8.5 to give it a try, but still no luck...)

[04/04/2023-12:35:54] [E] Error[9]: [myelinBuilderUtils.cpp::getMyelinSupportType::1106] Error Code 9: Internal Error (Gather_3669 requires foreign node optimizations incompatible with data-dependent shapes, so treating as error.)
[04/04/2023-12:35:54] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[04/04/2023-12:35:54] [E] Engine could not be created from network
[04/04/2023-12:35:54] [E] Building engine failed
[04/04/2023-12:35:54] [E] Failed to create engine from model or file.
[04/04/2023-12:35:54] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8503] # trtexec --onnx=updated_model.onnx --saveEngine=updated_model.trt --minShapes=image:0x240x320x3 --optShapes=image:1x480x640x3 --maxShapes=image:1x480x640x3 --preview=-fasterDynamicShapes0805 --buildOnly --verbose

In netron, i can see Gather_3669 is after a NMS node whose output shape is indeed depend on it's inputs(as i described in here). so from the above error message, is "supporing data-dependent shapes" a new feature in 8.6(because no such error in 8.6)?

Environment

TensorRT Version: 8.5.3.1
NVIDIA GPU: RTX 3090
NVIDIA Driver Version: 470.129.06
CUDA Version: 11.2
CUDNN Version: 8.6
Operating System: ubuntu 20.04
Python Version (if applicable): 3.8
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 1.9
Baremetal or Container (if so, version): container

Relevant Files

Steps To Reproduce

Environment

TensorRT Version:
NVIDIA GPU:
NVIDIA Driver Version:
CUDA Version:
CUDNN Version:
Operating System:
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):

Relevant Files

Steps To Reproduce

[zerollzeng]

We don't support data-dependent shapes for plugin currently. The output shape must be able to compute from inputs' shape.

[kfiring]

We don't support data-dependent shapes for plugin currently. The output shape must be able to compute from inputs' shape.

please take look at my other issue, in 8.6 i did not encounter this error.

[Egorundel]

I have a similar problem. Wrote an issue here: NVIDIA/TensorRT/issues/3372

[kamiLight]

We don't support data-dependent shapes for plugin currently. The output shape must be able to compute from inputs' shape.

what's the meaning of data-dependent shapes, could you please provide some exsamples? thanks!

[demuxin]

We don't support data-dependent shapes for plugin currently. The output shape must be able to compute from inputs' shape.

what's the meaning of data-dependent shapes, could you please provide some exsamples? thanks!

Like the NMS function, the number of output boxes is uncertain. The number of output boxes depends on the data specific to the input boxes.

[demuxin]

We don't support data-dependent shapes for plugin currently. The output shape must be able to compute from inputs' shape.

Hi @zerollzeng , is this feature currently supported ?

[linc-nv]

TensorRT is adding DDS support in native TensorRT (ETA about April, 25)