[CSIT-1951] Combination of AVF and vhost drops all 9000B packets

[vvalderrv]

Description

More details need to be added, but the stats are here [0].

[0] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2402-2n-icx/24/log.html.gz#s1-s1-s1-s1-s3-t6-k3-k7-k1-k1-k1-k8-k14-k1-k1-k1-k1

Assignee

Unassigned

Reporter

Vratko Polak

Comments

• vrpolak (Mon, 18 Nov 2024 13:10:26 +0000): Still present [4] in rls2410, with "buffer alloc error" preventing RX.

[4] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2410-2n-icx/30/log.html.gz#s1-s1-s1-s1-s4-t6-k3-k7-k1-k1-k1-k8-k14-k1-k1-k1-k1

• vrpolak (Tue, 30 Jul 2024 12:32:26 +0000):

  Still present [3] in rls2406.

Buffer alloc errors remind me of CSIT-1954, maybe the core issue is similar, just vhost makes the failure deterministic due to the additional vhost-user-input "no available buffer" errors?

[3] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2406-2n-icx/33/log.html.gz#s1-s1-s1-s1-s1-t6-k3-k7-k1-k1-k1-k8-k14-k1-k1-k1-k1

• vrpolak (Wed, 27 Mar 2024 10:14:24 +0000): That rare crash should be under a different ticket. This issue of deterministic drops [2] is similar to CSIT-1845 but that is random and rare, this is much more repeatable.

[2] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2402-2n-icx/24/log.html.gz#s1-s1-s1-s1-s1-t6-k3-k7-k1-k1-k1-k8-k14-k1-k1-k1-k1

• vrpolak (Tue, 19 Mar 2024 15:55:22 +0000): Core dump [1] suggests maybe out of memory?

[1] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-icx/335/log.html.gz#s1-s1-s1-s1-s1-t1-k3-k4-k1

Original issue: https://jira.fd.io/browse/CSIT-1951

[vvalderrv]

Still present [4] in rls2410, with "buffer alloc error" preventing RX.

[4] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2410-2n-icx/30/log.html.gz#s1-s1-s1-s1-s4-t6-k3-k7-k1-k1-k1-k8-k14-k1-k1-k1-k1

[vvalderrv]

Still present [3] in rls2406.
Buffer alloc errors remind me of CSIT-1954, maybe the core issue is similar, just vhost makes the failure deterministic due to the additional vhost-user-input "no available buffer" errors?

[3] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2406-2n-icx/33/log.html.gz#s1-s1-s1-s1-s1-t6-k3-k7-k1-k1-k1-k8-k14-k1-k1-k1-k1

[vvalderrv]

That rare crash should be under a different ticket. This issue of deterministic drops [2] is similar to CSIT-1845 but that is random and rare, this is much more repeatable.

[2] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-report-coverage-2402-2n-icx/24/log.html.gz#s1-s1-s1-s1-s1-t6-k3-k7-k1-k1-k1-k8-k14-k1-k1-k1-k1

[vvalderrv]

Core dump [1] suggests maybe out of memory?

[1] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-icx/335/log.html.gz#s1-s1-s1-s1-s1-t1-k3-k4-k1