[CSIT-1968] 3n-icx: two testbeds behave differently in some ipsec tests with AVF

[vvalderrv]

Description

The performance impact depends on tunnel count, the signal is strongest [0] at 1000tnl (not sure why not at 10000tnl).

The main (int-aes256gcm) ipsec tests are affected, but only with AVF driver, only 1518B tests (not IMIX). Only 2c and 4c tests (not 1c even if it uses two workers). Only MRR, not PDR.

Comparing telemetry using verify job on the two [1] testbeds [2], the most obvious symptom is Vectors/Call on the non-primary workers (RSS on AVF puts all dut-dut flows onto the first worker RXQ). As the bottleneck is decrypt on the first worker, it is not clear how non-primary workers affect that (and not on 2T1C).

[0] https://csit.fd.io/trending/#eNrtlk1uwyAQhU_jbqKRDLHrbLqIm3tUBE9iJP8QhiZ1T1_iWhpHUaVGirxiA4g38IZPbwH53uEHYfOW5GVSlIksTBWGZL1dhelsLaw7MPoLRJoeUVqBG5HqE6jzAbQbrO9B5GKzB6EBfW1sZiyhDtWp7xq6QNjZK0IwnQeFJPPXo26hde7qIt-vLtWnv7FkxdYDK382wvXKoeIDv_2x6pFmRv_vlm84ONUimW_ka8bXc4UOPGeivnX3g52pE4RiN1YsB99WEX4xQXgCfBmTfw9fLpR8GZP_KPznJT-Lyb-Hny2U_Cwm_1H4nPx899L1rh1_PWFNdX8B74xqaNr6AQUtkVA

[1] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-3n-icx/1092/log.html.gz#s1-s1-s1-s1-s1-t2-k2-k10-k9-k10-k1-k1-k1-k12

[2] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-3n-icx/1093/log.html.gz#s1-s1-s1-s1-s1-t2-k2-k10-k9-k10-k1-k1-k1-k12

Assignee

Vratko Polak

Reporter

Vratko Polak

Comments

No comments.

Original issue: https://jira.fd.io/browse/CSIT-1968