Define Acceptable Ranges for Measurement Values

[nibanks]

We should define acceptable ranges for both the goodput and cross-traffic measurements. Successful values in those ranges would continue to be marked as green on the interop site. Successful values outside of those ranges would be marked as yellow or orange.

At least for now, we should probably go with conservative ranges. IMO, these should be good:

goodput: x >= 7500
cross-traffic: 3500 <= x <= 6500

Ideally, I'd prefer values closer to:

goodput: x >= 9000
cross-traffic: 4250 <= x <= 5750

[marten-seemann]

I was wondering if we could even go a step further. This many numbers are really hard to interpret.
What do you think of using something like a progress bar (left side: 0, right side: 10000) to display the transfer speed? Maybe we can even find a JavaScript library that allows us to display the standard deviation somehow.
We could also set the color of the progress bar based on the value, as you suggest.

[nibanks]

So, right side would be green and left side would be red (failure would be treated as zero), and we color scale between them? I'd be fine with that too, I think. I'd probably need to see what it looks like though before final sign off.

[marten-seemann]

Was thinking of something along those lines:

[nibanks]

I'd probably have to see it fully to be able to decide if I like it or not. But I'm open to the solution definitely.

[LPardue]

maybe @rmarx can transfer his qvis skills to this

[rmarx]

Something like @marten-seemann's mockup is not too difficult to do with an SVG template. With a minor amount of JavaScript (or potentially just CSS) I can add onhover to show the raw values. But please agree first on the approach :P

[marten-seemann]

I hacked a few lines of JavaScript (without any library) to get this:

No coloring yet, and no standard deviations.

[janaiyengar]

This looks great, except for the C runs. Those should be "distance from ideal" -- basically, (x-5000) -- not simply throughput. The sign matters since the QUIC connection can be too aggressive or too conservative.

But yeah, I love the visualizations! Maybe they could be colored based on their closeness to the ideal value? Basically map 0 -> 10M to Red -> Green? One idea is to set:

R = 255 - int(BW/10000 * 255)
G = 255 - R
B = 0

[marten-seemann]

We probably need to adjust the scale a bit. Showing 0 - 10000 doesn't really make sense. There's a big difference between an implementation that achieves 8500 and one that achieves 9500.
Maybe we could zoom in to the 8000 - 10000 range, and fail all tests that fall outside that range.