Exhaustive pcap testing

[philrz]

Issues like the one fixed in brimdata/super#462 remind us that we're likely to encounter pcap oddities in the wild. Even if we can't anticipate every corner case and have perfectly-crafted error handling for each one, ideally we'd be able to handle them gracefully.

To weed out known problems in advance, one thing we could do is throw as much diverse test data at it as we can. Some known pcap sources:

• https://code.wireshark.org/review/gitweb?p=wireshark.git;a=tree;f=test/captures;h=b8d1ddffc8defc7961c3e8194fe7c9cc3f6f286c;hb=HEAD
• https://www.netresec.com/?page=PcapFiles

Here's some of my own testing ideas for consideration:

• It seems a no-brainer would be to make sure brimcap analyze and brimcap index run on them without complaint.
• Use tshark to extract each udp/tcp flow from the original pcap (tcp.stream eq N etc.) and acquire each flow's 5-tuple and timestamp/duration information, then use that to construct a brimcap search command line that tries to extract the equivalent flow from the original pcap file, then confirm it succeeds and the mini pcap for the flow we extracted matches the one extracted via tshark