Examples

[kidder]

We want to set up a structure for examples/benchmarks/regression_tests that would serve as:

• examples for new users
• a set of things that developers can run to check for bugs/anti-optimizations when making significant changes
• preliminary benchmarks when porting to a new machine.

Start with one example each of an elliptic solve, DG evolution, subcell evolution, CCE evolution.
I think each case should have:

• a tutorial walk-through including plots of expected output
• input files
• single node and multi-node submit scripts for clusters
• a script for verifying output.

I imagine this setup will evolve over time, but I think we could start with a few examples from our recent papers.

[kidder]

From discussion with @nilsvu and @nilsdeppe we will make a directory doc/examples/EXAMPLE to hold the input files, submission scripts and verification scripts used for each example.

[kidder]

We will want:

• To keep PR reviews light in order to get the examples in quickly rather than perfect
• Jupyter notebooks could be a nice format
• Should be integrated with our documentation

[kidder]

Ideally we would like to periodically (weekly/biweekly/monthly?) run the examples on all clusters in order to keep them up to date as well as catch performance regressions.

Could use existing testing structure with different string than Parse Execute or ExecuteAndCheckOutput. say Regression

[kidder]

We do not want to store large output files and compare differences as in SpEC

[nilsvu]

Here's a first example: #3820

[nilsvu]

I believe we can use nbconvert to convert Jupyter notebooks to markdown files, including their cell output and rendered plots etc, which then get picked up by Doxygen automatically. We can run nbconvert over all ipynb files in docs/Examples as part of the doc and doc-check targets.

[nilsdeppe]

That sounds reasonable. We'll just want to make sure we can also build the docs when nbconvert isn't installed stubbing/eliding the examples. But other than that, I think that sounds like a good idea, especially if we can set it up in a way that we only run nbconvert if the notebooks have changed. Building docs already takes way longer than I'd like :(