Forecast skill

[ecomodeller]

ModelSkill already supports forecast skill thanks to flexible grouping on aux variables.

Is there anyting else we need to streamline forecast skill?

• Data handling, what is the best format to preserve necessary metadata, e.g. time of forecast

[MortenRugbjerg]

• Forecast data file structure: It would be good if it was possible to extract forecast times series directly from the BlueCast dfsu forecast files named H12, B1 and B2. (And also use dfs0 files with the same naming convention.)
• Hitrate with a given "deviation" is an accepted accuracy measure by forecast clients. As it is already a part of ModelSkill I guess that it will automatically be a part of ForecastSkill.

[ecomodeller]

To be able to calculate lead time, we need the time of forecast in addition to the valid time (available in dfs{u,0}) files).

Since dfs files lacks support for arbitrary metadata, we can not encode time of forecast into the file, so this information has to be supplied to modelskill in some other way, csv?

time_of_forecast	filename
2024-04-01 18:12:12	foo_B1.dfsu
2024-04-01 18:12:12	foo_B2.dfsu

[MortenRugbjerg]

The "basetime" of the forecast (when the forecast cycle starts, not when the content of the file starts) is already a part of the file name, so should be available.

[jsmariegaard]

attrs?

[jpalm3r]

Is the skill() method able to handle multiple models and aux_items?

I was trying to write a simple example and I noticed this:

Notice that I could either compute the skill by model or by aux (lead_time), but not both. Am I doing something wrong?

[jpalm3r]

I have talked with @jsmariegaard and we have figured out how to do it: it was just a matter of writing cmp.skill(by = ["model", "lead_time"]).