Create a `ReservoirComputingBenchmarks` library

[MartinuzziFrancesco]

It would be nice to provide a streamlined way to user for testing the models they build with RC.jl. I think setting up a library with some basic benchmark functionalities should be relatively quick and straightforward. Take RCBench as a reference point for the initial tasks:

• Nonlinear Transformation
• Nonlinear Auto-Regressive Moving Average
• Memory Capacity
• Nonlinear Memory
• Information Processing Capacity
• Sin(x) Approximation
• Kernel Rank
• Generalization Rank

Also I would need to update this issue linking the specific papers

[Saswatsusmoy]

Hey @MartinuzziFrancesco

I went through RCBench and the key papers behind each task (Jaeger 2002, Dambre et al. 2012, Legenstein & Maass 2007, etc.) so I have a decent picture of what's involved. Before I dive in though, wanted to get your thoughts on a few things.

First off, should this be a separate registered package or just live inside this repo as an extension? I'm leaning towards a separate package since it keeps the core lightweight, but happy to go either way.

Also for the API, RCBench goes fully model-agnostic, just takes raw arrays. But since we're in the Julia/RC.jl ecosystem we could make it nicer, like passing a trained model directly and having the benchmark handle the rest. What do you think? Or maybe support both?

I probably wouldn't try to land all 8 tasks at once. I was thinking start with Memory Capacity, NARMA, and IPC as the core those cover the most ground (linear memory, joint memory + nonlinearity and full capacity decomposition). Then add NLT, Kernel Rank, Generalization Rank, etc. incrementally. Does that ordering make sense to you?

Oh and one more thing, should I bother with plotting in the first pass or just keep it computation-only and add visualization later?

Let me know and I can put together an initial design proposal.

[MartinuzziFrancesco]

Hi, thanks for looking into this! I think this should be a separate package but still under ReservoirComputing.jl, so it would be living in lib.

I think for starters a good approach would be to also make it model agnostic, that way they can be reused also in other settings. They may be used in RC literature, but they could still be useful for other fields. In a second moment we can dispatch the benchmarks to make them more suited to rc.jl.

Yes, I think those three are a good place to have a first version going. Adding other ones can be done incrementally like you suggested.

I wouldn't bother with the plots at all, I think that's something better left with the user. Maybe in a second time it could be an interesting direction to explore, but I would prioritize having a full set of benchmarks before the plotting.