Relationship between optimization, targets, and parameters/distribution

[TimothyWillard]

Optimizers define optimization algorithms and consume a target and a simulator.

Targets define the target to optimize, be it likelihood, distance for likelihood free, etc. Should contain specification of how outputs of simulators translate to a single evaluable target.

There needs to be a way to specify which parameters to specify, and frequently the prior distribution for said parameter.

Complications arise because not all optimizers will support any given parameter/distribution.

Possible solutions are:

1. DistributionABC that subclasses ParameterABC and provides a likelihood/distance method.
2. Optimizers/targets can provide overrides for parameters/distributions in the case that they do not support something?

[pearsonca]

To my mind, our MVP Target is a likelihood.

For TargetProtocol, the callable signature definition, I'm proposing:

receive a simulated array of numbers (the engine output at this stage, but we should think about whether to insert a transformation external to Target.evaluate), a standard array of numbers, and then kwargs for potential parameters to calculation. Return an array of numbers.

I think this supports likelihood.

But: I'm not sure if we want standard as its own thing, or as a parameter? If we were doing cost optimization e.g. there isn't really a standard. Having it in the signature provides useful hints for one of our main cases (inference), but confuses the cases we want to get into (any other kind of optimization).

[pearsonca]

For optimizer, the appropriate view is analogous to the system + engine: an system is a function that maps state + parameters to "next" in some sense. An engine is a function that translates a system into a new function, which maps initial state, time points, and parameters into states-at-times.

An optimizer is a function translates the engine-yielded-translation-of-system + the target function into a new function of just parameters (and then finds optima).

hmm, but we kind of want to distinguish the finding-the-optima bit from evaluating the target function around the parameter space, because we definitely want e.g. the parameters and likelihood surface outputs.