Add prepare function for condition-level object preparation

[mronkko]

Summary

Implements #74 by adding a prepare parameter to runSimulation() that executes once per simulation condition to prepare/modify fixed_objects before replications run.

Implementation

The prepare function:

• Accepts condition and fixed_objects as arguments
• Returns the modified fixed_objects
• Is called in Analysis() function (before replications, alongside where summarise is called after replications)
• Executes once per design row (condition)
• Modified fixed_objects is passed to all replications for that condition

Use Case

Pre-compute expensive condition-specific objects (design matrices, correlation matrices, lookup tables) once per condition instead of per replication. This avoids both:

• Memory issues from pre-computing objects for all conditions upfront
• Performance issues from recomputing expensive objects for every replication

Example Usage

Design <- createDesign(N = c(100, 1000, 10000))

prepare <- function(condition, fixed_objects) {
    # Pre-compute expensive condition-specific objects
    fixed_objects$design_matrix <- matrix(rnorm(condition$N * 10), ncol=10)
    fixed_objects$lookup_table <- compute_expensive_lookup(condition$N)
    return(fixed_objects)
}

generate <- function(condition, fixed_objects) {
    # Use prepared objects from fixed_objects
    X <- fixed_objects$design_matrix
    y <- X %*% rnorm(10) + rnorm(nrow(X))
    data.frame(y=y, X)
}

runSimulation(Design, replications=1000,
              prepare=prepare,
              generate=generate,
              analyse=analyse,
              summarise=summarise)

Changes

• Added prepare parameter to runSimulation() function signature
• Added comprehensive documentation for the prepare parameter
• Added validation for prepare function signature (must include condition, fixed_objects)
• Added prepare parameter to Analysis() function
• Implemented prepare call in Analysis() (once per condition, before replication loop)
• Added parallel cluster export for prepare when provided
• Added prepare globals to check.globals functionality
• Full backward compatibility (defaults to NULL)

Testing

✅ Prepare function correctly modifies fixed_objects per condition
✅ Modified objects available in all user functions
✅ Backward compatibility maintained (existing code works unchanged)
✅ Proper error handling when prepare fails

[philchalmers]

I generally like this structure now, thanks. The use case is fine, but maybe not the best way to think about how to use this in the documentation.

The way that I see this being useful is if within the prepare() definition something like fixed_objects$expensive_stuff <- readRDS('prepare/expensive_stuff') were used. The main reason is that information inside of prepare() is not returned by runSimulation() as the objects are expected to eat a good amount of RAM, so you wouldn't want these stored. Moreover, you'd certainty want to know what the information in prepare() actually look like given that they are a key component of the experiment (hence, we should note that any use of random number generation will be lost with this approach, and therefore saving RDS objects beforehand would be a more reasonable strategy).

As and aside, I particularly like this readRDS() idea in situations where binary files are precompiled locally and distributed on the cluster, as that should be considered a set once and forget it part of the codebase.

[mronkko]

I implemented seed storing for prepare in the pull request.

Pre-generating and loading the prepared objects is a solution, but it is not always an ideal approach:

1. The pre-generation can be costly and thus better run on a cluster instead of a local computer. The files can also be large, making storage and transfer cumbersome.
2. For reproducibility by others (i.e. how easy the code is to run and undesrstand), it might be better to have one simulation file that does all preparation in one function instead of two functions for calculating and loading precalcultated results.

This change allows both use cases, 1) prepare as a loader and 2) prepare as a data generator shared with all replications.

[philchalmers]

I implemented seed storing for prepare in the pull request.

Pre-generating and loading the prepared objects is a solution, but it is not always an ideal approach:

1. The pre-generation can be costly and thus better run on a cluster instead of a local computer. The files can also be large, making storage and transfer cumbersome.

True, but this should be considered the exception rather than the rule. I was referring to highlighting this in the documentation as the object generation within prepare() is unnecessary for a wide majority of simulations. Moreover the prepare() step is run on a single core on the cluster, while all the prepare() functions across the design could easily be run in parallel locally or, for instance, on a SLURM landing node, and stored as individual and tractable objects. If the objects themselves are large I don't see why uploads to the cluster are going to be an issue, unless for some reason bandwidth is the issue. Of course, if the objects are so large that the they can only be stored temporarily on the distributed arrays then you're forced to used this approach, in which case tracking what the actual generated objects were at a later time will be a time consuming nightmare.....

2. For reproducibility by others (i.e. how easy the code is to run and undesrstand), it might be better to have one simulation file that does all preparation in one function instead of two functions for calculating and loading precalcultated results.

The two step can be performed using the usual source() approach early in the object preparation stage on or off the landing node, while prepare() does the ladder. I don't see the need to split more than is already available.

This change allows both use cases, 1) prepare as a loader and 2) prepare as a data generator shared with all replications.

Great, I think this is coming together. Could you update the NEWS.md file to reflect the two pulls, and switch your ctb status to aut in DESCRIPTION? A few tests should probably be added to the tests/ directory as well just to make sure this works consistently in future releases.

[philchalmers]

This should be moved to .on.Attach() as it affects the other .Random.seed instances too

[mronkko]

I pushed the change that moves RNG initialization to .onAttach()

[mronkko]

I added tests, updated documentation, added new release to NEWS and fixed my details in DESCRIPTION.