Standardized Hardware Controller Interface

[aferk]

To attach different devices to the ICON in the real-time loop (hardware worker), we would like to standardize the interface to the hardware, such that we offer

• setting data (experiment sequences)
• getting status (during runtime -> contains flag, status and extra data)
• getting result (after runtime -> contains flag, results)

To correctly describe the form of both results and status data, readout metadata can be generalized / used, which also allows storing in HDF5.

Device connection should be separated from the main ICON configuration file and stored in a separate configuration file detailing

• Device Identifier
• Hardware Controller (device specific implemementation)
• Device / Controller Specific: IP / Device Name / Hostname / ...

This could be the same for slow (non-real time) devices such as pydase services, where the Client is then just pydase.
How the connection to the device is made and what other information is provided in the configuration is dependent on the device itself.

Now we need to map output products of an experiment to a device and results / status to the HDF file. For the second step, readout metadata (or a generalization) should be used. For the first step we have several options:

• Use Pycrystal Hardware description
• Use Experiment config (somewhat cumbersome, most likely the same over all experiments)
• Use decorators on experiment functions
• Create new metadata objects from each experiment

[g-braeunlich]

Created a first draft of the hardware controller interface and moved the zedboard controller to an own file.
It is still the default and we still only can have 1 hardware controller.
Question: why do you want a separate config file and not integrate the specific config in the icon config as we did with the experiment library client?

[g-braeunlich]

Next Question (interface to the experiment library):

• I assume that one experiment will be bound to one hardware, or should it be possible to send the payload generated by generate_json_sequence to multiple hardware instances (of the same type)?
• Is ExperimentDict the right place to put the info which hardware controller to use for an experiment?

[aferk]

Question: why do you want a separate config file and not integrate the specific config in the icon config as we did with the experiment library client?

I specified a separate config, as ICON is not dependent on the hardware controller. In principle, ICON should be able to start and run without a hardware controller specified. This would allow the user to configure the hardware via GUI.

I assume that one experiment will be bound to one hardware, or should it be possible to send the payload generated by generate_json_sequence to multiple hardware instances (of the same type)?

Each experiment should be allowed to use multiple hardware controllers. As mentioned in the issue, we have several options on how to define this, but in principle it should be configurable which "payload" (generate_json_sequence) is send to which hardware. One payload can also be sent to multiple hardware controllers (which will be the case for now).

Is ExperimentDict the right place to put the info which hardware controller to use for an experiment?

With this context, yes, ExperimentDict needs to be extended to allow a mapping from "payload" to the hardware controller.

[g-braeunlich]

I specified a separate config, as ICON is not dependent on the hardware controller. In principle, ICON should be able to start and run without a hardware controller specified. This would allow the user to configure the hardware via GUI.

Not quite sure if I understand. All that would also be possible if we would use the present config file (we also can run icon without pycrystal now in theory and still configure whatever experiment library client in the present config, also everything also can be configured in the web ui). Of course I would be also fine with a separate config file. Just want to understand why.

Each experiment should be allowed to use multiple hardware controllers. ...

OK. Then by "Use Pycrystal Hardware description" you mean something that generate_json_sequence() should now return a dict:
{<hardware description>: <payload>} (or list of tuples)? The we would not need additional metadata in ExperimentDict.

[aferk]

All that would also be possible if we would use the present config file (we also can run icon without pycrystal now in theory and still configure whatever experiment library client in the present config, also everything also can be configured in the web ui). Of course I would be also fine with a separate config file. Just want to understand why.

Indeed, all that would be possible with one config file. In this sense it is a matter of taste, if we want one long config file or multiple divided into different topics. As I proposed to also store the connected pydase services in this new config file, I thought it would make sense to separate it from the main config file and not grow the main config file.
We can also keep this in one config file, if you prefer.

OK. Then by "Use Pycrystal Hardware description" you mean something that generate_json_sequence() should now return a dict:
{<hardware description>: <payload>} (or list of tuples)? The we would not need additional metadata in ExperimentDict.

Here I meant that we could add a configuration section in the hardware description file which is parsed by the PycrystalClient. This section defines which output then maps to which device, e.g.
{<device id>: <function>}
or more specific, for the Zedboard:
{"my_zedboard": "generate_json_sequence"}
Then, if I add a second Zedboard and a Fastino:

{"my_zedboard": "generate_json_sequence",
"my_zedboard2": "generate_json_sequence",
"my_fastino": "generate_waveforms",
}

This could also be done by decorators in the experiment, e.g.

@device("my_zedboard")
@device("my_zedboard2")
def generate_json_sequence()
    ...

Or by separate metadata generated from PycrystalClient, or by experiment config, as listed above.
I think the hardware description file or the decorators would be the easiest and most user friendly options.
However, we need to be able to deal with experiments running on a subset of devices, i.e. not all specified devices are used in all experiments.

[g-braeunlich]

How do we decide on which hardware a job runs?
Should this be selectable in the UI or will this be encoded in the experiment somehow?

[aferk]

How do we decide on which hardware a job runs?

A job always blocks all hardware (devices), independent of, if there is a payload for this hardware (device) or not.
At the moment there is no concept of concurrency planned or possible.

[g-braeunlich]

OK. So that means that if a user starts a job, the preprocessor will create a sequence which will be sent to all zedboards and will also create hardware instructions for all other hardware types which will be sent to all matching devices (e.g. fastino)?

[aferk]

The pre-processor will create multiple sequences / waveforms ("payloads") which are distributed to the devices.
One payload can go to one or multiple devices. For now, we will distribute one sequence payload to all zedboards, but in principle this could also be different sequences to each zedboard. As in, it is not a given that the same type of device receives the same payload.