Skip to content

CTI resurrection Phase 2: autofit sync — factor-graph aggregator port #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jammy2211 merged 1 commit into from
Jul 17, 2026
Merged

CTI resurrection Phase 2: autofit sync — factor-graph aggregator port #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,10 @@ ecosystem via the CTI resurrection epic
visualization layer rewritten on the matplotlib **function API**, mirroring
PyAutoGalaxy: per-domain `plot/*_plots.py` function modules, config-gated
`model/plotter.py` orchestrators, `autocti/util/plot_utils.py` helpers) are
complete. Remaining: Phase 2 autofit sync (5 aggregator tests skipped pending
the `AnalysisFactor`/`FactorGraphModel` port), Phase 3 CI + ecosystem
plumbing, Phase 4 workspace update, Phase 5 workspace_test rebuild + release.
complete, as is Phase 2 (autofit sync: multi-dataset fits and the aggregator
run through `af.AnalysisFactor`/`af.FactorGraphModel`; the test suite has no
skips). Remaining: Phase 3 CI + ecosystem plumbing, Phase 4 workspace update,
Phase 5 workspace_test rebuild + release.

## arcticpy (read before installing)

Expand Down
342 changes: 183 additions & 159 deletions autocti/aggregator/fit_dataset_1d.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,159 +1,183 @@
from __future__ import annotations
from typing import TYPE_CHECKING, List, Optional

from autocti.aggregator.abstract import AggBase

if TYPE_CHECKING:
from autocti.clocker.abstract import AbstractClocker
from autocti.dataset_1d.fit import FitDataset1D

import autofit as af

from autocti.aggregator.dataset_1d import _dataset_1d_list_from


def _fit_dataset_1d_list_from(
fit: af.Fit,
instance: Optional[af.ModelInstance] = None,
use_dataset_full: bool = False,
clocker_list: Optional[AbstractClocker] = None,
) -> List[FitDataset1D]:
"""
Returns a list of `FitDataset1D` object from a `PyAutoFit` sqlite database `Fit` object.

The results of a model-fit can be stored in a sqlite database, including the following attributes of the fit:

- The masked dataset (e.g. data / noise map / pre cti data) as .fits files (contained in `dataset` folder).
- The clocker used to add CTI in the fit (`dataset/clocker.json`).
- The settings used for clocking CIT (contained in `dataset/settings_cti.json`).

Each individual attribute can be loaded from the database via the `fit.value()` method.

This method combines all of these attributes and returns a list of `FitDataset1D` objects, by loading the masked
dataset adding CTI to its pre-cti data via the cti model and clocking and fitting the model image to the dataset.

If multiple `Dataset1D` objects were fitted simultaneously via analysis summing, the `fit.child_values()` method
is instead used to load lists of the datasets, perform the fit and return a list of `FitDataset1D` objects.

If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to fit the full dataset instead.

Parameters
----------
fit
A `PyAutoFit` `Fit` object which contains the results of a model-fit as an entry in a sqlite database.
instance
A manual instance that overwrites the max log likelihood instance in fit (e.g. for drawing the instance
randomly from the PDF).
use_dataset_full
If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to load instead the full `Dataset1D` objects.
clocker_list
If input, overwrites the clocker used in the fit with a new clocker which is used to perform the fit.
"""

from autocti.dataset_1d.fit import FitDataset1D

dataset_list = _dataset_1d_list_from(fit=fit, use_dataset_full=use_dataset_full)

if clocker_list is None:
if not fit.children:
clocker_list = [fit.value(name="clocker")]
else:
clocker_list = fit.child_values(name="clocker")

if instance is not None:
cti = instance.cti
else:
cti = fit.instance.cti

post_cti_data_list = [
clocker.add_cti(data=dataset.pre_cti_data, cti=cti)
for dataset, clocker in zip(dataset_list, clocker_list)
]

return [
FitDataset1D(
dataset=dataset,
post_cti_data=post_cti_data,
)
for dataset, post_cti_data in zip(dataset_list, post_cti_data_list)
]


class FitDataset1DAgg(AggBase):
def __init__(
self,
aggregator: af.Aggregator,
use_dataset_full: bool = False,
clocker_list: Optional[List[AbstractClocker]] = None,
):
"""
Interfaces with an `PyAutoFit` aggregator object to create instances of `Dataset1D` objects from the results
of a model-fit.

The results of a model-fit can be stored in a sqlite database, including the following attributes of the fit:

- The masked dataset (e.g. data / noise map / pre cti data) as .fits files (contained in `dataset` folder).
- The clocker used to add CTI in the fit (`dataset/clocker.json`).
- The settings used for clocking CIT (contained in `dataset/settings_cti.json`).

The `aggregator` contains the path to each of these files, and they can be loaded individually. This class
can load them all at once and create a `FitDataset1D` object via the `_fit_dataset_1d_from` method.

This class's methods returns generators which create the instances of the `FitDataset1D` objects. This ensures
that large sets of results can be efficiently loaded from the hard-disk and do not require storing all
`Dataset1D` instances in the memory at once.

For example, if the `aggregator` contains 3 model-fits, this class can be used to create a generator which
creates instances of the corresponding 3 `Dataset1D` objects.

If multiple `Dataset1D` objects were fitted simultaneously via analysis summing, the `fit.child_values()` method
is instead used to load lists of the datasets, perform the fit and return a list of `FitDataset1D` objects.

If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to fit the full dataset instead.

This can be done manually, but this object provides a more concise API.

Parameters
----------
aggregator
A `PyAutoFit` aggregator object which can load the results of model-fits.
use_dataset_full
If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore
accessible to the database, the input `use_dataset_full` can be switched in to load instead the
full `Dataset1D` objects.
clocker_list
If input, overwrites the clocker used in the fit with a new clocker which is used to perform the fit.
"""
super().__init__(
aggregator=aggregator,
use_dataset_full=use_dataset_full,
clocker_list=clocker_list,
)

def object_via_gen_from(
self, fit, instance: Optional[af.ModelInstance] = None
) -> List[FitDataset1D]:
"""
Returns a generator of `FitDataset1D` objects from an input aggregator.

See `__init__` for a description of how the `FitDataset1D` objects are created by this method.

If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to fit the full dataset instead.

Parameters
----------
fit
A `PyAutoFit` `Fit` object which contains the results of a model-fit as an entry in a sqlite database.
cti
The CTI model used to add CTI to the dataset to perform the fit.
"""
return _fit_dataset_1d_list_from(
fit=fit,
instance=instance,
use_dataset_full=self.use_dataset_full,
clocker_list=self.clocker_list,
)
from __future__ import annotations
from typing import TYPE_CHECKING, List, Optional

from autocti.aggregator.abstract import AggBase

if TYPE_CHECKING:
from autocti.clocker.abstract import AbstractClocker
from autocti.dataset_1d.fit import FitDataset1D

import autofit as af

from autocti.aggregator.dataset_1d import _dataset_1d_list_from


def _cti_list_from(source, total_datasets: int):
"""
Extract one CTI model per dataset from a model instance.

A single-analysis instance exposes ``instance.cti`` directly; a factor-graph
instance (multi-dataset fit) is an indexed collection with one child
instance per factor.
"""
if hasattr(source, "cti"):
return [source.cti] * total_datasets

# A factor-graph instance also carries the FactorGraphModel itself as a
# trailing child, so only children with a CTI model are taken.
cti_list = [child.cti for child in source if hasattr(child, "cti")]

if len(cti_list) != total_datasets:
raise ValueError(
f"The instance contains {len(cti_list)} CTI models but the fit has "
f"{total_datasets} datasets."
)

return cti_list


def _fit_dataset_1d_list_from(
fit: af.Fit,
instance: Optional[af.ModelInstance] = None,
use_dataset_full: bool = False,
clocker_list: Optional[AbstractClocker] = None,
) -> List[FitDataset1D]:
"""
Returns a list of `FitDataset1D` object from a `PyAutoFit` sqlite database `Fit` object.

The results of a model-fit can be stored in a sqlite database, including the following attributes of the fit:

- The masked dataset (e.g. data / noise map / pre cti data) as .fits files (contained in `dataset` folder).
- The clocker used to add CTI in the fit (`dataset/clocker.json`).
- The settings used for clocking CIT (contained in `dataset/settings_cti.json`).

Each individual attribute can be loaded from the database via the `fit.value()` method.

This method combines all of these attributes and returns a list of `FitDataset1D` objects, by loading the masked
dataset adding CTI to its pre-cti data via the cti model and clocking and fitting the model image to the dataset.

If multiple `Dataset1D` objects were fitted simultaneously via analysis summing, the `fit.child_values()` method
is instead used to load lists of the datasets, perform the fit and return a list of `FitDataset1D` objects.

If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to fit the full dataset instead.

Parameters
----------
fit
A `PyAutoFit` `Fit` object which contains the results of a model-fit as an entry in a sqlite database.
instance
A manual instance that overwrites the max log likelihood instance in fit (e.g. for drawing the instance
randomly from the PDF).
use_dataset_full
If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to load instead the full `Dataset1D` objects.
clocker_list
If input, overwrites the clocker used in the fit with a new clocker which is used to perform the fit.
"""

from autocti.dataset_1d.fit import FitDataset1D

dataset_list = _dataset_1d_list_from(fit=fit, use_dataset_full=use_dataset_full)

if clocker_list is None:
if not fit.children:
clocker_list = [fit.value(name="clocker")]
else:
clocker_list = fit.child_values(name="clocker")

cti_list = _cti_list_from(
source=instance if instance is not None else fit.instance,
total_datasets=len(dataset_list),
)

post_cti_data_list = [
clocker.add_cti(data=dataset.pre_cti_data, cti=cti)
for dataset, clocker, cti in zip(dataset_list, clocker_list, cti_list)
]

return [
FitDataset1D(
dataset=dataset,
post_cti_data=post_cti_data,
)
for dataset, post_cti_data in zip(dataset_list, post_cti_data_list)
]


class FitDataset1DAgg(AggBase):
def __init__(
self,
aggregator: af.Aggregator,
use_dataset_full: bool = False,
clocker_list: Optional[List[AbstractClocker]] = None,
):
"""
Interfaces with an `PyAutoFit` aggregator object to create instances of `Dataset1D` objects from the results
of a model-fit.

The results of a model-fit can be stored in a sqlite database, including the following attributes of the fit:

- The masked dataset (e.g. data / noise map / pre cti data) as .fits files (contained in `dataset` folder).
- The clocker used to add CTI in the fit (`dataset/clocker.json`).
- The settings used for clocking CIT (contained in `dataset/settings_cti.json`).

The `aggregator` contains the path to each of these files, and they can be loaded individually. This class
can load them all at once and create a `FitDataset1D` object via the `_fit_dataset_1d_from` method.

This class's methods returns generators which create the instances of the `FitDataset1D` objects. This ensures
that large sets of results can be efficiently loaded from the hard-disk and do not require storing all
`Dataset1D` instances in the memory at once.

For example, if the `aggregator` contains 3 model-fits, this class can be used to create a generator which
creates instances of the corresponding 3 `Dataset1D` objects.

If multiple `Dataset1D` objects were fitted simultaneously via analysis summing, the `fit.child_values()` method
is instead used to load lists of the datasets, perform the fit and return a list of `FitDataset1D` objects.

If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to fit the full dataset instead.

This can be done manually, but this object provides a more concise API.

Parameters
----------
aggregator
A `PyAutoFit` aggregator object which can load the results of model-fits.
use_dataset_full
If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore
accessible to the database, the input `use_dataset_full` can be switched in to load instead the
full `Dataset1D` objects.
clocker_list
If input, overwrites the clocker used in the fit with a new clocker which is used to perform the fit.
"""
super().__init__(
aggregator=aggregator,
use_dataset_full=use_dataset_full,
clocker_list=clocker_list,
)

def object_via_gen_from(
self, fit, instance: Optional[af.ModelInstance] = None
) -> List[FitDataset1D]:
"""
Returns a generator of `FitDataset1D` objects from an input aggregator.

See `__init__` for a description of how the `FitDataset1D` objects are created by this method.

If a `dataset_full` is input into the `Analysis` class when a model-fit is performed and therefore accessible
to the database, the input `use_dataset_full` can be switched in to fit the full dataset instead.

Parameters
----------
fit
A `PyAutoFit` `Fit` object which contains the results of a model-fit as an entry in a sqlite database.
cti
The CTI model used to add CTI to the dataset to perform the fit.
"""
return _fit_dataset_1d_list_from(
fit=fit,
instance=instance,
use_dataset_full=self.use_dataset_full,
clocker_list=self.clocker_list,
)
Loading
Loading