From 83bb668a23cec1c21d8b06b81d02084d06d756e2 Mon Sep 17 00:00:00 2001 From: Eric Neilsen Date: Fri, 10 Jul 2026 08:12:59 -0700 Subject: [PATCH] Fix table markdown --- maf/tutorial/01_Introduction_to_MAF.ipynb | 164 ++++++++++++++++++++-- 1 file changed, 154 insertions(+), 10 deletions(-) diff --git a/maf/tutorial/01_Introduction_to_MAF.ipynb b/maf/tutorial/01_Introduction_to_MAF.ipynb index 8afe99a..36d353a 100644 --- a/maf/tutorial/01_Introduction_to_MAF.ipynb +++ b/maf/tutorial/01_Introduction_to_MAF.ipynb @@ -2,6 +2,7 @@ "cells": [ { "cell_type": "markdown", + "id": "ad42e922", "metadata": {}, "source": [ "# Introduction to MAF" @@ -9,6 +10,7 @@ }, { "cell_type": "markdown", + "id": "9ee019f3", "metadata": {}, "source": [ " \n", @@ -20,6 +22,7 @@ }, { "cell_type": "markdown", + "id": "b30858d9", "metadata": {}, "source": [ "## 1. Basic concepts" @@ -27,6 +30,7 @@ }, { "cell_type": "markdown", + "id": "529a63ad", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] @@ -43,6 +47,7 @@ }, { "cell_type": "markdown", + "id": "a936351e", "metadata": {}, "source": [ "### 1.2 Essential elements" @@ -50,6 +55,7 @@ }, { "cell_type": "markdown", + "id": "50b3443c", "metadata": {}, "source": [ "#### Purpose\n", @@ -63,6 +69,7 @@ }, { "cell_type": "markdown", + "id": "86b39564", "metadata": {}, "source": [ "#### Metric values and slices\n", @@ -87,6 +94,7 @@ }, { "cell_type": "markdown", + "id": "1412b020", "metadata": {}, "source": [ "#### Plots and summary statistics\n", @@ -98,6 +106,7 @@ }, { "cell_type": "markdown", + "id": "0fa288a4", "metadata": {}, "source": [ "#### Basic elements of the MAF infrastructure\n", @@ -107,16 +116,17 @@ "| element | base class | description |\n", "|:------- |:---------- |:----------- |\n", "| database | `str` | The database provides access to an sqlite database of visits, usually generated by the operations simulater, opsim. In current versions of rubin_sim, the \"database\" is simply the name of the database, and is thus just a string. |\n", - "| constraint | `str` | The constraint is the content of an SQL `WHERE` clause that selects which visits from the visit database should be included in the calculation metrics. This can be `None`.|\n", - "| slicer | `rubin_sim.maf.slicers.BaseSlicer` | A *slicer* divides visits into subsets, similar to the SQL `GROUP BY` clause or the `pandas.DataFrame.groupby` method. Every MAF *slicer* is a python iterator. Each item returned by the iterator is a dictionary with two elements: `idxs`, a `numpy.array` of visit indices that fall in the slice; and `slicePoint`, which indicates the parameters of the slice. The value of the `slicePoint` depends on the specific slicer. For example, the `OneDSlicer`, which divides visits up into bins by a value (as for a histogram), returns `slicePoint`s that are dictionaries specifying the number and location of each bin; while a `HealpixSlicer` returns `slicePoint`s that specify the healpix nside used, the healpix id, and its RA and declination.\n", - "| metric | `rubin_sim.maf.metrics.BaseMetric` | A *metric* calculates a values for each *slice* returned by iterating over `slicer`. Each metric class defines a `run` method that takes a `numpy.recarray` of visits and a (perhaps optional) `slicePoint`, and returns a result, typically but not always scalar. For example, the `maxMetric` returns the maximum value of some parameter (specified in initialization of the metric) taken by visits in the supplied `numpy.recarary` of visits. Many metrics depend only on the visits in each slice, and can be used with arbitrary slicers. A few, though, use data in the `slicePoint`, and so may only be used with *slicers* whose `slicePoint`s include the data they require.\n", - "| plotter | `rubin_sim.maf.plots.BasePlotter` | A *plotter* builds `matplotlib.figure.Figure` that represents the `slicePoints` and values computed my *metrics* graphically. A *plotter* is a python callable object which takes as paramters the metric values (computed by a *metric*) and a *slicer*. Figure parameters can be customized using an optional `userPlotDict` parameter. *Slicers* often have a default set of plotters, so users may not need to specify them.\n", - "| metric bundle | `rubin_sim.maf.metricBundles.MetricBundle` | An instance of a *metric bundle* combines an instance of a constraint (which is allowed to be an empty string), a slicer, a metric, and (optionally) a list of plotters to form a set. If the plotters are not specified explicitly, the default set specified by the *slicer* are used. **The *metric bundle* is the primary unit of work in MAF,** containing the elements needed to specify how a metric is to be computed and presented, and storing the results and metadata (e.g. the instance of the *slicer* with its *slice points*) needed to interpret them. \n", - "| metric bundle group | `rubin_sim.maf.metricBundles.MetricBundleGroup` | A *metric bundle group* combines a *database*, a dictionary of *metric bundles*, and (optionally) a database and directory in which to store results. The `MetricBundleGroup.run_all` method queries the database for the necessary data, calculates metrics using each bundle group, and saves the results both within the `MetricBundle` instances and on disk. The `MetricBundleGroup.plot_all` method runs all plotters in all metric bundles. The purpose of a *metric bundle group* is to efficiently run *metric bundles*, \"filling out\" the bundles with metric values and computed slice points, while avoiding duplication of computations when multiple bundle groups have common elements (e.g. *slicers* and database queries). " + "| constraint | `str` | The constraint is the content of an SQL `WHERE` clause that selects which visits from the visit database should be included in the calculation metrics. This can be `None`. |\n", + "| slicer | `rubin_sim.maf.slicers.BaseSlicer` | A *slicer* divides visits into subsets, similar to the SQL `GROUP BY` clause or the `pandas.DataFrame.groupby` method. Every MAF *slicer* is a python iterator. Each item returned by the iterator is a dictionary with two elements: `idxs`, a `numpy.array` of visit indices that fall in the slice; and `slicePoint`, which indicates the parameters of the slice. The value of the `slicePoint` depends on the specific slicer. For example, the `OneDSlicer`, which divides visits up into bins by a value (as for a histogram), returns `slicePoint`s that are dictionaries specifying the number and location of each bin; while a `HealpixSlicer` returns `slicePoint`s that specify the healpix nside used, the healpix id, and its RA and declination. |\n", + "| metric | `rubin_sim.maf.metrics.BaseMetric` | A *metric* calculates a values for each *slice* returned by iterating over `slicer`. Each metric class defines a `run` method that takes a `numpy.recarray` of visits and a (perhaps optional) `slicePoint`, and returns a result, typically but not always scalar. For example, the `maxMetric` returns the maximum value of some parameter (specified in initialization of the metric) taken by visits in the supplied `numpy.recarary` of visits. Many metrics depend only on the visits in each slice, and can be used with arbitrary slicers. A few, though, use data in the `slicePoint`, and so may only be used with *slicers* whose `slicePoint`s include the data they require. |\n", + "| plotter | `rubin_sim.maf.plots.BasePlotter` | A *plotter* builds `matplotlib.figure.Figure` that represents the `slicePoints` and values computed my *metrics* graphically. A *plotter* is a python callable object which takes as paramters the metric values (computed by a *metric*) and a *slicer*. Figure parameters can be customized using an optional `userPlotDict` parameter. *Slicers* often have a default set of plotters, so users may not need to specify them. |\n", + "| metric bundle | `rubin_sim.maf.metricBundles.MetricBundle` | An instance of a *metric bundle* combines an instance of a constraint (which is allowed to be an empty string), a slicer, a metric, and (optionally) a list of plotters to form a set. If the plotters are not specified explicitly, the default set specified by the *slicer* are used. **The *metric bundle* is the primary unit of work in MAF,** containing the elements needed to specify how a metric is to be computed and presented, and storing the results and metadata (e.g. the instance of the *slicer* with its *slice points*) needed to interpret them. |\n", + "| metric bundle group | `rubin_sim.maf.metricBundles.MetricBundleGroup` | A *metric bundle group* combines a *database*, a dictionary of *metric bundles*, and (optionally) a database and directory in which to store results. The `MetricBundleGroup.run_all` method queries the database for the necessary data, calculates metrics using each bundle group, and saves the results both within the `MetricBundle` instances and on disk. The `MetricBundleGroup.plot_all` method runs all plotters in all metric bundles. The purpose of a *metric bundle group* is to efficiently run *metric bundles*, \"filling out\" the bundles with metric values and computed slice points, while avoiding duplication of computations when multiple bundle groups have common elements (e.g. *slicers* and database queries). |" ] }, { "cell_type": "markdown", + "id": "1816f37d", "metadata": {}, "source": [ "### 1.3 A typical MAF workflow" @@ -124,6 +134,7 @@ }, { "cell_type": "markdown", + "id": "347991b9", "metadata": {}, "source": [ "A typical workflow for calculating MAF metrics on a single run has the following stages:\n", @@ -142,6 +153,7 @@ }, { "cell_type": "markdown", + "id": "ddfbab2a", "metadata": {}, "source": [ "## 2. Notebook preliminaries" @@ -149,6 +161,7 @@ }, { "cell_type": "markdown", + "id": "991e4f2d", "metadata": {}, "source": [ "### 2.1 Installing MAF" @@ -156,6 +169,7 @@ }, { "cell_type": "markdown", + "id": "e96759fb", "metadata": {}, "source": [ "MAF is part of the `rubin_sim` product. Instructions for using conda to install `rubin_sim` can be found in the README of the the `rubin_sim` [github product](https://github.com/lsst/rubin_sim)." @@ -163,6 +177,7 @@ }, { "cell_type": "markdown", + "id": "49eee356", "metadata": {}, "source": [ "### 2.2 Developer aids" @@ -170,6 +185,7 @@ }, { "cell_type": "markdown", + "id": "d2b038c3", "metadata": {}, "source": [ "The following is a development style aid; only uncomment if developing the notebook:" @@ -178,6 +194,7 @@ { "cell_type": "code", "execution_count": 1, + "id": "ee9de6b0", "metadata": {}, "outputs": [], "source": [ @@ -188,6 +205,7 @@ }, { "cell_type": "markdown", + "id": "88538fc1", "metadata": {}, "source": [ "### 2.3 Import required python modules" @@ -195,6 +213,7 @@ }, { "cell_type": "markdown", + "id": "9ff7a217", "metadata": {}, "source": [ "This tutorial requires MAF itself, which can be imported thus:" @@ -203,6 +222,7 @@ { "cell_type": "code", "execution_count": 2, + "id": "5194cfaa", "metadata": {}, "outputs": [], "source": [ @@ -218,6 +238,7 @@ }, { "cell_type": "markdown", + "id": "5ad45f63", "metadata": {}, "source": [ "Show which version of MAF this notebook was last run with:" @@ -226,6 +247,7 @@ { "cell_type": "code", "execution_count": 3, + "id": "fc889550", "metadata": {}, "outputs": [ { @@ -246,6 +268,7 @@ }, { "cell_type": "markdown", + "id": "25b848c9", "metadata": {}, "source": [ "### 2.4 Set the storage directory for this notebook" @@ -253,6 +276,7 @@ }, { "cell_type": "markdown", + "id": "4d382757", "metadata": {}, "source": [ "This notebook will produce output files. By default (if `data_dir = None` in the cell below), the notebook will create a temporary directory for them. If a temporary directory is created, it will automatically be deleted when the notebook kernel is stopped or restard. If you want to keep the output, set `data_dir` in the cell below to the directory where you want to keep the output." @@ -261,6 +285,7 @@ { "cell_type": "code", "execution_count": 4, + "id": "0852daec", "metadata": {}, "outputs": [], "source": [ @@ -269,6 +294,7 @@ }, { "cell_type": "markdown", + "id": "5809c5a7", "metadata": {}, "source": [ "To use the current local directory as the `data_dir`, uncomment this cell:" @@ -277,6 +303,7 @@ { "cell_type": "code", "execution_count": 5, + "id": "7e9dfe16", "metadata": {}, "outputs": [], "source": [ @@ -285,6 +312,7 @@ }, { "cell_type": "markdown", + "id": "33af434e", "metadata": {}, "source": [ "Now, if we have not customized our `data_dir`, create a temporary directory to use for this notebook. **Note that all data in this temporary directory will be deleted when the notebook kernel is stopped or restarted.**" @@ -293,6 +321,7 @@ { "cell_type": "code", "execution_count": 6, + "id": "54bf3109", "metadata": {}, "outputs": [ { @@ -316,6 +345,7 @@ }, { "cell_type": "markdown", + "id": "1955d5a3", "metadata": {}, "source": [ "### 2.5 Get a database to work with" @@ -323,6 +353,7 @@ }, { "cell_type": "markdown", + "id": "75ac1692", "metadata": {}, "source": [ "To calculate metrics, we first need the data from simulations to run metrics on.\n", @@ -333,6 +364,7 @@ { "cell_type": "code", "execution_count": 7, + "id": "56c78062", "metadata": {}, "outputs": [ { @@ -353,6 +385,7 @@ }, { "cell_type": "markdown", + "id": "c2a874a8", "metadata": {}, "source": [ "It's also helpful to have a run name, a string that labels our simulation. Let's build one from the file name:" @@ -361,6 +394,7 @@ { "cell_type": "code", "execution_count": 8, + "id": "0a3f0513", "metadata": {}, "outputs": [ { @@ -383,6 +417,7 @@ }, { "cell_type": "markdown", + "id": "5ae43bcd", "metadata": {}, "source": [ "## 3. A detailed example: the histogram of airmass of visits in *g*" @@ -390,6 +425,7 @@ }, { "cell_type": "markdown", + "id": "02b233d1", "metadata": {}, "source": [ "### 3.1 Computing the metric" @@ -397,6 +433,7 @@ }, { "cell_type": "markdown", + "id": "0222e6be", "metadata": {}, "source": [ "In this simple example, we will create a histogram of the airmass of visits in *g* band." @@ -404,6 +441,7 @@ }, { "cell_type": "markdown", + "id": "e216a315", "metadata": {}, "source": [ "Identify the opsim sqlite database:" @@ -412,6 +450,7 @@ { "cell_type": "code", "execution_count": 9, + "id": "42f25eaa", "metadata": {}, "outputs": [], "source": [ @@ -420,6 +459,7 @@ }, { "cell_type": "markdown", + "id": "4c0cab93", "metadata": {}, "source": [ "Specify that we want to consider only *g* band visits using the *constraint*:" @@ -428,6 +468,7 @@ { "cell_type": "code", "execution_count": 10, + "id": "39b6b097", "metadata": {}, "outputs": [], "source": [ @@ -436,6 +477,7 @@ }, { "cell_type": "markdown", + "id": "781288e2", "metadata": {}, "source": [ "Then create a slicer that \"slices\" the visits by airmass:" @@ -444,6 +486,7 @@ { "cell_type": "code", "execution_count": 11, + "id": "2e0274c8", "metadata": {}, "outputs": [], "source": [ @@ -454,6 +497,7 @@ }, { "cell_type": "markdown", + "id": "6a74ff0d", "metadata": {}, "source": [ "We just want to count the number of visits in each bin:" @@ -462,6 +506,7 @@ { "cell_type": "code", "execution_count": 12, + "id": "ca967a45", "metadata": {}, "outputs": [], "source": [ @@ -470,6 +515,7 @@ }, { "cell_type": "markdown", + "id": "e10c6e86", "metadata": {}, "source": [ "Now, we group these together in a metric bundle. (We do not need to specify a plotter, because `maf.OneDSlicer` provides a suitable one by default.)" @@ -478,6 +524,7 @@ { "cell_type": "code", "execution_count": 13, + "id": "0ae30584", "metadata": {}, "outputs": [], "source": [ @@ -491,6 +538,7 @@ }, { "cell_type": "markdown", + "id": "21d5f199", "metadata": {}, "source": [ "The `run_name` is optional, but useful for constructing file names and storing metadata in output." @@ -498,6 +546,7 @@ }, { "cell_type": "markdown", + "id": "64ba3305", "metadata": {}, "source": [ "Combine the metric bundle with the database to form a *metric bundle group*. Furthermore, let's use the `out_dir` parameter to specify that the results be stored in our `data_dir`." @@ -506,6 +555,7 @@ { "cell_type": "code", "execution_count": 14, + "id": "8f630931", "metadata": {}, "outputs": [], "source": [ @@ -516,6 +566,7 @@ }, { "cell_type": "markdown", + "id": "722b44de", "metadata": {}, "source": [ "Actually calculate the metrics:" @@ -524,6 +575,7 @@ { "cell_type": "code", "execution_count": 15, + "id": "a26c2304", "metadata": {}, "outputs": [], "source": [ @@ -532,6 +584,7 @@ }, { "cell_type": "markdown", + "id": "e5d2fcd2", "metadata": {}, "source": [ "### 3.2 Making a plot" @@ -540,6 +593,7 @@ { "cell_type": "code", "execution_count": 16, + "id": "356b9929", "metadata": {}, "outputs": [ { @@ -559,6 +613,7 @@ }, { "cell_type": "markdown", + "id": "26abfa4c", "metadata": {}, "source": [ "Be default, `maf.MetricBundleGroup.plot_all` saves plots as pdf files in the current working directory. The save directory can be changed by setting the `out_dir` parameter when creating the `MetricBundleGroup`, and the output format by setting `figformat` parameter when calling its `plot_all` method." @@ -566,6 +621,7 @@ }, { "cell_type": "markdown", + "id": "ae1989d0", "metadata": {}, "source": [ "### 3.3 Using metric values and slice points directly" @@ -573,6 +629,7 @@ }, { "cell_type": "markdown", + "id": "ef68cce7", "metadata": {}, "source": [ "We are not limited to creating plots. The *metric bundle* objecs provide access to the metric values and *slice points* themselves, allowing us to use any python tools we like for further analysis or visualization." @@ -580,6 +637,7 @@ }, { "cell_type": "markdown", + "id": "83dc7004", "metadata": {}, "source": [ "These are typically in the form of a `numpy.ma.MaskedArray`:" @@ -588,6 +646,7 @@ { "cell_type": "code", "execution_count": 17, + "id": "42a4745b", "metadata": {}, "outputs": [ { @@ -616,6 +675,7 @@ }, { "cell_type": "markdown", + "id": "ee375972", "metadata": {}, "source": [ "Understanding the values in `MetricBundle.metric_values` often depends on knowing the *slice points* used. In this example, knowing the numbers of visits in each bin is of little help without knowledge of where the bins are. We can get this information by looking directly at the slicer:" @@ -624,6 +684,7 @@ { "cell_type": "code", "execution_count": 18, + "id": "45468202", "metadata": {}, "outputs": [ { @@ -647,6 +708,7 @@ }, { "cell_type": "markdown", + "id": "5ad52588", "metadata": {}, "source": [ "In this case, the `example1_bg.bundleDict['Airmass histogram (g)'].slicer.slicePoints['bins']` gives the bin edges, allowing us to determine which metric values correspond to which bins." @@ -654,6 +716,7 @@ }, { "cell_type": "markdown", + "id": "12ca6621", "metadata": {}, "source": [ "### 3.4 Saved results" @@ -661,6 +724,7 @@ }, { "cell_type": "markdown", + "id": "597bb05d", "metadata": {}, "source": [ "In addition to providing access to the plots and metric and slice values through the python interpreter, the *metric bundle group* also saved the plots and values in the `outDir` we provided:" @@ -669,6 +733,7 @@ { "cell_type": "code", "execution_count": 19, + "id": "1e1b0c18", "metadata": {}, "outputs": [ { @@ -690,6 +755,7 @@ }, { "cell_type": "markdown", + "id": "805523d8", "metadata": {}, "source": [ "We can see that the plot we just made is present as a `pdf` file. The `npz` file contains the metric and slicer values, from which we can recreate our metric bundle:" @@ -698,6 +764,7 @@ { "cell_type": "code", "execution_count": 20, + "id": "64e594bf", "metadata": {}, "outputs": [], "source": [ @@ -713,6 +780,7 @@ }, { "cell_type": "markdown", + "id": "ed217b6e", "metadata": {}, "source": [ "The metric values are now reloaded:" @@ -721,6 +789,7 @@ { "cell_type": "code", "execution_count": 21, + "id": "94a91574", "metadata": {}, "outputs": [ { @@ -749,6 +818,7 @@ }, { "cell_type": "markdown", + "id": "9f23b36a", "metadata": {}, "source": [ "as is the slicer with its slice points" @@ -757,6 +827,7 @@ { "cell_type": "code", "execution_count": 22, + "id": "5b5b3e98", "metadata": {}, "outputs": [ { @@ -777,6 +848,7 @@ { "cell_type": "code", "execution_count": 23, + "id": "5038f7ac", "metadata": {}, "outputs": [ { @@ -800,6 +872,7 @@ }, { "cell_type": "markdown", + "id": "dd1dbf88", "metadata": {}, "source": [ "and further information regarding how the metric, slicer, or sql constraint were set up:" @@ -808,6 +881,7 @@ { "cell_type": "code", "execution_count": 24, + "id": "c73ebc5c", "metadata": {}, "outputs": [ { @@ -827,6 +901,7 @@ }, { "cell_type": "markdown", + "id": "4134d18e", "metadata": {}, "source": [ "### 3.5 Plot customization" @@ -834,6 +909,7 @@ }, { "cell_type": "markdown", + "id": "2f940f32", "metadata": {}, "source": [ "If you want to customize the appearance of the plot, you can set `plotDict` when creating the `MetricBundle`, for example:" @@ -842,6 +918,7 @@ { "cell_type": "code", "execution_count": 25, + "id": "2d0e5ce3", "metadata": {}, "outputs": [ { @@ -874,6 +951,7 @@ }, { "cell_type": "markdown", + "id": "6154b1c2", "metadata": {}, "source": [ "## 4. Comparing two simulations" @@ -881,6 +959,7 @@ }, { "cell_type": "markdown", + "id": "c4b95a2d", "metadata": {}, "source": [ "To compare two simulations, we need a second one to compare our first one with.\n", @@ -891,6 +970,7 @@ { "cell_type": "code", "execution_count": 26, + "id": "4e1ac5fb", "metadata": {}, "outputs": [], "source": [ @@ -901,6 +981,7 @@ }, { "cell_type": "markdown", + "id": "28d2c35b", "metadata": {}, "source": [ "If we are at NOIR's datalab, we might be able to find it in a standard location, so look there:" @@ -909,6 +990,7 @@ { "cell_type": "code", "execution_count": 27, + "id": "6e9225ec", "metadata": {}, "outputs": [], "source": [ @@ -932,6 +1014,7 @@ }, { "cell_type": "markdown", + "id": "c18932a2", "metadata": {}, "source": [ "If it isn't already there, try to download it:" @@ -940,6 +1023,7 @@ { "cell_type": "code", "execution_count": 28, + "id": "2d8249ff", "metadata": {}, "outputs": [ { @@ -967,6 +1051,7 @@ }, { "cell_type": "markdown", + "id": "f467c71c", "metadata": {}, "source": [ "and assign it a run name, just as we did with our baseline simulation above:" @@ -975,6 +1060,7 @@ { "cell_type": "code", "execution_count": 29, + "id": "6ae28f15", "metadata": {}, "outputs": [ { @@ -997,6 +1083,7 @@ }, { "cell_type": "markdown", + "id": "1e5a4283", "metadata": {}, "source": [ "Repeat the process show in example 1 to calculate metrics for the other example database:" @@ -1005,6 +1092,7 @@ { "cell_type": "code", "execution_count": 30, + "id": "d64aac5f", "metadata": {}, "outputs": [], "source": [ @@ -1027,6 +1115,7 @@ }, { "cell_type": "markdown", + "id": "3c9c6628", "metadata": {}, "source": [ "We can use a *plot handler* to combine plots of the first and second databases:" @@ -1035,6 +1124,7 @@ { "cell_type": "code", "execution_count": 31, + "id": "19c68a3e", "metadata": {}, "outputs": [ { @@ -1062,6 +1152,7 @@ }, { "cell_type": "markdown", + "id": "24a6679c", "metadata": {}, "source": [ "## 5. A metric that maps the sky: depth in *r*" @@ -1069,6 +1160,7 @@ }, { "cell_type": "markdown", + "id": "e3c3a253", "metadata": {}, "source": [ "Making maps follows a similar procedure, but uses a slicer that slices based on position in the sky, usually `rubin_sim.maf.slicers.HealpixSlicer`.\n", @@ -1077,6 +1169,7 @@ }, { "cell_type": "markdown", + "id": "2ec22fa8", "metadata": {}, "source": [ "First, lets sets the constraint only to look at visits in *r*:" @@ -1085,6 +1178,7 @@ { "cell_type": "code", "execution_count": 32, + "id": "0a35cf14", "metadata": {}, "outputs": [], "source": [ @@ -1093,6 +1187,7 @@ }, { "cell_type": "markdown", + "id": "d3dc21dd", "metadata": {}, "source": [ "To make a map, we need to slice the visits by pointing on the sky. Note that the same visit may appear in multiple slices, so this will work correctly even in the resolution of the map is smaller than the camera footprint." @@ -1101,6 +1196,7 @@ { "cell_type": "code", "execution_count": 33, + "id": "8fb6c4b3", "metadata": {}, "outputs": [ { @@ -1117,6 +1213,7 @@ }, { "cell_type": "markdown", + "id": "66eafb99", "metadata": {}, "source": [ "We want to map the coadd depth, so we select a metric that estimates the coadd depth from all visits in a slice:" @@ -1125,6 +1222,7 @@ { "cell_type": "code", "execution_count": 34, + "id": "542b4fba", "metadata": {}, "outputs": [], "source": [ @@ -1133,6 +1231,7 @@ }, { "cell_type": "markdown", + "id": "e5f83e9d", "metadata": {}, "source": [ "Now, we group these together in a metric bundle, create a metric bundle group, and calculate the metrics just as we did in example 1. Note that we are reusing the same database we used in example 1." @@ -1141,6 +1240,7 @@ { "cell_type": "code", "execution_count": 35, + "id": "8f86c52c", "metadata": {}, "outputs": [], "source": [ @@ -1159,6 +1259,7 @@ { "cell_type": "code", "execution_count": 36, + "id": "600ef7cc", "metadata": {}, "outputs": [ { @@ -1188,6 +1289,7 @@ }, { "cell_type": "markdown", + "id": "daab0ba3", "metadata": {}, "source": [ "Again, the values of the metric and slice point can be used directly:" @@ -1196,6 +1298,7 @@ { "cell_type": "code", "execution_count": 37, + "id": "3b35e29e", "metadata": {}, "outputs": [ { @@ -1218,6 +1321,7 @@ }, { "cell_type": "markdown", + "id": "52b71e8f", "metadata": {}, "source": [ "The *slice points* themselves are expressed using a different structure, reflecting the different way in which the visits were sliced:" @@ -1226,6 +1330,7 @@ { "cell_type": "code", "execution_count": 38, + "id": "9703a74f", "metadata": {}, "outputs": [ { @@ -1254,6 +1359,7 @@ }, { "cell_type": "markdown", + "id": "cd7eb91d", "metadata": {}, "source": [ "Note that, for the `maf.HealpixSlicer` and `maf.HealpixSubsetSlicer`, the `sid` is the `healpix` index (`ipix` in the [healpy documentation](https://healpy.readthedocs.io/en/latest/)), and `ra` and `dec` are in radians, not degrees. " @@ -1261,6 +1367,7 @@ }, { "cell_type": "markdown", + "id": "47bd2a1f", "metadata": {}, "source": [ "## 6. Plot customization after initial plotting" @@ -1268,6 +1375,7 @@ }, { "cell_type": "markdown", + "id": "0399f811", "metadata": {}, "source": [ "Plots may be customized after the initial execution through the bundle group, without recomputation of the metric. This is done by using the `plot` method of the metric bundle itself." @@ -1275,6 +1383,7 @@ }, { "cell_type": "markdown", + "id": "c026c5b9", "metadata": {}, "source": [ "For example, you can adjust the parameters of the Mollweide projection to put the south pole at the center, such that the footprint is in the low distortion area at the center of the projection, and the area near the north pole at the high distortion area on the far left and right:" @@ -1283,6 +1392,7 @@ { "cell_type": "code", "execution_count": 39, + "id": "89571d18", "metadata": {}, "outputs": [ { @@ -1314,6 +1424,7 @@ }, { "cell_type": "markdown", + "id": "751ffdaa", "metadata": {}, "source": [ "You can, of course, also use the `PlotHandler` again. Note that changes you made directly to the bundle's plot_dict are kept until they are reset. Changes passed to the `PlotHandler` `plot_dicts` kwarg are temporary, for that plot only. " @@ -1322,6 +1433,7 @@ { "cell_type": "code", "execution_count": 40, + "id": "32c0ba79", "metadata": {}, "outputs": [ { @@ -1347,6 +1459,7 @@ { "cell_type": "code", "execution_count": 41, + "id": "6b8afb80", "metadata": {}, "outputs": [ { @@ -1369,6 +1482,7 @@ }, { "cell_type": "markdown", + "id": "a468bd0c", "metadata": {}, "source": [ "## 7. Summary statistics" @@ -1376,6 +1490,7 @@ }, { "cell_type": "markdown", + "id": "be57e8b3", "metadata": {}, "source": [ "In addition to creating figures, MAF can calculate \"summary statistics\" using summary metrics. These are any `Metric` which is run on the bundle's metric_values themselves, instead of the data slices (opsim) values. Simple metrics such as `MaxMetric`, `MeanMetric`, `MinMetric` are commonly used; however these summary metrics could be more complex, such as calculating the 3x2ptFoM summary statistic after calculating the extragalactic coadded depth." @@ -1383,6 +1498,7 @@ }, { "cell_type": "markdown", + "id": "cf09cd53", "metadata": {}, "source": [ "For example, to get statistics on the coadd depth over the sky, we can add summary metrics to example 3." @@ -1390,6 +1506,7 @@ }, { "cell_type": "markdown", + "id": "fd4d8f1f", "metadata": {}, "source": [ "First, make a list of the metrics, define a new metric bundle that includes them, and create a bundle group to actually calculate the values of the metrics:" @@ -1398,6 +1515,7 @@ { "cell_type": "code", "execution_count": 42, + "id": "27b534e5", "metadata": {}, "outputs": [ { @@ -1431,6 +1549,7 @@ }, { "cell_type": "markdown", + "id": "91ff8e67", "metadata": {}, "source": [ "Then, use the bundle group to drive the calculation of the metrics and summary metrics:" @@ -1439,6 +1558,7 @@ { "cell_type": "code", "execution_count": 43, + "id": "bbfce029", "metadata": {}, "outputs": [], "source": [ @@ -1448,6 +1568,7 @@ }, { "cell_type": "markdown", + "id": "b605749e", "metadata": {}, "source": [ "If you have the metric values already calculated and then need to add a summary metric, this can be done by adding these additional summary metrics, then calling the appropriate function on the metric bundle: " @@ -1456,6 +1577,7 @@ { "cell_type": "code", "execution_count": 44, + "id": "b7832bf8", "metadata": {}, "outputs": [], "source": [ @@ -1468,6 +1590,7 @@ }, { "cell_type": "markdown", + "id": "8169164d", "metadata": {}, "source": [ "Finally, look at the summary values:" @@ -1476,6 +1599,7 @@ { "cell_type": "code", "execution_count": 45, + "id": "ff65c101", "metadata": {}, "outputs": [ { @@ -1499,6 +1623,7 @@ }, { "cell_type": "markdown", + "id": "ae308e89", "metadata": {}, "source": [ "## 8. Working with multiple metrics on the same simulation at once" @@ -1506,6 +1631,7 @@ }, { "cell_type": "markdown", + "id": "d118fc2b", "metadata": {}, "source": [ "As implied by the name \"metric bundle group\", multiple metric bundles can be combined into the some group and calculated \"in one go\"." @@ -1513,6 +1639,7 @@ }, { "cell_type": "markdown", + "id": "49eebc9c", "metadata": {}, "source": [ "Start by creating two new *metric bundles*:" @@ -1521,6 +1648,7 @@ { "cell_type": "code", "execution_count": 46, + "id": "6e76d896", "metadata": {}, "outputs": [], "source": [ @@ -1548,6 +1676,7 @@ }, { "cell_type": "markdown", + "id": "0552d9f4", "metadata": {}, "source": [ "Now add them both to the same *bundle group*:" @@ -1556,6 +1685,7 @@ { "cell_type": "code", "execution_count": 47, + "id": "e93c2570", "metadata": {}, "outputs": [ { @@ -1589,6 +1719,7 @@ }, { "cell_type": "markdown", + "id": "b0c3a045", "metadata": {}, "source": [ "## 9. Finding available database columns with which to express constraints, slices, and metrics -- and STACKERS" @@ -1596,6 +1727,7 @@ }, { "cell_type": "markdown", + "id": "0f85eba2", "metadata": {}, "source": [ "Documentation on the contents of modern opsim output databases can be found in the `rubin_scheduler` [documentation](https://rubin-scheduler.lsst.io) at https://rubin-scheduler.lsst.io/fbs-output-schema.html" @@ -1604,6 +1736,7 @@ { "cell_type": "code", "execution_count": 48, + "id": "2e1d61da", "metadata": {}, "outputs": [ { @@ -1637,6 +1770,7 @@ }, { "cell_type": "markdown", + "id": "edd8d9ad", "metadata": {}, "source": [ "In addition to the columns present in the database, a number of quantities derived from them can be automatically calculated \"on the fly\" and used as if they were database columns. MAF calculates thes derived parameters using *stackers*. " @@ -1644,6 +1778,7 @@ }, { "cell_type": "markdown", + "id": "3e05e25d", "metadata": {}, "source": [ "For example, hour angle is not a column in the database, but MAF includes the `rubin_sim.maf.HourAngleStacker`, and so `HA` can be used as if it were a column:" @@ -1652,6 +1787,7 @@ { "cell_type": "code", "execution_count": 49, + "id": "33637eb7", "metadata": {}, "outputs": [ { @@ -1686,6 +1822,7 @@ }, { "cell_type": "markdown", + "id": "60cde4e3", "metadata": {}, "source": [ "You can find the available *stackers* in the [`rubin-sim` documentation](https://rubin-sim.lsst.io/maf-api-stackers.html) or just list them within python along with the columns they create like this: " @@ -1694,6 +1831,7 @@ { "cell_type": "code", "execution_count": 50, + "id": "da3f78bc", "metadata": {}, "outputs": [ { @@ -1766,6 +1904,7 @@ }, { "cell_type": "markdown", + "id": "3c6e6da5", "metadata": {}, "source": [ "## 10. Finding more documentation" @@ -1773,6 +1912,7 @@ }, { "cell_type": "markdown", + "id": "8af232d6", "metadata": {}, "source": [ "More documentation on available `Metrics`, `Slicers` and `Stackers` can be found in the MAF [API documentation](https://rubin-sim.lsst.io/api.html). A short list of all available metrics can also be found in the [`rubin-sim` documentation](https://rubin-sim.lsst.io/maf-metric-list.html), with more information in the [API documentation](https://rubin-sim.lsst.io/api.html).\n", @@ -1786,6 +1926,7 @@ { "cell_type": "code", "execution_count": 51, + "id": "20a85020", "metadata": {}, "outputs": [ { @@ -1818,6 +1959,7 @@ }, { "cell_type": "markdown", + "id": "a6f7b816", "metadata": {}, "source": [ "In addition, there is a built-in \"help\" function for these base classes that will list all of the contents of the registry and with a `doc=True` kwarg, also show a summary of the docstring." @@ -1826,6 +1968,7 @@ { "cell_type": "code", "execution_count": 52, + "id": "1cff69b9", "metadata": {}, "outputs": [ { @@ -2024,6 +2167,7 @@ }, { "cell_type": "markdown", + "id": "aa84235e", "metadata": {}, "source": [ "**Acknowledgements:** These tutorial notebooks have benefited from previous work in MAF tutorials, including not only the [tutorial notebooks in sims_MAF-contrib](https://github.com/LSST-nonproject/sims_maf_contrib/tree/master/tutorials) but also those by Weixiang Yu, Gordon Richards, and Will Clarkson in their [LSST_OpSim](https://github.com/RichardsGroup/LSST_OpSim) repository, inspired the material to be included here. Stylistic elements of these notebooks were guided by the DP0.1 notebooks developed by Melissa Graham and the Rubin Observatory Community Engagement Team." @@ -2032,9 +2176,9 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "LSST", "language": "python", - "name": "python3" + "name": "lsst" }, "language_info": { "codemirror_mode": { @@ -2046,9 +2190,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.6" + "version": "3.13.9" } }, "nbformat": 4, - "nbformat_minor": 4 + "nbformat_minor": 5 }