Found while running notebooks/pipeline/assessment_quality.ipynb at commit
7952236 with prediction_field = "prediction" (the score-your-own-model path,
rather than the default assessor-value path).
Two problems in the data-loading cell (the cell containing the
read_sales_univ fallback chain):
- NameError. The
if prediction_field == "prediction": branch references
sales_filters and univ_filters:
if len(sales_filters) > 0:
df_sales = select_filter(df_sales, sales_filters)
...
if len(univ_filters) > 0:
df_sales = select_filter(df_univ, univ_filters)
Neither variable is defined anywhere in the notebook (the configuration cell
defines other knobs but not these), so evaluating your own model's predictions
raises NameError before any study runs. The default path never touches the
branch, which is presumably why it has gone unnoticed.
- Wrong variable. The second snippet above assigns the filtered universe to
df_sales and then sets sales_univ_pair.universe = df_univ, the unfiltered
frame, so even with the variables defined, the universe filter result is
discarded (and the sales frame is clobbered).
Suggested fix: define sales_filters = [] and univ_filters = [] in the
configuration cell, and change the second assignment to
df_univ = select_filter(df_univ, univ_filters).
(Separate from the two library-side crashes in this notebook already reported
in issue #359 and fixed by PR #357; this one is in the notebook itself.)
Found while running
notebooks/pipeline/assessment_quality.ipynbat commit7952236 with
prediction_field = "prediction"(the score-your-own-model path,rather than the default assessor-value path).
Two problems in the data-loading cell (the cell containing the
read_sales_univfallback chain):if prediction_field == "prediction":branch referencessales_filtersanduniv_filters:Neither variable is defined anywhere in the notebook (the configuration cell
defines other knobs but not these), so evaluating your own model's predictions
raises NameError before any study runs. The default path never touches the
branch, which is presumably why it has gone unnoticed.
df_salesand then setssales_univ_pair.universe = df_univ, the unfilteredframe, so even with the variables defined, the universe filter result is
discarded (and the sales frame is clobbered).
Suggested fix: define
sales_filters = []anduniv_filters = []in theconfiguration cell, and change the second assignment to
df_univ = select_filter(df_univ, univ_filters).(Separate from the two library-side crashes in this notebook already reported
in issue #359 and fixed by PR #357; this one is in the notebook itself.)