Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
5078c8f
Initial plan
Copilot Apr 16, 2026
704f50d
Clarify deviation, error/noise, and residual terminology
Copilot Apr 16, 2026
bad929d
Refine terminology links and consistency for error vs residual defini…
Copilot Apr 16, 2026
70dfc40
Apply suggestions from code review
d-morrison Apr 16, 2026
7d348a1
Apply erf vs devn notation distinction across estimation and probabil…
Copilot Apr 16, 2026
17ea9a8
Define statistical signal and refine residual wording in linear-model…
Copilot Apr 16, 2026
2b246e5
Address review thread on error terminology and residual notation cons…
Copilot Apr 16, 2026
e239721
Apply suggestions from code review
d-morrison Apr 16, 2026
9482d8f
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 16, 2026
2f3c412
Use resid notation and consolidate residual definitions into shared s…
Copilot Apr 16, 2026
f43bdf4
Clarify and align terminology for signal, deviation, error/noise, res…
Copilot Apr 16, 2026
881dd5c
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 16, 2026
6b715c4
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 20, 2026
7a57d6c
Refine deviation terminology and Gaussian diagnostics assumptions
Copilot Apr 20, 2026
aad3a7f
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 20, 2026
2667982
Refine deviation note to reserve term error for estimation deviations
Copilot Apr 21, 2026
d1f152c
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 21, 2026
f1cb563
Remove local macro includes and rely on shared macro source
Copilot Apr 21, 2026
41b644d
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 21, 2026
515bd7b
Rewrite residual-noise estimate using deviation notation
Copilot Apr 21, 2026
679eff9
Apply suggestions from code review
d-morrison Apr 21, 2026
79e5047
Refine estimation terminology list for deviation and residual definit…
Copilot Apr 21, 2026
c90ae78
Use cf abbreviation in deviation terminology note
Copilot Apr 21, 2026
246298f
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 21, 2026
d60b3f8
Align deviation anchors and residual/deviation wording across chapters
Copilot Apr 21, 2026
6b42698
Use expectation macro in residual deviation equation
Copilot Apr 21, 2026
22d7346
Apply suggestions from code review
d-morrison Apr 21, 2026
a9181c3
Clarify residual section wording and standardize standardized-residua…
Copilot Apr 21, 2026
4d764b6
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 21, 2026
2258ffe
Align deviation terminology with subpopulation means and retire err g…
Copilot Apr 22, 2026
79ddb2f
Refine standardized residual SD notation and expectation macro usage
Copilot Apr 22, 2026
5cf6e53
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 22, 2026
c12152f
Apply suggestions from code review
d-morrison Apr 22, 2026
f9e9f89
Align diagnostics notation and split signal definition into dedicated…
Copilot Apr 22, 2026
a7ff67b
Clarify deviation/residual terminology and align cross-chapter notati…
Copilot Apr 22, 2026
4412de6
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 22, 2026
bbda477
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 23, 2026
2edd288
Plan follow-up fixes from latest review thread
Copilot Apr 23, 2026
f6f4b20
Fix standardized residual distribution wording and fitted-value notation
Copilot Apr 23, 2026
a122997
Apply suggestions from code review
d-morrison Apr 23, 2026
ed6c40b
Quantify residual covariance and standardized residual covariance
Copilot Apr 23, 2026
746d344
Apply suggestions from code review
d-morrison Apr 23, 2026
f91bed2
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 23, 2026
fc77219
Update _subfiles/Linear-models-overview/_sec_linreg_diagnostics.qmd
d-morrison Apr 23, 2026
c233284
Align conditional mean notation in diagnostics section
Copilot Apr 23, 2026
55ae1b2
Address diagnostics review thread notation and residual details
Copilot Apr 23, 2026
d08a982
Agent-Logs-Url: https://github.com/d-morrison/rme/sessions/d783ca6b-3…
Copilot Apr 23, 2026
0ee141f
Merge origin/main into copilot branch and resolve diagnostics conflict
Copilot Apr 23, 2026
322a513
Refine QQ-plot notes after merge conflict resolution
Copilot Apr 23, 2026
d227a69
Update _def-residual-deviation.qmd
d-morrison Apr 23, 2026
a1d5576
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 23, 2026
0d9f514
Update _def-residual-deviation.qmd
d-morrison Apr 23, 2026
252ac75
Update probability.qmd
d-morrison Apr 23, 2026
28fef06
Apply suggestions from code review
d-morrison Apr 24, 2026
b9630dc
docs: resolve unresolved review comments on residual terminology and …
Copilot Apr 24, 2026
de1e48d
Clarify terminology and update residuals section
d-morrison Apr 24, 2026
4d7a551
Update _def-residual-deviation.qmd
d-morrison Apr 24, 2026
7739649
Update definition and references for 'residual'
d-morrison Apr 24, 2026
b653153
Apply suggestions from code review
d-morrison Apr 24, 2026
cfec148
Apply suggestions from code review
d-morrison Apr 24, 2026
0613c1a
Apply suggestions from code review
d-morrison Apr 24, 2026
f20e889
docs: apply latest review-thread fixes for residual notation and stan…
Copilot Apr 24, 2026
2ad4927
Resolve residual-notation review comments in linear-model diagnostics
Copilot Apr 24, 2026
3a30d21
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 24, 2026
83480ff
docs: define hat matrix before residual covariance use and update ren…
Copilot Apr 24, 2026
ae4a367
Apply suggestions from code review
d-morrison Apr 24, 2026
97394fa
docs: address open residual diagnostics review comments
Copilot Apr 24, 2026
c81a59c
docs: fix notation consistency in residual variance proof
Copilot Apr 24, 2026
b8ce628
update renv
d-morrison Apr 25, 2026
b94de35
Fix dprodf macro usage in residual theorem
Copilot Apr 25, 2026
9eac88c
Fix lint-thread notation and formatting; revert renv/activate drift
Copilot Apr 25, 2026
261ad9d
Plan: investigate failing CI run 24919228108
Copilot Apr 27, 2026
75d2703
CI: pin preview workflow R version to 4.5.2 and revert renv drift
Copilot Apr 27, 2026
802d356
CI: provide GITHUB_PAT to setup-renv in preview workflow
Copilot Apr 27, 2026
072647d
CI: disable pak in preview workflow setup-renv step
Copilot Apr 27, 2026
a46be49
chore: plan workflow issue investigation
Copilot Apr 27, 2026
47eac42
ci: stabilize preview workflow setup and revert unintended renv check…
Copilot Apr 27, 2026
4309aaa
chore: plan merge from update-book-for-r-4-6-0
Copilot Apr 27, 2026
a1f15af
Merge remote-tracking branch 'origin/copilot/update-book-for-r-4-6-0'…
Copilot Apr 27, 2026
f76bcdf
Merge branch 'main' into copilot/define-relationships-errors-residuals
d-morrison Apr 27, 2026
83d1d29
Update .github/workflows/preview.yml
d-morrison Apr 27, 2026
22ff560
update renv
d-morrison Apr 27, 2026
a5385c1
try to fix
d-morrison Apr 27, 2026
5b09b88
Apply suggestions from code review
d-morrison Apr 27, 2026
4e46d48
add broom.helpers
d-morrison Apr 27, 2026
7a2c299
Clarify residual theorem with indexed/vector fitted-value notation
Copilot Apr 27, 2026
c87ac2f
Merge branch 'copilot/define-relationships-errors-residuals' of githu…
d-morrison Apr 27, 2026
0676119
Address review thread feedback and CI libnode issue
Copilot Apr 27, 2026
2f051d5
Use design-matrix conditioning in Gaussian residual statement
Copilot Apr 27, 2026
fcacfd6
Use explicit residual-vector notation and document macro policy
Copilot Apr 27, 2026
87768e8
Apply suggestion from @Copilot
d-morrison Apr 27, 2026
47bc855
Apply suggestions from code review
d-morrison Apr 27, 2026
c1cbd65
Apply latest PR thread feedback on deps and notation
Copilot Apr 27, 2026
799d425
Align residual-unbiased theorem assumptions with conditional proof
Copilot Apr 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@ _freeze/
^\.github$
^README\.Rmd$
^data-raw$
^\.positai$
^\.claude$
Comment thread
d-morrison marked this conversation as resolved.
8 changes: 7 additions & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ Key macros to use:
- **Aligned equations**: Use `\ba` / `\ea` for `\begin{aligned}` / `\end{aligned}`
- **Greek letters**: Use `\b` for $\beta$, `\g` for $\gamma$, `\a` for $\alpha$
- **Formatting**: Use `\red{...}` and `\blue{...}` for colored text in math
- **Deviation/error notation**: Use `\erf{...}` for deviations of estimates/estimators from their estimands; use `\devn(...)` for all other deviations (e.g., observations from population means)

matrix-product helper macros:

Expand All @@ -198,7 +199,7 @@ matrix-product helper macros:

Residual and deviation helper macros include:

- `\err` for generic error or residual terms
- `\err` is deprecated; prefer `\devn(...)` for new non-estimation deviations while legacy uses are being phased out
- `\erf{\theta}` for estimate/estimand deviations
- `\devn(...)` for other deviations
- `\resid` for residual symbols (`r`)
Expand All @@ -213,6 +214,11 @@ Residual and deviation helper macros include:

Always check `latex-macros/macros.qmd` for available macros before writing raw LaTeX.

When a repeatedly used expression needs a new macro,
add it to `latex-macros/macros.qmd`
in the `latex-macros` submodule
and push that submodule update.

## Color Coding Strategy for Math Expressions

Use `\red{...}` and `\blue{...}` purposefully and consistently to help readers:
Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/lint-changed-files.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,18 @@ jobs:
- uses: actions/checkout@v5

- uses: r-lib/actions/setup-r@v2
with:
r-version: '4.6.0'
use-public-rspm: true

- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y jags libcurl4-openssl-dev libpng-dev libfontconfig1-dev libjpeg-dev libglpk-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libtiff5-dev libwebp-dev
sudo apt-get install -y jags libcurl4-openssl-dev libpng-dev libfontconfig1-dev libjpeg-dev libglpk-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libtiff5-dev libwebp-dev libnode109

- uses: r-lib/actions/setup-renv@v2
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

- name: Add lintr options
run: |
Expand All @@ -33,6 +38,7 @@ jobs:
run: |
files <- gh::gh("GET https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files")
changed_files <- purrr::map_chr(files, "filename")
changed_files <- setdiff(changed_files, "renv/activate.R")
all_files <- list.files(recursive = TRUE)
exclusions_list <- as.list(setdiff(all_files, changed_files))
lintr::lint_dir(exclusions = exclusions_list)
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,18 +43,21 @@ jobs:
with:
tinytex: false

- uses: r-lib/actions/setup-r@HEAD
- uses: r-lib/actions/setup-r@v2
with:
r-version: '4.6.0'
use-public-rspm: true

- name: Install system dependencies
run: |
sudo apt-get update
# `libglpk-dev` is required for the igraph/ggdag/ggraph dependency stack.
# Keep this dependency aligned with other workflows that restore/render the same renv set.
sudo apt-get install -y jags libcurl4-openssl-dev libpng-dev libfontconfig1-dev libjpeg-dev libglpk-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libtiff5-dev libwebp-dev
sudo apt-get install -y jags libcurl4-openssl-dev libpng-dev libfontconfig1-dev libjpeg-dev libglpk-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libtiff5-dev libwebp-dev libnode109

- uses: r-lib/actions/setup-renv@v2
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
Comment thread
d-morrison marked this conversation as resolved.

- name: Install local package
run: |
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,19 @@ jobs:
with:
tinytex: true

- uses: r-lib/actions/setup-r@HEAD
- uses: r-lib/actions/setup-r@v2
with:
r-version: '4.6.0'
use-public-rspm: true

- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y jags libcurl4-openssl-dev libpng-dev libfontconfig1-dev libjpeg-dev libglpk-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libtiff5-dev libwebp-dev
sudo apt-get install -y jags libcurl4-openssl-dev libpng-dev libfontconfig1-dev libjpeg-dev libglpk-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libtiff5-dev libwebp-dev libnode109

- uses: r-lib/actions/setup-renv@v2
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

- name: Install local package
run: |
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,5 @@ renv/lock/
renv/python/
renv/sandbox/
renv/staging/
.positai
.claude
1 change: 1 addition & 0 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ Suggests:
plotly,
rjags,
runjags,
datawizard (>= 1.3.1),
survminer,
tidyr,
spelling,
Expand Down
65 changes: 49 additions & 16 deletions _subfiles/Linear-models-overview/_def-residual-deviation.qmd
Original file line number Diff line number Diff line change
@@ -1,26 +1,59 @@
:::{#def-resid-noise}
#### Residual noise/deviation from the population mean
:::{#def-deviation-subpop-mean}
#### Deviation of an observation from its subpopulation mean

The **residual noise** in a probabilistic model $p(Y)$,
also known as the
**residual deviation of an observation from its population mean**
or **residual** for short,
is the difference between an observed value $y$ and its population mean:
The **deviation of an observation from its subpopulation mean**
in a probabilistic model $p(Y \mid X)$,
is the difference between an observed value
and its model-implied mean given covariates:

$$\devn(y) \eqdef y - \Expf{Y}$$ {#eq-def-resid}
$$\devn(y_i) \eqdef y_i - \E{Y_i \mid X_i}$$ {#eq-def-deviation-subpop-mean}
:::

:::{.notes}
We use the same notation for residual noise that we used for [errors](estimation.qmd#def-error).
Many sources call deviations "error" or "noise."
The model-implied mean
can be viewed as an estimate of $Y_i$,
before $y_i$ is observed.
However, an estimation error is defined as an estimate minus its estimand
(see [estimation error](estimation.qmd#def-estimation-error)),
and deviation is the observed value minus its estimand,
so the deviation is actually the *negative* of the estimation error of the mean $\E{Y_i \mid X_i}$
with respect to its estimand $Y_i$:
Comment thread
d-morrison marked this conversation as resolved.

$\Expf{Y}$ can be viewed as an estimate of $Y$, before $y$ is observed.
Conversely, each observation $y$ can be viewed as an estimate of $\Expf{Y}$
(albeit an imprecise one, individually, since $n=1$).
$$\devn(y_i) = -\paren{\E{Y_i \mid X_i} - y_i}$$

Comment thread
d-morrison marked this conversation as resolved.
On the other hand,
each observation $y_i$ can be viewed
as a nonparametric estimate of $\mu_i = \E{Y_i \mid X_i}$
(albeit an imprecise one, individually, since $n=1$):

$$y_i = \hmu_i^{(NP)}$$

where $\hmu_i^{(NP)}$ denotes
the nonparametric estimate from a single observed value $y_i$.

Thus,
the deviation can be interpreted as the estimation error of $y_i$
with respect to $\E{Y_i \mid X_i}$:
Comment thread
d-morrison marked this conversation as resolved.

$$
\ba
\devn(y_i)
&= y_i - \mu_i
\\
&= \hmu_i^{(NP)} - \mu_i
\\
&= \erf{\hmu_i^{(NP)}}
\ea
$$

:::

We can rearrange @eq-def-resid
to view $y$ as
the sum of its mean plus the residual noise:
---

We can rearrange @eq-def-deviation-subpop-mean
to view $y_i$ as
the sum of its mean plus the deviation
(often called error/noise):

$$y = \Exp{Y} + \epsf{y}$$
$$y_i = \E{Y_i \mid X_i} + \devn(y_i)$$
9 changes: 9 additions & 0 deletions _subfiles/Linear-models-overview/_def-signal.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
:::{#def-signal}
#### Signal (statistical sense)

In statistical modeling,
the **signal** is the deterministic part of the model.
For mean-based models,
the signal is the model-implied mean function,
for example $\E{Y \mid X}$ (or $\E{Y}$ when there are no covariates).
:::
Loading
Loading