v0.2.0

.github/CONTRIBUTING.md

@@ -104,12 +104,53 @@ as well as the burden on users to understand all of the options.
 Use sensible defaults instead.
 Function and argument names should also follow the house rules (see below).
 
+One word means one thing, on both sides of the seam between the formula front
+end and the engine. The engine was ported from `MrQAP` and used its own
+vocabulary; the front end's words won, since those are the ones users read:
+
+| Word | Means | Not |
+|---|---|---|
+| `times` | how many permutations | `reps` |
+| `directed` | logical, whether i→j differs from j→i | `mode`, `"digraph"`/`"graph"` |
+| `permute` | what the null distribution permutes: `"predictor"` or `"outcome"` | `nullhyp`, `method`, `"qapspp"`/`"qapy"` |
+| `.data` | the network the user passes in | — |
+| `matlist` | the named list of matrices the engine fits | `data` |
+| `net` | one coerced network, inside the formula front end | `data` |
+
+`mode` is reserved for a nodeset, as in one-mode and two-mode, which is what it
+means everywhere else in the ecosystem. Do not use it for directedness.
+`permute` replaced `method` because "method" says nothing about what differs;
+`"predictor"` and `"outcome"` name the thing that is actually shuffled.
+`data` is retired as an identifier: it named the network in one half of
+[R/model_regression.R](../R/model_regression.R) and the matrix list in the
+other, one letter away from `.data`. Reserve `data =` for the argument a model
+fitter takes.
+
 When writing documentation or NEWS items, prefer breaking lines at punctuation.
 
 Make it clear when you are referring to functions by adding backticks and parentheses,
 e.g. `a_function()`, and arguments by adding an equals sign, e.g. `argument=`.
 Argument values or variables can be in double quotation marks, e.g. "value".
 
+## Parked extensions
+
+Five model extensions sit on `feature/*` branches while the architecture
+settles, each tracked by a Github issue and each reinstated by reverting one
+commit on `develop`:
+
+| Branch | Removes | Issue |
+|---|---|---|
+| `feature/multinomial-comparison` | `family = "multinom"`, and the `comparison`/`reference` controls | [#7](https://github.com/stocnet/infernet/issues/7) |
+| `feature/fixest-fixed-effects` | the `fixest_se_cluster` control and the `{fixest}` branch | [#8](https://github.com/stocnet/infernet/issues/8) |
+| `feature/glmmtmb-mixed` | mixed negbin and mixed zip | [#9](https://github.com/stocnet/infernet/issues/9) |
+| `feature/gmm-estimator` | the `estimator` control and `R/qap_gmm.R` | [#10](https://github.com/stocnet/infernet/issues/10) |
+| `feature/torch-gpu` | `R/qap_gpu.R` and the `use_gpu` control | [#11](https://github.com/stocnet/infernet/issues/11) |
+
+Do not reinstate one by reverting onto `develop` without reading its issue:
+several need rewriting against the merged engine rather than reverting onto it.
+Do not add a new model family that needs a new `Suggests` package until the
+two engines are one, for the same reason these left.
+
 ## Package architecture
 
 ### Project overview
@@ -184,9 +225,10 @@ the internal engine ported from `MrQAP` is named `qap_*.R`.
 |---|---|
 | `model_tests.R` | the test family: `test_random()` (CUG), `test_configuration()`, `test_permutation()` (QAP), and `print.network_test()` |
 | `model_regression.R` | `net_regression()`, the formula front end (`convertToMatrixList()`, `getRHSNames()`, `specificationAdvice()`), and the `print.*` methods for its results |
-| `qap_engine.R` | `QAPglm()` and `QAPglmPermEst()` — the matrix-level engine that performs the baseline fit and the permutation inference |
+| `qap_engine.R` | `QAPengine()` and `QAPPermEst()` — the one matrix-level engine, for both a dyadic network and a cognitive social structure |
+| `qap_shapes.R` | the four things the two shapes do differently, and nothing else |
 | `qap_utils.R` | formula parsing, input validation, `future` plumbing, matrix permutation (`RMPerm()`), the model-fitting dispatcher `fit_qap_model()`, and the permutation aggregators |
-| `qap_css.R` | `QAPcss()` and `QAPcssPermEst()` — the parallel engine for cognitive social structures |
+| `qap_css.R` | what a CSS needs that a dyadic network does not: a vectoriser for a three-dimensional array, and a print method |
 | `qap_gmm.R` | GMM moment conditions and residual functions for the `estimator = "gmm"` path |
 | `qap_gpu.R` | the optional `{torch}` batch OLS path, `gpu_batch_ols()` |
 | `qap_confusion.R` | probabilistic confusion matrices for binary outcomes |
@@ -212,21 +254,49 @@ regression entry point. Its control flow is:
    network, drops the ones that are missing a predictor with a warning,
    and pools the rest.
 3. Resolve `family = "auto"` against the dependent variable
-   (binomial for a 0/1 outcome, gaussian otherwise), and resolve `mode` and
+   (binomial for a 0/1 outcome, gaussian otherwise), and resolve `directed` and
    `diag` from the network with `manynet::is_directed()` and `manynet::is_complex()`.
+   Report each resolution with `snet_info()`: a model the user did not state is
+   one they cannot describe in a paper.
 4. Call `QAPglm()`, which parses the formula, fits the baseline model once
-   via `fit_qap_model()`, then runs `reps` permutations and aggregates them.
+   via `fit_qap_model()`, then runs `times` permutations and aggregates them.
 5. Attach a probabilistic confusion matrix where the outcome is binary,
    and class the result `net_regression`.
 
-Inside `QAPglm()` the null hypothesis decides the permutation scheme:
-`"qapy"` permutes the dependent matrix only, while `"qapspp"` implements Dekker
-et al.'s double semi-partialling, running one permutation set per main predictor
-after residualising it against the others.
+Inside `QAPglm()` the `permute` control names what the null distribution
+permutes: `"outcome"` permutes the dependent matrix only, while `"predictor"`
+implements Dekker et al.'s double semi-partialling, running one permutation set
+per main predictor after residualising it against the others.
+With one predictor there is nothing to residualise against, so `"predictor"`
+falls back to `"outcome"` and says so.
 Permuted coefficients and test statistics are then compared against the baseline
 by `compare_perm_to_baseline()` and reduced to `lower`/`larger`/`abs`
 p-value matrices by `aggregate_perm_results()`.
-`QAPcss()` mirrors this same permute-refit-aggregate architecture for CSS data.
+### One engine, two shapes
+
+`QAPengine()` fits a dyadic network and a cognitive social structure through the
+same skeleton. They differ in four places and nowhere else, and those four live
+in a *shape* returned by `.qap_shape()`
+([R/qap_shapes.R](../R/qap_shapes.R)):
+
+| Field | Dyadic | Cognitive |
+|---|---|---|
+| `vectorise()` | `make_qap_data()`, one row per dyad | `make_css_data()`, one row per dyad per perceiver |
+| `permute()` | `RMPerm()` | `RMPerm(CSS = TRUE)` |
+| `unresidualise()` | `residuals_to_matrix()` | `residuals_to_array()` |
+| `rand_slots` | sender, receiver, network | and perceiver |
+
+A fifth field, `max_trials`, says how many permutations to redraw before giving
+up: one for a dyadic network, since a degenerate draw is simply dropped and
+counted, and 10,000 for a CSS, whose sparse arrays often permute into an
+outcome with a single value.
+
+Add a shape rather than a second engine. A random-intercept slot a shape does
+not list cannot be requested, so a perceiver intercept on a dyadic network
+aborts by name rather than producing a formula that will not parse.
+
+Before this merge the two were `QAPglm()` and `QAPcss()`, 55% the same code, and
+every fix had to be made twice. One of them was made in only one place.
 
 The formula front end accepts these terms, and a new one should be added
 to `getRHSNames()` and `convertToMatrixList()` together:
@@ -328,13 +398,28 @@ Users opt in with e.g. `options(snet_verbosity = "verbose")`.
 These wrappers pass their input to `{cli}`, so:
 
 - Braces interpolate, replacing `paste()`: `snet_abort("{.val {dep}} is not in the data.")`.
+- A brace expression beginning with a dot is read as a *style*, not as code, so
+  `{.val {.directed_label(x)}}` aborts with "Invalid cli literal". Resolve a
+  call to a dot-prefixed function into a local variable first.
+- `snet_info()` pastes its arguments, so pass separate strings for a longer
+  message rather than a named `c()` vector: the names are dropped and the
+  strings run together without a space.
 - Use `{cli}` inline classes to mark up what you refer to — `{.fn}` for functions,
   `{.arg}`/`{.var}` for arguments and variables, `{.val}` for values,
   `{.pkg}` for packages, `{.url}` for links.
 - Use `{cli}`'s pluralisation rather than hand-written branches:
   `snet_warn("Dropped {length(dropped)} network{?s}.")`.
 
 Prefer "`{.arg times}` must be a positive whole number" over "invalid input".
+
+Report every default the model resolves for itself, with `snet_info()`: the
+family read from the outcome's values, the directedness read from the network,
+and any fallback such as `permute = "predictor"` reducing to `"outcome"`.
+A model the user did not state is one they cannot describe in a paper.
+Because this output is silent by default, a broken message is invisible in
+every other test, so cover it in
+[tests/testthat/test-qap_reporting.R](../tests/testthat/test-qap_reporting.R),
+which runs with `snet_verbosity = "verbose"`.
 Where a function needs a package from `Suggests`, name it and say how to get it:
 `snet_abort(c("The {.pkg lme4} package is required for random effects.", i = "Install it with {.run install.packages(\"lme4\")}."))`.
 

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: infernet
 Title: Inferential Models for Many Different Types of Networks
-Version: 0.1.1
+Version: 0.2.0
 Description: A set of tools for testing networks.
    It includes functions for univariate and multivariate 
    conditional uniform graph and quadratic assignment procedure testing,
@@ -39,14 +39,9 @@ Imports:
     reformulas
 Suggests:
     lme4,
-    nnet,
-    fixest,
-    gmm,
     MASS,
     pscl,
-    testthat (>= 3.0.0),
-    torch,
-    glmmTMB
+    testthat (>= 3.0.0)
 Config/Needs/build:
     roxygen2,
     devtools

NAMESPACE

@@ -1,5 +1,6 @@
 # Generated by roxygen2: do not edit by hand
 
+S3method(print,QAPCSS)
 S3method(print,net_regression)
 S3method(print,network_test)
 export(net_regression)

NEWS.md

@@ -1,3 +1,61 @@
+# infernet 0.2.0
+
+## Package
+
+- Branching off five model extensions reduces `Suggests` packages from eight to three
+- Updated CONTRIBUTING with the vocabulary table and the reporting rule
+
+## Regression
+
+- Fixed `net_regression()` failing on a two-mode network with more columns than
+  rows (closes #4)
+  - Validity was built rows-by-rows, so wider predictor extended it with `NA` 
+    and the dyad count came back as `NA`
+  - Reported 448 by 12489 network now fits on all 5,595,072 dyads
+- Fixed `groups=` being refused on a two-mode network unless it matched the row
+  mode, though either mode may be the one that is blocked
+- Standardised vocabulary in the engine to match the front end:
+  - Renamed `reps=` to `times=` including on the returned fit
+  - Renamed `method=`/`nullhyp=` to `permute=`, as method can be ambiguous
+    - `method = "qap"`/`nullhyp = "qapspp"` is now `permute = "predictor"`
+    - `method = "qapy"` is now `permute = "outcome"`
+  - Renamed `mode=` to `directed=`, reserving mode for one-mode and two-mode networks
+    - `mode = "undirected"` is now `directed = FALSE`
+  - `data` is retired as potentially confusing:
+    - `.data` remains the network the user passes in
+    - `matlist` is the named list of matrices the engine fits
+    - `net` is one coerced network, inside the formula front end
+- Added `snet_info()` reporting of every default the model resolves for itself
+  - Family chosen from the outcome's values
+  - Directedness from the network
+  - `permute = "predictor"` falling back to `"outcome"` with one predictor
+- Merged `QAPglm()` and `QAPcss()` engines into one, `QAPengine()`
+  - 55% the same code, reduces code from 791 lines to 552
+  - Differences in treatment are now four functions: vectorisation, permutation, 
+    returning residuals, and identifying random intercepts
+- Removed the `torch` GPU path (`feature/torch-gpu`)
+  - Gaussian only, duplicated for CSS, no test, and no hosted runner has a
+    CUDA device; `{torch}` in Suggests broke the CI build
+- Removed the `gmm` estimator and the `estimator` control (`feature/gmm-estimator`)
+  - It warned that the coefficient covariance matrix was singular on every
+    family, on well-conditioned data
+- Removed the mixed negbin and mixed zip paths (`feature/glmmtmb-mixed`)
+  - `{glmmTMB}` carries 62 recursive dependencies and must match `{TMB}`
+  - The standard `negbin` and `zip` paths are unaffected
+- Removed `family = "multinom"` and the `comparison`/`reference` controls
+  (`feature/multinomial-comparison`)
+  - Unreachable from the front end, and its pairwise branch forked both
+    engines at 21 points
+- Removed the `fixest_se_cluster` control (`feature/fixest-fixed-effects`)
+  - A bar in the formula now means an `{lme4}` random-effect term, and
+    nothing else; `parse_qap_formula()` drops from three branches to one
+
+## Tests
+
+- Added a wide two-mode fixture and two regression tests for #4
+- Added `test-qap_reporting.R`, which runs with `snet_verbosity = "verbose"`
+- Added `test-qap_shape_css.R`, which fits a cognitive social structure
+
 # infernet 0.1.1
 
 ## Package