Skip to content

Added bugfixes for release 1.0.0#7

Merged
Thematiq merged 2 commits intomasterfrom
chore/v1-fixes
May 7, 2026
Merged

Added bugfixes for release 1.0.0#7
Thematiq merged 2 commits intomasterfrom
chore/v1-fixes

Conversation

@Thematiq
Copy link
Copy Markdown
Member

@Thematiq Thematiq commented May 7, 2026

No description provided.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates tie-handling semantics in win-table construction, expands test coverage around those behaviors, and adds supporting release artifacts (NOTICE, example notebook, dev dependency updates).

Changes:

  • Adjust paired/local-ROPE win-table construction and tie-solver behavior (spread/add/forget) and add targeted tests.
  • Update PyMC model input typing/casting for win counts.
  • Add a Critical Difference Diagram example notebook, a NOTICE file for the reference implementation, and add graphviz to dev dependencies (with lockfile updates).

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
bbttest/bbt/alg.py Changes tie-solver behavior (spread now assigns 0.5 per tie) and adjusts paired local-ROPE table construction.
bbttest/bbt/model.py Updates win1/win2 typing and casts n/w1/w2 to int32 inside the PyMC model builder.
bbttest/bbt/_utils.py Updates _validate_params to allow extra kwargs when the function declares **kwargs.
tests/bbt/test_alg.py Adds tests for paired local-ROPE path and explicit tie-solver semantics for spread/add/forget.
tests/bbt/test__utils.py Adds a test ensuring _validate_params ignores unexpected kwargs when **kwargs is present.
examples/02_critical_difference_diagram.ipynb Adds an example notebook demonstrating CDD plotting with PyBBT.
pyproject.toml Adds graphviz to the dev dependency group.
uv.lock Lockfile update to include graphviz.
NOTICE.md Adds NOTICE content for the referenced bbtcomp implementation/paper.
Comments suppressed due to low confidence (1)

bbttest/bbt/alg.py:124

  • _solve_ties now produces fractional win counts for tie_solver="spread" (e.g., 0.5 per tie). Downstream, _mcmcbbt_pymc/_build_bbt_model uses a Binomial likelihood that requires integer n and integer observed wins, and it currently truncates win1/win2 to ints. As a result, spread will silently drop the 0.5 contributions (and can also make n inconsistent with the intended tie semantics). Consider either (a) converting fractional tables to an equivalent integer representation before sampling (e.g., scale w1/w2/n by 2 when halves are present), or (b) rejecting spread/fractional tables when using the Binomial model and requiring tie_solver="davidson" for tie-aware sampling.
def _solve_ties(table: np.ndarray, tie_solver: str) -> np.ndarray:
    if tie_solver == "davidson":
        return table
    if tie_solver == "spread":
        tie_val = table[:, TIE_COL] / 2
    elif tie_solver == "add":
        tie_val = table[:, TIE_COL]
    else:
        tie_val = 0
    table[:, ALG1_COL] += tie_val
    table[:, ALG2_COL] += tie_val
    return table

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread bbttest/bbt/model.py Outdated
Comment thread examples/02_critical_difference_diagram.ipynb
@Thematiq Thematiq merged commit 27f92eb into master May 7, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants