Skip to content

Add summary statistics to ProductMeasure - #608

Open
Laasya-73 wants to merge 4 commits into
developfrom
feature/product-measure-statistics
Open

Add summary statistics to ProductMeasure#608
Laasya-73 wants to merge 4 commits into
developfrom
feature/product-measure-statistics

Conversation

@Laasya-73

Copy link
Copy Markdown
Collaborator

This PR adds combined summary statistics to ProductMeasure.

ProductMeasure now derives statistics for its independent marginal blocks by:

  • concatenating marginal means
  • concatenating marginal variances
  • concatenating marginal standard deviations
  • constructing a block-diagonal covariance matrix from the marginal covariance matrices

This allows users to access statistics for the complete product distribution directly instead of calculating them separately for each marginal.

Changes

  • Added combined summary-statistic support to ProductMeasure.
  • Added block-diagonal covariance construction for independent marginal blocks.
  • Added tests for one-dimensional and multidimensional marginal combinations.
  • Added coverage for scalar one-dimensional statistics.
  • Added tests for behavior when marginal statistics are unavailable.

@Laasya-73 Laasya-73 self-assigned this Aug 19, 2026
@Laasya-73 Laasya-73 added the enhancement New feature or request label Aug 19, 2026
@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.22642% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.70%. Comparing base (dc3d923) to head (536d3cd).

Files with missing lines Patch % Lines
qmcpy/true_measure/product_measure.py 96.22% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #608      +/-   ##
===========================================
+ Coverage    88.65%   88.70%   +0.04%     
===========================================
  Files          104      104              
  Lines         8508     8561      +53     
===========================================
+ Hits          7543     7594      +51     
- Misses         965      967       +2     
Flag Coverage Δ
alltests 88.70% <96.22%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
qmcpy/true_measure/product_measure.py 96.42% <96.22%> (-0.19%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds combined summary statistics to ProductMeasure.

Changes:

  • Concatenates marginal means, variances, and standard deviations.
  • Constructs block-diagonal covariance matrices.
  • Adds coverage for scalar, multidimensional, ordering, and unavailable-statistic cases.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
qmcpy/true_measure/product_measure.py Implements combined statistics.
test/test_product_measure.py Tests statistic aggregation and errors.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread qmcpy/true_measure/product_measure.py Outdated
Comment on lines +241 to +242
if sparse.issparse(block):
block = block.toarray()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit Fix ProductMeasure.covariance to preserve sparse blocks. The covariance property now:

  • Skips toarray() on sparse blocks, preserving their shape for validation
  • Uses sparse.block_diag(blocks, format="dia") and sets .data.setflags(write=False) when all blocks are sparse
  • Falls back to the dense path (with toarray() only at that point) when blocks are mixed sparse/dense

Co-authored-by: fjhickernell <817530+fjhickernell@users.noreply.github.com>

@fjhickernell fjhickernell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried it with the AreWeThereYet notebook for MATH 565 and it worked like a charm.

@fjhickernell

fjhickernell commented Aug 21, 2026

Copy link
Copy Markdown
Member

@Laasya-73, running by copilot once more but it looks ready for you to squash and merge.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread test/test_product_measure.py Outdated
np.testing.assert_allclose(
tm.standard_deviation, [np.sqrt(4.0 / 3.0), np.sqrt(3.0)]
)
np.testing.assert_allclose(tm.covariance, np.diag([4.0 / 3.0, 3.0]))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. The covariance result is now converted to a dense array via toarray() when sparse before passing to assert_allclose.

Comment thread test/test_product_measure.py Outdated
Comment on lines +69 to +70
for statistic in ("mean", "variance", "standard_deviation", "covariance"):
assert not getattr(tm, statistic).flags.writeable

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. The immutability loop now checks value.data.flags for sparse matrices and value.flags for dense arrays.

Co-authored-by: fjhickernell <817530+fjhickernell@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

qmcpy/true_measure/product_measure.py:267

  • Return a protected view here, not the owned array itself. An owned array made read-only with setflags(write=False) can be made writable again by calling result.setflags(write=True), whereas the established statistics contract explicitly checks that this raises (test/test_true_measures.py:221-225). The mixed sparse/dense covariance path therefore exposes a mutable result unlike the other dense statistics.
        return self._read_only_array(covariance)

@sou-cheng-choi

Copy link
Copy Markdown
Member

@Laasya-73

A dense block densifies an otherwise sparse covariance:

from qmcpy import ProductMeasure, DummySampler, Uniform, Gaussian
    from scipy import sparse
    import numpy as np
    d = 128
    product = ProductMeasure(
        DummySampler(d + 2),
        [Uniform(DummySampler(d)),
         Gaussian(DummySampler(2), covariance=np.array([[1.0, 0.5], [0.5, 1.0]]))])
    covariance = product.covariance
    sparse_equivalent = sparse.block_diag(
        [marginal.covariance for marginal in product.marginals],
        format="csr")

@sou-cheng-choi sou-cheng-choi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See example above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants