Skip to content

Fix Entropy Calculation #4

Merged
GFJHogue merged 2 commits intomainfrom
xindi_development
Aug 19, 2025
Merged

Fix Entropy Calculation #4
GFJHogue merged 2 commits intomainfrom
xindi_development

Conversation

@xindizhang
Copy link
Copy Markdown
Contributor

Fix Entropy Calculation

  • Before: calculate entropy and average entropy across folds
  • Now: calculate averaged probability distribution and calculate entropy based on ensemble probability

@GFJHogue
Copy link
Copy Markdown
Collaborator

Thanks @xindizhang!
I ended up moving the entropy calculations to use PyTorch's built-in torch.Categorical.entropy() implementation for i) a stronger guarantee of correctness and ii) less computational error for probabilities near-zero.
I also added type-hints to make future debugging easier.

In my own testing, all probability outputs are identical to v3.0.1 (good) and entropy values between Xindi's suggested code vs. my update to use torch.Categorical.entropy() match to 6 decimal places (also good).

@GFJHogue GFJHogue merged commit 6aece24 into main Aug 19, 2025
1 check passed
@GFJHogue GFJHogue deleted the xindi_development branch August 19, 2025 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants