Skip to content

Refactoring 2026 - #188

Open
sjanssen2 wants to merge 16 commits into
masterfrom
trajectory
Open

Refactoring 2026#188
sjanssen2 wants to merge 16 commits into
masterfrom
trajectory

Conversation

@sjanssen2

Copy link
Copy Markdown
Owner

This PR is a larger and partially braking braking change!

  • functions in analyses.py use the _executor function which in turn uses a caching mechanism to avoid re-computation. The cache hash computation for pandas Series / DataFrames was flawed for many years as I simply used the str() function to then compute a hash of this representation. For larger DataFrames, ellipsis were returned by str(), such that only the first and last 10 rows / cols were actually included in the string. Changes in the larger rest of data did not effect the hash, i.e. collisions were potentially very HIGH / likely. I've changes that, which means for you, that ggmap most likely will not find existing cached results in your analysis working directory and you need to re-compute (or rename the cached files accordingly). No results were wrong at any time!
  • execute.py: I decided to re-structure my spaghetti code a bit, i.e. move all functionality necessary to run external programs to the new file "execute.py"
  • analyses.py:
    • I've harmonized the way how verbose is handled as an argument.
    • older versions of sepp (which I think nobody used for long time) are now moved to deprecated.py
    • blastn_local can now handle LARGE result hitlists by applying filters "early", i.e. while parsing individual chunks
  • correlations.py: annoyed by manually constructing / testing "formulas" for the linear model for forward step redundancy analyses for effect size computation and the fact that I had to call alpha- and beta- versions manually, I put everything in one function redundancy and implemented an "auto" mode which first iterates ALL metadata columns (except those defined in omit) and computes individual effects and than heuristically composes a model that explains most of variability. Note: a smart manual "design" might still out compete this automatic mechanism and you might want to iteratively add columns to omit if you know about confounders!
  • snippets.py: I've added three more functions to visualize alpha- / beta- and ancom results of testing for differences for a metadata column hue (e.g. healthy/diseased) across multiple timepoints time (e.g. days after intervention) in an optionally "stratified" (e.g. body site) fashion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant