Skip to content

update dev with gh pages - #47

Merged
evolp merged 17 commits into
devfrom
master
Jun 10, 2026
Merged

update dev with gh pages#47
evolp merged 17 commits into
devfrom
master

Conversation

@evolp

@evolp evolp commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

evolp and others added 17 commits October 10, 2024 12:03
Add performance optimizations
New `Reads` struct and revamped summarizers

- new `reads::Reads` struct for compact reads storage
- new handling of ambiguous bases
- move summarize methods to `SummaryData`
- add new functionality to `SummaryData` (p-values, edge multiplicities)
- `KmerSummarizer` removed
New DOT edge format

- customizable format for `dot` edges
- `Colors` (from `dbg`) to calculate variable colors
- default methods for formatting edges and nodes: `Node::edge_dot_default` and `node_dot_default`
- fix nucleotide order in `EdgeMult`
Paired & Stranded reads

paired: 
- `ReadsPaired` stuct to store up to three `Reads` in and how they are paired
- methods to iterate over the reads

stranded: 
- `Stranded` enum stored in `Reads` stuct
- adaption of `filter_kmers` and `filter_kmers_parallel` to 
   - keep forward reads as they are
   - reverse  complement reverse reads
   - min rc unstranded reads

fix:
- add `DebruijnGraph::fix_edge_mults` to remove multiplicities of filtered nodes
`EdgeMult` compression 

- add `EdgeMult` compatibility with graph compression
- remove parallel compression methods
new `IDSumData` summarizer, restructure `Colors` with `ColorMode` enum, new `serde` module

new `IDSumData` summarizer
- track IDs as `u16`s (e.g. gene IDs)

restructure `Colors` with `ColorMode` enum
- color dot nodes according to their tracked ID or their supergroup (e.g. orthogroups) 
- set the color mode (mentioned above + fold change + sample group) with the `ColorMode` enum

new `serde` module
- easy to use structs + methods for serializing and deserializing reads, k-mers, and graphs
More summarizers, `id4b` feature

new summarizers:
- `TagsData`: only 1-bit enocoded tags
- `IDData`: only IDs
- `IDTagsCountsData` and `IDTagsCountsPEMData`: both IDs and tags +  Counts etc.

new `id4b` feature:
- previously there could be max. 65k unique IDs (stored in `u16`)
- with the feature (compile flag) `id4b`, IDs are instead stored in `u32`s, raising this maximum to 4 billion
- all k-mer sizes up to 64
- new method  `DebruijnGraph::to_dot_with_paths`: prints to dot while highlighting a path
- new method `DebruijnGraph::iter_edges`: iterates over all edges of the graph
- new method `DebruijnGraph::filter_edges`: removes edges below certain coverage
Transcript Mapping
- new `IDEMData` summarizer: can hold (gene) IDs and edge coverage
- new option to translate (group) IDs into ID-groups (orthogroups) when printing node info
- new `DebruijnGraph::map_transcripts` method: maps transcripts from a fasta file to a stranded & uncompressed graph
- new `IDMapEMData` summarizer: can hold additional mapped IDs
Removing ladders +  tips
Optional `Exts` in `Reads`, memory limit fixes
Utilize PHRED-scores for error removal, add 3D json compatibility

### Utilize PHRED-scores for error removal

- the quality scores from the sequencing reads can be stored in a `Reads`
- the scores are binned into four categories in accordance with [this](https://www.illumina.com/content/dam/illumina-marketing/documents/products/appnotes/novaseq-hiseq-q30-app-note-770-2017-010.pdf) document on Illumina NovaSeq Quality Scores.
- it can be used to assess the validity of each k-mer during graph construction
  - option 1: only include k-mers which have occurred with all bases having at least a specific quality
  - option 2: only include edges of k-mer occurrences with all bases having at least a specific quality, unless this would disconnect the k-mer on one or both sides
- the quality scores can also be saved in the graph nodes by using the `SummaryData` implementations `IDMapEMQualityData`,`SumMapEMQualityData`, or `TagsCountsPEMQualityData` - this can be used to apply two error removal algorithms to the graph
  - `DebruijnGraph::remove_lq_ladders_tips` will remove any ladders (bubbles) and tips which are not supported by good quality k-mers. 
  - `DebruijnGraph::remove_lq_splits` is more radical and will identify nodes at which the path splits into a good and bad quality path, or at which a good and bad quality path merge, and remove the connection to the bad quality path

### 3D JSON compatibility

- a `DebruijnGraph` can now be saved to a JSON file compatible with [https://github.com/vasturiano/3d-force-graph](https://github.com/vasturiano/3d-force-graph)
- default functions for node and edge format, which include all node information, are available: `Node::node_json_default` and `Node::edge_json_default`
Memory fixes, extended error removal methods, add `EdgeMap`

- increase memory efficiency of `BaseGraph`
- remove disconnection check in k-mer quality edge filter during graph construction
    - the filter now excludes all k-mer occurrences with a low quality, even if this removes the k-mer from the graph or disconnects it on one side
 -  implement default methods in `SummaryData` and remove them from the trait implementations
 - add `DebruinGraph::map_transcripts_to_edges` which maps sequences from a FASTA reference to the edges of a graph instead of the nodes
 - add `DebruijnGraph::remove_lc_paths` which works similar to `remove_lq_paths` but uses the edge coverage instead of the node quality
 - adapt error removal and reference mapping algorithms to be able to handle unstranded graphs
Publish the Rust documentation for the crate on GitHub Pages
@evolp
evolp marked this pull request as ready for review June 10, 2026 09:38
@evolp
evolp merged commit 02efd4c into dev Jun 10, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants