chore(MUATracePlot): temporarily limited key_source for testing/validation#197
Merged
Conversation
key_source to a high limit for testing/validationkey_source for testing/validation
ttngu207
approved these changes
Dec 11, 2025
ttngu207
requested changes
Dec 11, 2025
Contributor
ttngu207
left a comment
There was a problem hiding this comment.
The PR title is temporarily limited key_source for testing/validation - but the changes do not seem to have any effect on key_source
ttngu207
approved these changes
Dec 11, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi @ttngu207 @judewerth,
Jude and I discussed the new pipeline extension proposed in #196 to combine LFP and MUA data and manage large data volumes in blocks of sessions for coherence analysis.
The next key step we agree on is to manually insert a 1-hour block of
MUAEphysSessiondata so we can verify that the processing time forMUASpikesand the relevant metrics inChannelare feasible. We do not need to process theMUATracePlotdata at this point, as it is the bottleneck in the pipeline regarding resources and time.(1) In this PR, I modify the
MUATracePlotkey_sourceto limit plot generation, since that appears to be the current bottleneck for block analysis and data processing and storage. Once this PR is merged, I will rebuild the standard worker.(2) After completing that step, I will inform Jude so he can manually insert the 1-hour block into
MUAEphysSession(since this table is to be included in the worker flow to prevent analyzing all current data).(3) We will review this together before moving on to larger block sizes, as it is critical not to overload the system with many jobs that could impact performance.
(4) If we agree that processing the 1-hour test is time-efficient, we can then consider running a full day of data for
MUASpikesanalysis to assess how the pipeline performs with larger datasets.Based on these performance tests, we will decide how best to structure the new pipeline extension outlined in #196.