[TASK] Refactor Pipeworks Directory into Sub-Directories Based on Pipeline for the Thinking Dataset Project #68
Description
Is your feature request related to a problem? Please describe.
The current pipeworks directory lacks, making it difficult to manage and locate various pipes within the thinking dataset project. This disorganization leads to inefficiencies and potential during data processing and analysis.
the solution you'd like
Refactor pipeworks directory into-directories based on pipelines, such as upload, generate, etc. This will enhance accessibility and streamline the by categorizing pipes according to their specific functions.
alternatives you've considered
- Manual organization by team members - This approach is time-consuming prone to human.
- Using third-party software - While this might solve the issue, it introduce compatibility challenges and additional costs.
Additional context
Implementing this refactoring greatly enhance our project's overall efficiency. Screens of current directory structure and any relevant diagrams are attached for reference.
