Open
Conversation
Introduces a curated gallery of 8 pre-built flow templates that showcase Flowfile's ETL capabilities. Templates use read nodes pointing to CSV files downloaded from GitHub and cached locally in ~/.flowfile/template_data/. Backend: - Template definitions with 8 flows (Beginner/Intermediate/Advanced) - Data downloader that fetches CSVs from GitHub on first use - GET /templates/ and POST /templates/create_from_template/ endpoints - template_data_directory added to FlowfileStorage Frontend: - TemplatesView with category filtering and template cards - Templates API service - Sidebar navigation entry and designer empty state integration - Transparency notice about data download from GitHub Templates: Sales Overview, Customer Dedup, Employee Cleanup, Order Join, Survey Pivot, Web Analytics Funnel, Customer 360, Fuzzy Match https://claude.ai/code/session_01LPWpAV5Xqk9ZJUTSPQytaZ
Templates are now defined as YAML files in data/templates/flows/ instead of being constructed programmatically in Python. This is cleaner, easier to maintain, and consistent with how flows are stored throughout the project. - 8 flow YAML files with embedded _template_meta and _required_csv_files - template_definitions.py now loads and parses YAMLs with lazy registry - data_downloader.py supports downloading flow YAMLs for PyPI installs - New GET /templates/ensure_available/ endpoint for production environments - Frontend calls ensureAvailable() before listing templates https://claude.ai/code/session_01LPWpAV5Xqk9ZJUTSPQytaZ
✅ Deploy Preview for flowfile-wasm canceled.
|
Edwardvaneechoud
changed the title
Resolve conflict in routes.py: keep both new db_schemas/db_tables endpoints from main and template endpoints from this branch. https://claude.ai/code/session_01LPWpAV5Xqk9ZJUTSPQytaZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several new data flow templates and a sample dataset to support data cleaning, enrichment, and joining operations. These templates are designed to demonstrate common data engineering tasks such as deduplication, enrichment, fuzzy matching, and directory cleanup using CSV files as input.
The most important changes are:
New Data Flow Templates:
employee_directory_cleanup.yamltemplate, which reads employee data, creates a full name column, filters to active employees, and sorts by department and salary.customer_360.yamltemplate to join customers, orders, and support tickets, aggregate metrics, compute a customer score, and sort results.order_enrichment.yamltemplate to join orders with product details, calculate total price, and select relevant columns.customer_deduplication.yamltemplate to deduplicate customers by email and clean up column names.product_fuzzy_match.yamltemplate to perform fuzzy matching between internal and supplier product catalogs and filter matches by score.Sample Data:
employees.csvdataset with fields such asemp_id,first_name,last_name,department,status, andsalary, which is used by the new employee directory cleanup flow.