[ENH] Added support for 7B level language model learnwares.#2
Closed
Asymptotez wants to merge 165 commits intomainfrom
Closed
[ENH] Added support for 7B level language model learnwares.#2Asymptotez wants to merge 165 commits intomainfrom
Asymptotez wants to merge 165 commits intomainfrom
Conversation
[DOC] modify details in docs
[DOC] change repo url
[DOC] change html tags
[FIX, DOC] fix readthedocs config bug, modify framework svg
[FIX] update readthedocs yaml
[MNT] update readthedocs config
[MNT] add sphinx_book_theme req
[FIX] fix readthedocs req with pip
[FIX] fix readthedocs req to install all
[DOC, FIX] update badge, update autodoc config, fix no logo in doc bug
[MNT] publish 0.3.2 version
[DOC] polish contents
[DOC] modify details
Feature/llm workflow
[MNT] modify workflow details
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for 7B-level language model learnwares by introducing a new LLM workflow, benchmark configurations, and example task definitions, and bumps the package version to 0.4.0.post1.
- Added YAML task definitions for various financial datasets (cra_*, convfinqa, cd, etc.)
- Introduced
examples/dataset_llm_workflowwitheval_config.py,build_market.py, and updated benchmark code - Updated documentation (README, CHANGES.rst) to include new LLM results and versioning
Reviewed Changes
Copilot reviewed 100 out of 100 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| examples/dataset_llm_workflow/extra_tasks/flare/*.yaml | New task definitions for LLM workflows |
| examples/dataset_llm_workflow/eval_config.py | Configured benchmark sets for medical, math, and finance |
| examples/dataset_llm_workflow/build_market.py | Added LLM market-building script |
| examples/dataset_llm_workflow/benchmark/config.py | Updated LEARNWARE_FIN, commented task entries |
| examples/dataset_llm_workflow/benchmark/base.py | New Benchmark class to manage learnware datasets |
| examples/dataset_llm_workflow/README.md | Usage instructions and results for LLM workflow |
| README.md & README_zh.md | Added LLM experimental results sections |
| docs/components/market.rst | Fixed typo in add_learnware description |
| CHANGES.rst | Bumped version and listed new LLM support |
Comments suppressed due to low confidence (1)
examples/dataset_llm_workflow/extra_tasks/flare/convfinqa.yaml:2
- [nitpick] Class name 'ConvFinQA' uses PascalCase while other task function references are lowercase. Ensure consistency of naming conventions across task definitions.
class: !function flare.ConvFinQA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
How has his PR been tested?
pytest tests/test_workflowunder upper directory of learnware.Types of changes