Request to Open Source LLM Prompt Logic for Vulnerability Detection

[jmlvega]

First of all, thank you for the great work on this project! The use of LLMs to identify potential vulnerabilities not reported to CVE is a valuable and innovative contribution to the open source security ecosystem.

I’d like to request that, beyond publishing the intel, the project consider open-sourcing the logic behind the LLM component. Specifically:

• The prompts or instructions used to guide the LLM.
• Any rules, post-processing logic, or filtering heuristics.
• Examples of expected input/output formats.
• Evaluation criteria or fine-tuning strategies.

Why this would help:

• Transparency: Sharing the LLM logic would allow the broader community to better understand how vulnerabilities are detected.
• Collaboration: Developers and researchers could contribute improvements to the prompts, logic, or detection accuracy.
• Broader Impact: Organizations with private or proprietary codebases could apply this logic to improve their own vulnerability detection and prioritization, even where CVEs may not apply.

Thanks again for your work, and looking forward to your thoughts!

[sampion88]

Thanks for your input!
We are definitely open to share information like that. Atm, we are considering what we can expose and how we would do that.
Coming back to you later...

[pombredanne]

this would be great.