Secknowledge2 by d4nieldev · Pull Request #24 · IBM/fms-dgt

d4nieldev · 2026-05-10T10:55:48Z

Same old SecKnowledge 2 PR

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

Copilot

Pull request overview

Adds the SecKnowledge2 public databuilder pipeline plus supporting retrievers, prompts, and template data to enable instruction/answer “reformat + evidence enrichment” workflows.

Changes:

Introduces the secknowledge2 databuilder (task schema + generation pipeline) and two example tasks (vanilla/cybersecurity).
Adds web-search retrievers (DuckDuckGo + Google/Serper) and vector-store retrievers (in-memory + Elasticsearch) plus shared search-processing base.
Adds extensive prompt/template assets and wires new optional dependency groups.

Reviewed changes

Copilot reviewed 69 out of 71 changed files in this pull request and generated 21 comments.

Show a summary per file

File	Description
tasks/public/secknowledge2/vanilla/task.yaml	Adds a vanilla SecKnowledge2 task configuration using DuckDuckGo with Google fallback.
tasks/public/secknowledge2/cybersecurity/task.yaml	Adds a cybersecurity SecKnowledge2 task configuration.
pyproject.toml	Adds `search` and `secknowledge2` optional dependency groups.
fms_dgt/public/databuilders/secknowledge2/task.py	Defines SecKnowledge2 task + datapoint schemas.
fms_dgt/public/databuilders/secknowledge2/secknowledge2.yaml	Adds a databuilder config defining pipeline blocks/specs.
fms_dgt/public/databuilders/secknowledge2/README.md	Documents pipeline concepts and usage examples.
fms_dgt/public/databuilders/secknowledge2/prompts/search/webpage_summarizer/user.txt	Adds user prompt for webpage summarization.
fms_dgt/public/databuilders/secknowledge2/prompts/search/webpage_summarizer/system.txt	Adds system prompt for webpage summarization.
fms_dgt/public/databuilders/secknowledge2/prompts/search/query_filterer/user.txt	Adds user prompt for query filtering.
fms_dgt/public/databuilders/secknowledge2/prompts/search/query_filterer/system.txt	Adds system prompt for query filtering.
fms_dgt/public/databuilders/secknowledge2/prompts/search/query_builder/user.txt	Adds user prompt for query building.
fms_dgt/public/databuilders/secknowledge2/prompts/search/query_builder/system.txt	Adds system prompt for query building.
fms_dgt/public/databuilders/secknowledge2/prompts/rewriter/retrieval/user.txt	Adds user prompt for rewrite-with-evidence path.
fms_dgt/public/databuilders/secknowledge2/prompts/rewriter/retrieval/non_adaptive_system.txt	Adds non-adaptive rewriter system prompt (with evidence).
fms_dgt/public/databuilders/secknowledge2/prompts/rewriter/retrieval/adaptive_system.txt	Adds adaptive rewriter system prompt (with evidence).
fms_dgt/public/databuilders/secknowledge2/prompts/rewriter/no_retrieval/user.txt	Adds user prompt for rewrite-without-evidence path.
fms_dgt/public/databuilders/secknowledge2/prompts/rewriter/no_retrieval/non_adaptive_system.txt	Adds non-adaptive rewriter system prompt (no evidence).
fms_dgt/public/databuilders/secknowledge2/prompts/rewriter/no_retrieval/adaptive_system.txt	Adds adaptive rewriter system prompt (no evidence).
fms_dgt/public/databuilders/secknowledge2/prompts/judge/readability/user.txt	Adds readability judge user prompt.
fms_dgt/public/databuilders/secknowledge2/prompts/judge/readability/system.txt	Adds readability judge system prompt.
fms_dgt/public/databuilders/secknowledge2/prompts/judge/factuality/user.txt	Adds factuality judge user prompt.
fms_dgt/public/databuilders/secknowledge2/prompts/judge/factuality/system.txt	Adds factuality judge system prompt.
fms_dgt/public/databuilders/secknowledge2/prompts/classifier/subcategory/user.txt	Adds subcategory classifier user prompt.
fms_dgt/public/databuilders/secknowledge2/prompts/classifier/subcategory/system.txt	Adds subcategory classifier system prompt.
fms_dgt/public/databuilders/secknowledge2/prompts/classifier/category/user.txt	Adds category classifier user prompt.
fms_dgt/public/databuilders/secknowledge2/prompts/classifier/category/system.txt	Adds category classifier system prompt.
fms_dgt/public/databuilders/secknowledge2/helper/schemas.py	Adds Pydantic schemas for prompt directory structure.
fms_dgt/public/databuilders/secknowledge2/helper/categories.py	Adds template/category loading + lookup helpers.
fms_dgt/public/databuilders/secknowledge2/generate.py	Implements the full SecKnowledge2 generation pipeline.
fms_dgt/core/retrievers/unstructured_text/web_search/google_serper.py	Adds Google (Serper.dev) web search retriever implementation.
fms_dgt/core/retrievers/unstructured_text/web_search/duckduckgo.py	Adds DuckDuckGo web search retriever implementation.
fms_dgt/core/retrievers/unstructured_text/web_search/base.py	Adds shared base for web search + webpage processing.
fms_dgt/core/retrievers/unstructured_text/vector_store/in_memory.py	Adds in-memory vector-store retriever (LangChain-based).
fms_dgt/core/retrievers/unstructured_text/vector_store/elastic.py	Adds Elasticsearch-based vector retriever.
fms_dgt/core/retrievers/unstructured_text/base.py	Adds base interfaces/data objects for unstructured text retrievers.
fms_dgt/core/retrievers/registry.py	Adds registry/helpers for retriever dynamic loading.
data/public/secknowledge2/templates/vanilla/summarization.json	Adds vanilla template definitions (summarization).
data/public/secknowledge2/templates/vanilla/specialized_educational_dialog.json	Adds vanilla template definitions (specialized educational dialog).
data/public/secknowledge2/templates/vanilla/rewriting.json	Adds vanilla template definitions (rewriting).
data/public/secknowledge2/templates/vanilla/others.json	Adds vanilla template definitions (others).
data/public/secknowledge2/templates/vanilla/generation.json	Adds vanilla template definitions (generation).
data/public/secknowledge2/templates/vanilla/extraction.json	Adds vanilla template definitions (extraction).
data/public/secknowledge2/templates/vanilla/conversation.json	Adds vanilla template definitions (conversation).
data/public/secknowledge2/templates/vanilla/code.json	Adds vanilla template definitions (code).
data/public/secknowledge2/templates/vanilla/classification.json	Adds vanilla template definitions (classification).
data/public/secknowledge2/templates/vanilla/brainstorming.json	Adds vanilla template definitions (brainstorming).
data/public/secknowledge2/templates/security/XFEReportsFlan.json	Adds security templates (XFE reports).
data/public/secknowledge2/templates/security/WikiQAFlan.json	Adds security templates (WikiQA variants).
data/public/secknowledge2/templates/security/wiki_evol.json	Adds security templates (wiki evol).
data/public/secknowledge2/templates/security/StackExchangeFlan.json	Adds security templates (StackExchange).
data/public/secknowledge2/templates/security/SigmaFlan.json	Adds security templates (Sigma).
data/public/secknowledge2/templates/security/SecurityInterviewFlan.json	Adds security templates (security interview).
data/public/secknowledge2/templates/security/se_evol.json	Adds security templates (SE evol).
data/public/secknowledge2/templates/security/QradarTtpMappingFlan.json	Adds security templates (QRadar TTP mapping).
data/public/secknowledge2/templates/security/MitreFlan.json	Adds security templates (MITRE).
data/public/secknowledge2/templates/security/cybersecurity_sec_topics.json	Adds security templates (security topics).
data/public/secknowledge2/templates/security/cybersecurity_qa_evol.json	Adds security templates (QA evol).
data/public/secknowledge2/templates/security/CWE_Flan.json	Adds security templates (CWE).
data/public/secknowledge2/templates/security/CVEBasic.json	Adds security templates (CVE).
data/public/secknowledge2/templates/security/cti_evol.json	Adds security templates (CTI evol).
data/public/secknowledge2/templates/security/cissp_evol.json	Adds security templates (CISSP evol).
data/public/secknowledge2/templates/security/CISSFlan.json	Adds security templates (CISS).
data/public/secknowledge2/templates/security/CAPECFlan.json	Adds security templates (CAPEC).
data/public/secknowledge2/templates/security/BronFlan.json	Adds security templates (BRON).
data/public/secknowledge2/templates/security/AtomicRedTeaming.json	Adds security templates (Atomic Red Teaming).
data/public/secknowledge2/templates/security/apt-notes.json	Adds security templates (APT notes).
data/public/secknowledge2/example_data_vanilla.json	Adds example seed dataset for vanilla task.
data/public/secknowledge2/example_data_security.json	Adds example seed dataset for security task.
.github/CODEOWNERS	Adds CODEOWNERS entries for SecKnowledge2 and retrievers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        if not isinstance(limit, int) and limit <= 0:
+            raise ValueError("Must specify 'limit' field as an integer and greater than 0.")


+            # Step 2.a: Verify connection field
+            if CONNECTION_FIELD_ENDPOINT not in connection:
+                raise ValueError("Missing mandaroty 'endpoint' field in the connection field.")
+
+            if CONNECTION_FIELD_API_KEY not in connection or (
+                CONNECTION_FIELD_USERNAME not in connection
+                and CONNECTION_FIELD_PASSWORD not in connection
+            ):
+                raise ValueError(
+                    "Either 'api_key' or 'username' and 'password' fields must be specified in the connection field."
+                )


+            # Step 2.b.i: Check if SSL fingerprint is provided
+            if (
+                connection[CONNECTION_FIELD_SSL_FINGERPRINT]
+                and connection[CONNECTION_FIELD_SSL_FINGERPRINT]
+            ):
+                es_client_parameters["ssl_assert_fingerprint"] = connection[
+                    CONNECTION_FIELD_SSL_FINGERPRINT
+                ]
+            else:
+                es_client_parameters["verify_certs"] = False


+            # Step 2.a.ii: Execute query
+            try:
+                response = self._client.search(
+                    index=self._index_name,
+                    **query,
+                    size=limit,
+                )
+            except BadRequestError:
+                dgt_logger.warning("Incorrect request: %s", json.dumps(query))
+


+
+                    if metadata:
+                        processed_results.metadata = metadata
+
+                    # Add created document
+                    processed_results.append(processed_result)


+{structure}
+
+
+Please summarize the web page according to the insturcions.


+• Original question – the original user question  
+• Draft answer – a draft answer (may be incomplete)  
+• Required structure – formatting rules for the final answer  
+• Candidate search queries – N candidate Google search queries seperated by newline


+1. First, think step‑by‑step (do NOT reveal this reasoning).
+2. Identify the distinct information needed required to answer the question.
+3. Craft 1–{K} search phrases (5‑12 words each, no punctuation except quotes for phrases, no question marks).
+4. The search queries should differ from one another as much as possible in order to yield different search results and provide better coverage over the problem space. The larger word overlap between different search queries, the more likely they are to yeild similar or even identical results, which we want to avoid - so make sure you use different words for different search queries.


+You will be given the following inputs:
+1. Question - bounded by <question> and </questions>
+2. Response - bounded by <original_response> and </original_response>
+3. Evidence - bounded by <evidence> and </evidence>. Evidence is supplementary facts and information you can use to enrich the original response.


+You will be given the following inputs:
+1. Question - bounded by <question> and </questions>
+2. Response - bounded by <original_response> and </original_response>
+3. Template - bounded by <template> and </template>. The template describes the desired response format you should output.


d4nieldev added 16 commits November 9, 2025 09:14

added retrievers

3d57265

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

added data

b7b5516

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

added pipeline

75177f0

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

added tasks

639afb6

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

renamed realign to secknowledge2

1a41992

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

added pipeline dependencies

e9831be

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

broke down vanilla to categories

4a2ad2f

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

unpacked security templates

a51adaf

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

reorganized templates folder structure

edc1eb2

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

moved templates dir parameter to task (instead of databuilder)

82dc85a

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

updated readme

07eec2e

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

updated tasks

9ea11fc

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

changed llm provider to watsonx

62728aa

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

updated code owners

b772580

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

updated lock

bafd8bd

Signed-off-by: Daniel Ohayon <danielohayon2019@gmail.com>

Migrated changes from private repo

6c7d596

Copilot AI review requested due to automatic review settings May 10, 2026 10:55

Copilot started reviewing on behalf of d4nieldev May 10, 2026 10:56 View session

Copilot AI reviewed May 10, 2026

View reviewed changes

d4nieldev added 2 commits May 18, 2026 18:27

undeleted uv.lock

0d57bfd

Merge remote-tracking branch 'upstream/main' into secknowledge2

5c734b1

d4nieldev force-pushed the secknowledge2 branch from c803c3c to 5c734b1 Compare May 18, 2026 15:34

Navigation Menu

Uh oh!

Secknowledge2 - #24

Secknowledge2#24
d4nieldev wants to merge 18 commits into
IBM:mainfrom
d4nieldev:secknowledge2

d4nieldev commented May 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if not isinstance(limit, int) and limit <= 0:
		raise ValueError("Must specify 'limit' field as an integer and greater than 0.")

		{structure}


		Please summarize the web page according to the insturcions. No newline at end of file

Uh oh!

Conversation

d4nieldev commented May 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants