Skip to content

[Core] Refactor AssertionEngine to address issues from description #43

@priyanshus

Description

@priyanshus

What happened?

Issues

  • Dead Code: Unused searchPaths Variable Line No - 98
  • Probably this will cause an issue but before fixing make sure we test it throughly.
    const sdkPromptsDirSource = path.resolve(__dirname, '../../prompts');
    const sdkPromptsDirDist = path.resolve(__dirname, './prompts');
  • Centralize Prompt Discovery: Move the logic for resolving prompt file paths into a dedicated service or into PromptLoader. The AssertionEngine should not be concerned with file system path resolution.
  • Update unit tests.

Context around prompts

  • Evaliphy consumers can provide their own prompts through evaliphy.config.ts file. There is a key promptsDir which should be respected if set by user.
  • If prompts are not available under promptsDir , the SDK should fallback to prompts shipped by it.

Reproduction steps

No response

Evaliphy version

No response

Relevant log output

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinggood first issueGood for newcomershelp wantedExtra attention is needed

Type

No fields configured for Bug.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions