Skip to content

[Assertions] Current expect is too permissive, refactor it. #47

@priyanshus

Description

@priyanshus

What happened?

Currently, expect is too permissive. Because MatcherChain is generic over EvalInput and contains all possible matchers, TypeScript allows:

await expect("just a string").toBeFaithful();

However, This should always be taking query, context, and response to run the assertion.

Above will crash on run time because as below piece of code checks for query.

// packages/assertions/src/expect/expect.ts:74

if (!input.query) {
  throw new EvaliphyError(
    EvaliphyErrorCode.INVALID_ASSERTION_INPUT,
    "toBeFaithful requires a query..."
  );
}

Acceptance Criteria:

  • Update tests, add new tests
  • Update docs, update website
  • Update prompts and text assertions to use only one variable

Reproduction steps

No response

Evaliphy version

No response

Relevant log output

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions