v0.3.2

Changelog

• fix(deps): allow openssl 4.x for compatibility (#151) (b5dcaa9)

v0.3.1

Changelog

• Added support for Ruby 4.0 (#129)
• Added parameter support for remote evals (#133)
• Added project_id support to Functions#list and Prompt.load (#131) (Nick Giancola (@patbenatar))
• Added Prompt#version (#131) (Nick Giancola (@patbenatar))
• Addressed some Eval edge cases with improved defaults (#127)
• Some internal refactors for gemspec and Evals (#128)

v0.3.0

Changelog

• Added a Rails engine for mounting a remote eval server onto your existing Rails applications (#87 #114) (Gabriel Kent (@gabe-kent), Alan R. Fachini (@alfakini))
• Scorers can return metadata tags as a part of their result (#115, #120) (Alan R. Fachini (@alfakini))
• Scorers can return multiple scores from a single scorer (#125)
• Adjustments to eval traces to make them consistent with ones from other SDKs (#119, #121)
• Fixed filter_ai_spans being bypassed for root spans (#123, #124) (Chidi Williams (@chidiwilliams))

v0.2.1

Changelog

• Added support for trace: in Scorer blocks: this allows you to score an Eval case by looking at the LLM activity during the task, not just the input or output. (#112)
• A Scorer can now be retrieved by its name + project name (not just by ID) (#113)
• Scorer and Task now support keyword arguments for their blocks (e.g. |output:, expected:| instead of |input, output expected|) to allow for more concise expressions. (#111)
  • Using positional arguments are still supported but will produce a deprecation warning.
• Scorer and Task are now independent and accessible outside of Eval (#111)
  • The scorer and task functions in the Eval namespace are still supported will produce a deprecation warning.

v0.2.0

Changelog

• Added dev server support for running remote evals in Ruby applications. See setup instructions for more details. A special thanks to Alan R. Fachini (@alfakini) and Gabriel Kent (@gabe-kent) for their contributions to this new feature! (#87, #108) (Alan R. Fachini (@alfakini) Gabriel Kent (@gabe-kent))

v0.1.4

Changelog

• Scorers using invoke api now support objects instead of just strings (#104) (Barrett Pyke (@barrettpyke) Almir Davletov (@davletovalmir))
• Fixed API calls not following redirects for Evals (#101 #106) (Alan R. Fachini (@alfakini))
• Link experiments with datasets when using remote datasets (#103) (Alan R. Fachini (@alfakini))

v0.1.3

Changelog

• Added origin tagging to Eval datasets (for proper attribution) (#97)
• Datasets are now a first-class API (#98)
• Evals no longer accept state, use api objects instead (#98)
• Fixed Anthropic instrumentation not recording system prompts correctly (#100) (Jessica Alder (@jessicaalder-lgtm))

v0.1.2

Changelog

• Added support for prompts (#70, #77)

v0.1.1

Changelog

• Added moderations API support for openai and ruby-openai (#89, #92)
• Added support for anthropic messages beta API (#90, #93)
  • This is "best effort" support only. Since the Anthropic API is "beta" and may change between minor versions, instrumentation may break unexpectedly and support for beta features may be limited.
• The tracer now attempts to flush traces at the end of the process by default. Useful for short-lived processes like lambdas. (#91)

v0.1.0

Changelog

This release introduces auto-instrumentation to make installation quick & easy, a new Integration API for plugging in new instrumentation, as well as numerous bug fixes and minor enhancements to existing instrumentation (new fields, attributes, etc.)

New users can setup their applications by just setting their API key in their environment, then...

For Rails applications (or ones using Bundler.setup), add the following to your Gemfile:

gem "braintrust", require: "braintrust/setup"

Otherwise, for other Ruby applications, add the following to your application:

require "braintrust/setup"

We encourage existing users to switch over to the new auto-instrumentation setup, or if you prefer manual instrumentation, the new instrumentation API. The older API for instrumentation (e.g. .wrap(client)) is now deprecated and will raise warnings when used. It will be removed in the future.