baserun

Baserun now supports Jinja2 template baserun https://lnkd.in/gUVw23WB

Let’s go!

Let’s go!

Baserun can now evaluate & improve your prompts with AI — automatically! Try it now →  https://lnkd.in/gUVw23WB baserun

Crafting the perfect prompt can be challenging. There are tons of prompt techniques out there, but mastering them still requires lots of trial and error. What if we could make it easier? We're teaching an AI to understand all these tricks and write prompts for you. Here is a sneak peek of what's coming next. Stay tuned! https://lnkd.in/gUVw23WB baserun

Claude-3 is now available in Baserun Playground! Why Baserun Playground? - A collaborative workspace for teams to share prompts - Version history ensures you don't lose any changes - Dynamic Inputs for bulk testing - Compare prompt versions side by side Signup: https://lnkd.in/gUVw23WB baserun

Claude-3 is now available in Baserun Playground! Why Baserun Playground? - A collaborative workspace for teams to share prompts - Version history ensures you don't lose any changes - Dynamic Inputs for bulk testing - Compare prompt versions side by side Signup: https://lnkd.in/gUVw23WB baserun

This week at Baserun 3/1: Evaluation for Both Offline Testing and Production Monitoring Developers can now create evaluators using custom code or a custom LLM prompt to grade testing results pre-release or to monitor production post-release. - Support both unit tests and end-to-end tests. Like traces, users can assess the outputs for a particular LLM call or the entire pipeline. - Comparing changes side-by-side: With our comparison report, developers - can compare two branches before merging a PR to prevent regressions. - Maximum flexibility: Baserun supports automatic evaluators, human evaluators, feedback, and checks. Automatic evaluators include string match, fuzzy match, JSON validation, and Regex match. Model-graded evaluators include fact checking, closed QA, and security evaluations. Users can also define their own custom code evaluators or custom model-graded evaluators. More evaluators to come! - Saving costs on evaluation: Baserun automatically caches your evaluation results to avoid running redundant assessments. - Prototype and evaluate prompts in the Baserun UI: Anybody (not just developers!) can prototype ideas in the Playground, test and assess prompts with our Evaluation features directly in the UI, and then deploy those changes to staging or production environments using the Prompt Directory feature. https://lnkd.in/gDZ8r2Cx baserun

This week at Baserun 2/23: Custom model evaluator with the model of your choice & Prompt templating GA - New Offline Evaluation Reports: Many UI improvements have been made to how metrics and evaluation details are displayed. - Custom Model Grade Evaluation: Users can now select which OpenAI model to evaluate, including custom fine-tuned OpenAI models to grade LLM outputs. - Testing SDK Bug Fix: Fixed an issue with evaluator result synchronization. - Prompt Playground: Users can now load a pre-defined template in a playground session and save a version of the prompt template after editing and testing the new version. - JS SDK Now Supports Prompt Templates: Users can create or edit templates’ versions in either UI or code.  baserun

This week at Baserun 2/16: Custom model & Custom code evaluator. - Create evaluations tailored to meet your app's specific objectives; - Playground performance improvements: now you can stream multiple tests in parallel; - Comparison view performance improvements; - New evaluator editor UI (Beta); baserun