The news is that Langfuse, the LLM observability platform,
has made all it commercial product features available for free and open source. But first of all, what is Langfuse?

It's an alternative to Langsmith. And what is Langsmith? Langsmith is the tool with which you can evaluate your LLM applications and check that they functions as expected, as well as continuously monitoring them for performance, correctness, token consumption and cost. While Langsmith had a free plan for 1 developer seat, in order to get more seats and the full functionality and less limits you had to pay, plus it wasn't open source.

Therefore the emergence of Langfuse, which promises similar functionality but  free and opensource too if you go self-hosted, is welcome news, There's also the fully managed cloud version that you can subscribe to, well for a price.

While Langsmith has a community free plan, in order to get the full functionality coming in the shape of the Commercial package you had to pay. The news is that the company has finally doubled down on open source, and are releasing the Commercial/Pro/Enterprise Edition features as part of the the initial community package, extending the initial set of features, with :

LLM-as-a-judge evaluations
Run fully managed LLM-as-a-judge evaluations on production or development traces. Can be applied to any step within your application for step-wise evaluations.

Annotation queues
Baseline your evaluation workflow with human annotations via Annotation Queues.

Prompt experiments
Evaluate prompts and models on datasets directly in the user interface. No custom code is needed.

Playground
Test and iterate on your prompts directly in the Langfuse Prompt Playground. Tweak the prompt and the model parameters to see how different models responds to these changes inputs. This allows you to quickly iterate on your prompts and optimize them for the best results in your LLM app without having to switch between tools or use any code.

As such self-hosting it would give you those plus the features of the community package:

• MIT License
• All core platform features and APIs (observability, evaluation, prompt management, datasets, etc.)
• Scalability of Langfuse Cloud
• Deployment docs & Helm chart
• Enterprise SSO and RBAC
• Community support

Check the video bellow for a walkthrough of all Langfuse's available features.

To wrap it up, Langfuse caters for the missing links in the development of AI production grade applications; by providing
evaluation and insights into performance.

More Information

Langfuse

Langfuse Github

Related Articles

MCP Developers Summit - The Talks

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info