Platform components

Github

Chat with Founders

x-api-key

Parea AI

API Reference

Blog

HomePage

Support

A unified platform for prompt engineering and management, dataset and evaluation metric creation, and log and analytics exploration.

Overview

What is Parea AI?

Evaluation Quickstart

Monitor your LLM requests and application functions.

Monitoring Quickstart

Evaluation

Record discrete events or related events in your application from LLM requests/chains to functions.

Logging and Tracing

Enrich events with context and details to facilitate analysis and debugging.

Metadata

Images

Record feedback from users on the quality LLM results.

Feedback

Use details from a trace log to build datasets.

Dataset from Trace

From Trace to Playground

Organize logs, traces & experiments into projects.

Projects

Attach evaluations to a trace to identify failure cases

Evaluations in Trace

OpenAI auto-logging also works with the new OpenAI Assistants Python API.

OpenAI Assistant

Using templated messages in auto-traced OpenAI & Anthropic calls

Templated LLM Calls

Incorporate human review into your AI software evaluation process

Criterion

Logs View

Annotation Queue

Automatically create LLM evals aligned with manual annotations

Self-improving LLM Eval

Compare

Add functions to prompts in the playground.

Function calling

Datasets

Add evaluation metrics to the playground.

Evaluation metrics

Evaluate your prompts on a dataset with evaluation metrics.

Trigger experiments

Deploy versioned prompts and use them via the SDK.

Deployed Prompts

Instrumenting LLM frameworks & provider SDKs with Parea AI

Instrumenting your Langchain applications with Parea AI

LangChain

Instrument & test `instructor` code with Parea AI

Instructor

Instrumenting your DSPy application with Parea AI

DSPy

Instrumenting your LiteLLM proxy with Parea AI

LiteLLM Proxy

Instrumenting your SGLang application with Parea AI

SGLang

Instrumenting your OpenAI calls made through Trigger.dev with Parea

Trigger.dev

On-premise deployment of Parea in your environment

On-premise deployment of Parea in your environment via Docker

Docker

Tutorial on improving a Langchain RAG application using Parea's Evals, Tracing, and Playground.

Optimize a LangChain RAG App

How many samples are necessary to achieve good performance with DSPy?

Optimize a RAG DSPy Application

You will need an API Key and Organization to use the API/SDK.

Authentication

Get a completion response using either one of your organization's deployed prompts, or by providing completion details including prompt and inputs in the request.

Completion

Get Deployed Prompt

Log a (LLM) response to visualize inference results, or chains.

Create Dataset

Get Dataset

Add Test Cases

Record Feedback

Lists experiments given a set of filters.

List Experiments

Fetch Experiment Logs

Python

TypeScript

Automatic creation of LLM evals based on human review of responses

Self-improving, domain-specific LLM Evals

Practices to improve LLM apps component-wise

Tactics for multi-step AI app experimentation

Practical workflow focusing on continuous improvement and data-driven decision-making.

A Systematic Workflow to build Production-Ready LLM Applications

We will use Instructor in TypeScript to generate synthetic data for a question-answering task.

Synthetic Data Generation for Q&A Tasks

Haiku > GPT-4 Turbo > Opus >>> GPT-3.5 Turbo if not using parallel function calling.

Anthropic's Haiku Beats GPT-4 Turbo in Tool Use - Sometimes

How to measure the performance of retrieval applications without ground truth data.

Welcome

Guides

Tutorials

Overview

Platform components

Playground

Evaluations

Deployments

Welcome

Guides

Tutorials

​Platform components

Playground

Evaluations

Deployments

Platform components