# Parea AI

## Docs

- [Authentication](https://docs.parea.ai/api-reference/authentication.md): You will need an API Key and Organization to use the API/SDK.
- [Create Collection](https://docs.parea.ai/api-reference/datasets/create-collection.md): Creates a dataset
- [Create Test Cases For Dataset](https://docs.parea.ai/api-reference/datasets/create-test-cases-for-dataset.md): Adds items/test cases to a dataset. If the dataset does not exist, it will be created. Returns the IDs of the created items/test cases.
- [Get Test Case Collection By Identifier](https://docs.parea.ai/api-reference/datasets/get-test-case-collection-by-identifier.md): Fetches dataset/test case collection by its ID or name.
- [Update Test Case](https://docs.parea.ai/api-reference/datasets/update-test-case.md): Update the item/test case of the dataset with any given fields.
- [Create Experiment](https://docs.parea.ai/api-reference/experiments/create-experiment.md): Create an experiment and get the associated experiment uuid.
- [Finish Experiment](https://docs.parea.ai/api-reference/experiments/finish-experiment.md): Finishes an experiment, calculates stats and returns all stats for root trace logs
- [Get Experiment Stats](https://docs.parea.ai/api-reference/experiments/get-experiment-stats.md): Fetches aggregated stats for every root-level trace log in an experiment.
- [Get Experiment Trace Logs](https://docs.parea.ai/api-reference/experiments/get-experiment-trace-logs.md): Fetches all trace logs for an experiment.
- [List Experiments](https://docs.parea.ai/api-reference/experiments/list-experiments.md): Lists experiments given a set of filters incl. their high-level statistics.
- [Create or get Project](https://docs.parea.ai/api-reference/general/create-or-get-project.md): Create a project or get the project if it already exists. Is used to find out UUID of project.
- [Health Check](https://docs.parea.ai/api-reference/general/health-check.md)
- [Completion](https://docs.parea.ai/api-reference/llm-proxy/completion.md): Get a completion response using either one of your organization’s deployed prompts, or by providing completion details including prompt and inputs in the request. This endpoint acts as a LLM gateway/proxy endpoint to generate completions from different LLMs.
- [Fetch Deployed Prompt](https://docs.parea.ai/api-reference/llm-proxy/fetch-deployed-prompt.md): Given a deployment_id, fetches the deployed prompt and its details. Can be optionally used to fill-in the templated prompt with provided inputs.
- [Stream Completion](https://docs.parea.ai/api-reference/llm-proxy/stream-completion.md): Get a completion response using either one of your organization’s deployed prompts, or by providing completion details including prompt and inputs in the request.
- [Python](https://docs.parea.ai/api-reference/sdk/python.md)
- [TypeScript](https://docs.parea.ai/api-reference/sdk/typescript.md)
- [Get Trace Log](https://docs.parea.ai/api-reference/tracing/get-trace-log.md): Fetches a trace log by its UUID.
- [Get Trace Logs](https://docs.parea.ai/api-reference/tracing/get-trace-logs.md): Fetches trace logs and returns them as a paginated response.
- [Record Feedback](https://docs.parea.ai/api-reference/tracing/record-feedback.md): Record any (user) feedback & ground truth/correction of output for a log.
- [Record Trace Log](https://docs.parea.ai/api-reference/tracing/record-trace-log.md): Log a (LLM) span to visualize inference results, or chains.
- [Update Trace Log](https://docs.parea.ai/api-reference/tracing/update-trace-log.md): Update fields of a trace log.
- [Anthropic's Haiku Beats GPT-4 Turbo in Tool Use - Sometimes](https://docs.parea.ai/blog/benchmarking-anthropic-beta-tool-use.md): Haiku > GPT-4 Turbo > Opus >>> GPT-3.5 Turbo if not using parallel function calling.
- [Building and Evaluating Evals for Retrieval](https://docs.parea.ai/blog/building-and-evaluating-evals-for-retrieval.md): How to measure the performance of retrieval applications without ground truth data.
- [How has the ChatGPT model changed from March to June?](https://docs.parea.ai/blog/chatgpt-model-changed-from-march-to-june-2023.md)
- [Evaluation Metrics for LLM Applications In Production](https://docs.parea.ai/blog/eval-metrics-for-llm-apps-in-prod.md): How to measure the performance of LLM applications without ground truth data.
- [Hill climbing generative AI problems](https://docs.parea.ai/blog/hill-climbing-llm-summarization/label-data-and-improve-llm-apps.md): When ground truth values are expensive to obtain & launching fast is important
- [How to detect unreliable behavior of LLM apps](https://docs.parea.ai/blog/identify-unreliable-llm-app-inputs-of-production-traffic.md): LLM logs are useful but it's hard to prioritize which production log to review. High entropy responses are a good starting point.
- [Tactics for multi-step AI app experimentation](https://docs.parea.ai/blog/llm-app-multi-step-experimentation-tactics.md): Practices to improve LLM apps component-wise
- [LLM Evaluation Metrics for Labeled Data](https://docs.parea.ai/blog/llm-eval-metrics-for-labeled-data.md): How to measure the performance of LLM applications with ground truth data.
- [Self-improving, domain-specific LLM Evals](https://docs.parea.ai/blog/self-improving-domain-specific-alligned-llm-evals-with-human-judgement.md): Automatic creation of LLM evals based on human review of responses
- [Synthetic Data Generation for Q&A Tasks](https://docs.parea.ai/blog/synthetic-data-generation-instructor-typescript.md): We will use Instructor in TypeScript to generate synthetic data for a question-answering task.
- [A Systematic Workflow to build Production-Ready LLM Applications](https://docs.parea.ai/blog/workflow-for-production-llm-apps.md): Practical workflow focusing on continuous improvement and data-driven decision-making.
- [Evaluation](https://docs.parea.ai/evaluation/overview.md): Test & evaluate your LLM application
- [DSPy](https://docs.parea.ai/integrations/dspy/dspy.md): Instrumenting your DSPy application with Parea AI
- [Instructor](https://docs.parea.ai/integrations/instructor/instructor.md): Instrument & test `instructor` code with Parea AI
- [LangChain](https://docs.parea.ai/integrations/langchain.md): Instrumenting your Langchain applications with Parea AI
- [LiteLLM Proxy](https://docs.parea.ai/integrations/liteLLM.md): Instrumenting your LiteLLM proxy with Parea AI
- [Overview](https://docs.parea.ai/integrations/overview.md): Instrumenting LLM frameworks & provider SDKs with Parea AI
- [SGLang](https://docs.parea.ai/integrations/sglang.md): Instrumenting your SGLang application with Parea AI
- [Trigger.dev](https://docs.parea.ai/integrations/trigger-dev.md): Instrumenting your OpenAI calls made through Trigger.dev with Parea
- [Self-improving LLM Eval](https://docs.parea.ai/manual-review/bootstrapped-eval.md): Automatically create LLM evals aligned with manual annotations
- [Criterion](https://docs.parea.ai/manual-review/criterion.md): Define how responses should be annotated
- [Logs View](https://docs.parea.ai/manual-review/logs-view.md): Annotate & comment on logs.
- [Overview](https://docs.parea.ai/manual-review/overview.md): Incorporate human review into your AI software evaluation process
- [Annotation Queue](https://docs.parea.ai/manual-review/queue.md): Annotate & label in a queue
- [Templated LLM Calls](https://docs.parea.ai/observability/auto-trace-templated-messages.md): Using templated messages in auto-traced OpenAI & Anthropic calls
- [Dataset from Trace](https://docs.parea.ai/observability/dataset_from_trace.md): Use details from a trace log to build datasets.
- [Evaluations in Trace](https://docs.parea.ai/observability/evals-in-trace.md): Attach evaluations to a trace to identify failure cases
- [Feedback](https://docs.parea.ai/observability/feedback.md): Record feedback from users on the quality LLM results.
- [Images](https://docs.parea.ai/observability/images.md): Add images to trace logs.
- [Logging and Tracing](https://docs.parea.ai/observability/logging_and_tracing.md): Record discrete events or related events in your application from LLM requests/chains to functions.
- [Metadata](https://docs.parea.ai/observability/metadata.md): Enrich events with context and details to facilitate analysis and debugging.
- [From Trace to Playground](https://docs.parea.ai/observability/open_trace_in_playground.md): Open a LLM trace in the Playground.
- [OpenAI Assistant](https://docs.parea.ai/observability/openai_assistants_api.md): OpenAI auto-logging also works with the new OpenAI Assistants Python API.
- [Overview](https://docs.parea.ai/observability/overview.md)
- [Projects](https://docs.parea.ai/observability/projects.md): Organize logs, traces & experiments into projects.
- [Deployed Prompts](https://docs.parea.ai/platform/deployment.md): Deploy versioned prompts and use them via the SDK.
- [Compare](https://docs.parea.ai/platform/playground/compare.md): Iterate on and compare prompt versions.
- [Evaluation metrics](https://docs.parea.ai/platform/playground/evaluation_metrics.md): Add evaluation metrics to the playground.
- [Function calling](https://docs.parea.ai/platform/playground/functions.md): Add functions to prompts in the playground.
- [Datasets](https://docs.parea.ai/platform/playground/test_collection.md): Add datasets to the playground.
- [Trigger experiments](https://docs.parea.ai/platform/playground/trigger-experiments.md): Evaluate your prompts on a dataset with evaluation metrics.
- [Datasets](https://docs.parea.ai/platform/test_hub/overview.md): Create test case datasets and benchmarks
- [Docker](https://docs.parea.ai/self-hosting/docker.md): On-premise deployment of Parea in your environment via Docker
- [Overview](https://docs.parea.ai/self-hosting/overview.md): On-premise deployment of Parea in your environment
- [REST API Walkthrough](https://docs.parea.ai/tutorials/api-only/cookbook.md): Trace your app & log experiments via API
- [Optimize a RAG DSPy Application](https://docs.parea.ai/tutorials/dspy-rag-trace-evaluate/tutorial.md): How many samples are necessary to achieve good performance with DSPy?
- [Optimize a LangChain RAG App](https://docs.parea.ai/tutorials/getting-started-rag.md): Tutorial on improving a Langchain RAG application using Parea's Evals, Tracing, and Playground.
- [A/B Testing of LLM Apps](https://docs.parea.ai/tutorials/running-ab-tests/llm-generated-emails.md): Leverage user feedback to run A/B tests of prompts, models & other approaches
- [Monitoring Quickstart](https://docs.parea.ai/welcome/getting-started.md): Monitor your LLM requests and application functions.
- [Evaluation Quickstart](https://docs.parea.ai/welcome/getting-started-evaluation.md): Evaluate your LLM app.
- [What is Parea AI?](https://docs.parea.ai/welcome/what_is_parea_ai.md)

## OpenAPI Specs

- [openapi](https://docs.parea.ai/openapi.json)

## Optional

- [Github](https://github.com/parea-ai/parea-sdk)
- [Discord](https://discord.gg/p7CUwSKJD6)
- [Chat with Founders](https://calendly.com/parea-ai/chat)