In order to build production ready LLM applications, developers need to understand the state of their systems. You want to track key LLM metrics, such as request inputs and outputs, as well as model specific metadata, model parameters, tokens, cost, etc. But you also want to track your functions which may manipulate data from one LLM and chain it into another. Parea makes it easy to get this deep visibility into any LLM stack.

Prerequisites

  1. First, you’ll need a Parea API key. See Authentication to get started.
  2. For any model you want to use with the SDK, set up your Provider API keys.
  3. Install the Parea SDK.
pip install parea-ai

Logging

Parea automatically logs all LLM requests when using the SDK, or when using OpenAI’s API.

log_table

Usage

  • Python

  • Typescript

  • Curl

Parea supports automatic logging for OpenAI, Langchain, or any model if using Parea’s completion method (schema definition).

Parea Completion Method

The completion method allows you to call any LLM model you have access to on Parea with the same API interface.

You have granular control over what is logged via the parameters on Parea’s completion method.

  • log_omit_inputs: bool = field(default=False) # omit the inputs to the LLM call
  • log_omit_outputs: bool = field(default=False) # omit the outputs from the LLM call
  • log_omit: bool = field(default=False) # do not log anything
parea_completion.py
from parea import Parea
from parea.schemas import LLMInputs, Message, ModelParams, Role, Completion

p = Parea(api_key="PAREA_API_KEY")  # replace with your API key

response = p.completion(
    Completion(llm_configuration=LLMInputs(
        model="gpt-3.5-turbo",  # this can be any model enabled on Parea
        model_params=ModelParams(temp=0.5),
        messages=[Message(
            role=Role.user,
            content="Write a Hello World program in Python using FastAPI.",
        )],
    ))
)
print(response.content)

OpenAI API

If you want to use OpenAI directly, you can still get automatic logging using Parea’s wrap_openai_client helper.

openai.py
from openai import OpenAI
from parea import Parea

client = OpenAI(api_key="OPENAI_API_KEY")

# All you need to do is add these two lines
p = Parea(api_key="PAREA_API_KEY")  # replace with your API key
p.wrap_openai_client(client)

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    temperature=0.5,
    messages=[
        {
            "role": "user",
            "content": "Write a Hello World program in Python using FastAPI.",
        }
    ],
)
print(response.choices[0].message.content)

LangChain Framework

Parea also supports frameworks such as Langchain. You can use PareaAILangchainTracer as a callback to automatically log all requests and responses.

langchain.py
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from parea import Parea
from parea.utils.trace_integrations.langchain import PareaAILangchainTracer

# All you need to do is add these two lines
p = Parea(api_key="PAREA_API_KEY")  # replace with your API key
handler = PareaAILangchainTracer()

llm = ChatOpenAI(openai_api_key="OPENAI_API_KEY")  # replace with your API key
prompt = ChatPromptTemplate.from_messages([("user", "{input}")])
chain = prompt | llm | StrOutputParser()

response = chain.invoke(
    {"input": "Write a Hello World program in Python using FastAPI."},
    config={"callbacks": [handler]}, # <- use the callback handler here
)
print(response)

Tracing

If your LLM application has complex abstractions such as chains, agents, retrieval, tool usage, or external functions that modify or connect prompts, then you will want a trace to associate all your related logs. A Trace captures the entire lifecycle of a request and consists of one or more spans, representing different sub-steps.

trace_section

Usage

  • Python

  • Typescript

The @trace decorator allows you to associate multiple processes into a single parent trace. You only need to add the decorator to the top level function or any non-llm call function that you want to also track.

Parea Completion Method

trace_decorator.py
from parea import Parea, trace
from parea.schemas import LLMInputs, Message, ModelParams, Completion

p = Parea(api_key="PAREA_API_KEY") # replace with your API key

# We generally recommend creating a helper function to make LLM API calls.
def llm(messages: list[dict[str, str]]):
    return p.completion(
        Completion(
            llm_configuration=LLMInputs(
                model="gpt-3.5-turbo",
                model_params=ModelParams(temp=0.5),
                messages=[Message(**m) for m in messages],
            )
        )
    ).content

# (Optional) You can add a trace decorator to each prompt.
# This will give the Span the name of the function.
# Without the decorator the default name for all LLM call logs is `LLM`
@trace
def hello_world(lang: str, framework: str):
    return llm([{"role": "user", "content": f"Write a Hello World program in {lang} using {framework}."}])

@trace
def critique_code(code: str):
    return llm([{"role": "user", "content": f"How can we improve this code: \n {code}"}])

# Our top level function is called chain. By adding the trace decorator here,
# all sub-functions will automatically be logged and associated with this trace.
# Notice, you can also add metadata to the trace, we'll revisit this functionality later.
@trace(metadata={"purpose": "example"}, end_user_identifier="John Doe")
def chain(lang: str, framework: str) -> str:
    return critique_code(hello_world(lang, framework))

OpenAI API

If you want to use OpenAI directly, you can still get automatic logging using Parea’s wrap_openai_client helper.

openai_trace_decorator.py
from openai import OpenAI
from parea import Parea, trace

client = OpenAI(api_key="OPENAI_API_KEY")  # replace with your API key

# All you need to do is add these two lines
p = Parea(api_key="PAREA_API_KEY")  # replace with your API key
p.wrap_openai_client(client)


# We generally recommend creating a helper function to make LLM API calls.
def llm(messages: list[dict[str, str]]):
    response = client.chat.completions.create(model="gpt-3.5-turbo", temperature=0.5, messages=messages)
    return response.choices[0].message.content


# (Optional) You can add a trace decorator to each prompt.
# This will give the Span the name of the function.
# Without the decorator the default name for all LLM call logs is `llm-openai`
@trace
def hello_world(lang: str, framework: str):
    return llm([{"role": "user", "content": f"Write a Hello World program in {lang} using {framework}."}])

@trace
def critique_code(code: str):
    return llm([{"role": "user", "content": f"How can we improve this code: \n {code}"}])

# Our top level function is called chain. By adding the trace decorator here,
# all sub-functions will automatically be logged and associated with this trace.
# Notice, you can also add metadata to the trace, we'll revisit this functionality later.
@trace(metadata={"purpose": "example"}, end_user_identifier="John Doe")
def chain(lang: str, framework: str) -> str:
    return critique_code(hello_world(lang, framework))

Streaming

from parea import Parea
from parea.schemas import LLMInputs, ModelParams, Message, Completion

p = Parea(api_key="PAREA_API_KEY")  # replace with your API key

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
]


def llm_stream(messages):
    stream = p.stream(
        data=Completion(
            llm_configuration=LLMInputs(
                model="gpt-3.5-turbo-0125",  # Any model you have enabled on the Parea Platform
                model_params=ModelParams(temp=0.1),
                messages=[Message(**m) for m in messages],
            )
        )
    )
    for chunk in stream:
        print(chunk)


async def llm_astream(messages):
    stream = p.astream(
        data=Completion(
            llm_configuration=LLMInputs(
                model="claude-1",  # Any model you have enabled on the Parea Platform
                model_params=ModelParams(temp=0.1),
                messages=[Message(**m) for m in messages],
            )
        )
    )
    async for chunk in stream:
        print(chunk)

What’s Next

Now that you know how to create a trace you can enrich it with metadata or learn how to: