> ## Documentation Index
> Fetch the complete documentation index at: https://docs.parea.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# LangChain
> Instrumenting your Langchain applications with Parea AI
Parea AI works seamlessly with LangChain. LangChain provides a callbacks system to hook into the various stages of your LLM application.
With Parea's `PareaAILangchainTracer` you can automatically subscribe to all exposed LangChain events.
## Quickstart
Add the `PareaAILangchainTracer` as a callback when running your Langchain model/chain/agent:
```bash python theme={null}
pip install parea-ai langchain
```
```bash typescript theme={null}
npm install parea-ai @langchain/core
```
First initialize `Parea` and the `PareaAILangchainTracer`. Follow [these](/welcome/setup) steps to get an API key.
```python Python theme={null}
from parea import Parea
from parea.utils.trace_integrations.langchain import PareaAILangchainTracer
p = Parea(api_key="PAREA_API_KEY") # Replace with your Parea API key
handler = PareaAILangchainTracer()
```
```typescript Typescript theme={null}
import {Parea, PareaAILangchainTracer} from "parea-ai";
new Parea("PAREA_API_KEY"); // Replace with your Parea API key
const parea_handler = new PareaAILangchainTracer(); // This is a tracer that sends the data to Parea
```
Then initialize the tracer and add it as a callback to your LangChain application.
```python Python theme={null}
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
chat_model = ChatOpenAI()
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful AI."),
("human", "{input}")
])
chain = prompt | chat_model
chain.invoke(
{"input": "What is the meaning of life?"},
config={"callbacks": [handler]}
)
```
```typescript Typescript theme={null}
import {ChatOpenAI} from "@langchain/openai";
import {PromptTemplate} from "@langchain/core/prompts";
import {RunnableSequence} from "@langchain/core/runnables";
import {StreamingTextResponse} from "ai";
import {Parea, PareaAILangchainTracer} from "parea-ai";
new Parea("PAREA_API_KEY"); // Replace with your Parea API key
const parea_handler = new PareaAILangchainTracer(); // This is a tracer that sends the data to Parea
const condenseQuestionTemplate = `{chat_history} \n\n Follow Up Question: {question} \n Standalone question:`;
const condenseQuestionPrompt = PromptTemplate.fromTemplate(
condenseQuestionTemplate,
);
export async function main(req: Request) {
const {latestMessage, previousMessages} = await req.json();
const chatModel = new ChatOpenAI({
temperature: 0.5,
openAIApiKey: "OPENAI_API_KEY", // Replace with your OpenAI API key
modelName: "gpt-4",
streaming: true,
});
const chatHistoryQuestionChain = RunnableSequence.from([
condenseQuestionPrompt, chatModel,
]).withConfig({runName: "chatHistoryQuestionChain", callbacks: [parea_handler]}); // Add the handler callback to the sequence
return new StreamingTextResponse(await chatHistoryQuestionChain.stream({
chat_history: previousMessages, question: latestMessage,
}));
}
```
### Where to set callbacks
There are two ways callbacks can be passed - as constructor callbacks and as request callbacks.
* Constructor callbacks are defined in the constructor of an object and apply to all calls made on that specific object. The scope is limited to the object itself.
* Request callbacks are defined in the run() or apply() methods used for sending a request. They are applied to a specific request and all sub-requests it contains.
For example, passing a handler to the chain.run() method will be used for that particular request and any subsequent sub-requests triggered. Additional details [here](https://python.langchain.com/docs/modules/callbacks/).
Below are some examples of how to set callbacks in different scenarios.
```bash db_chain theme={null}
from langchain.llms import OpenAI
from langchain.utilities import SQLDatabase
from langchain_experimental.sql import SQLDatabaseChain
db = SQLDatabase.from_uri("sqlite:///Chinook.db")
llm = OpenAI(temperature=0, verbose=True)
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)
db_chain.run(
"How many employees are there?",
callbacks=[PareaAILangchainTracer()]
)
```
```bash model theme={null}
model = AnthropicFunctions(model="claude-2")
model.predict_messages(
[HumanMessage(content="whats the weather in boston?")],
functions=functions,
callbacks=[PareaAILangchainTracer()]
)
```
```bash lcel theme={null}
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
template = """You are a helpful assistant who generates comma separated lists.
A user will pass in a category, and you should generate 5 objects in that category in a comma separated list.
ONLY return a comma separated list, and nothing more."""
human_template = "{text}"
chat_prompt = ChatPromptTemplate.from_messages([
("system", template),
("human", human_template),
])
chain = chat_prompt | ChatOpenAI() | StrOutputParser()
chain.invoke(
{"text": "colors"},
config={"callbacks": [PareaAILangchainTracer()]
}
```
## Visualizing your traces
Every time you run a LangChain component with tracing, the call hierarchy for the run is saved and can be visualized
in the app on the Logs tab. You can drill down into the components inputs and outputs, parameters, response time, token usage, and other
important metadata.
## Evaluating a Langchain RAG application
Below is an example of how to use the `PareaAILangchainTracer` and Langchain to evaluate a RAG chain. The code can be found in the cookbook [here](https://github.com/parea-ai/parea-sdk-py/blob/d4d833d79fad4f9c367d38d9a9368153c9471459/cookbook/langchain/trace_langchain_RAG_evals.py).
```python Python theme={null}
import os
from datetime import datetime
from operator import itemgetter
from dotenv import load_dotenv
from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import RecursiveUrlLoader
from langchain.document_transformers import Html2TextTransformer
from langchain.embeddings import OpenAIEmbeddings
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
from langchain.text_splitter import TokenTextSplitter
from langchain.vectorstores import Chroma
from parea import Parea
from parea.utils.trace_integrations.langchain import PareaAILangchainTracer
from parea.evals.general import answer_matches_target_llm_grader_factory
from parea.evals.rag import context_query_relevancy_factory, percent_target_supported_by_context_factory
from parea.evals import EvalFuncTuple, run_evals_in_thread_and_log
from parea.schemas import Log
load_dotenv()
# Need to instantiate Parea for tracing and evals
p = Parea(api_key=os.getenv("PAREA_API_KEY"))
CONTEXT = None
def format_docs(docs) -> str:
global CONTEXT # Declare CONTEXT as a global variable
context = "\n\n".join(doc.page_content for doc in docs)
CONTEXT = context
return context
raw_documents = RecursiveUrlLoader("https://en.wikipedia.org/wiki/New_York_City").load()
transformed = Html2TextTransformer().transform_documents(raw_documents)
documents = TokenTextSplitter(model_name="gpt-3.5-turbo", chunk_size=2000, chunk_overlap=200).split_documents(transformed)
retriever = Chroma.from_documents(documents, OpenAIEmbeddings()).as_retriever()
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful documentation Q&A assistant, trained to answer questions from the provided context."
"\nThe current time is {time}.\n\nRelevant documents will be retrieved in the following messages.",
),
("system", "{context}"),
("human", "{question}"),
]
).partial(time=str(datetime.now()))
model = ChatOpenAI(model="gpt-3.5-turbo-16k", temperature=0)
response_generator = prompt | model | StrOutputParser()
chain = (
# The runnable map here routes the original inputs to a context and a question dictionary to pass to the response generator
{"context": itemgetter("question") | retriever | format_docs, "question": itemgetter("question")}
| response_generator
)
# Get started with Parea's auto evaluation metrics for RAG
EVALS = [
EvalFuncTuple(name="matches_target", func=answer_matches_target_llm_grader_factory(question_field="question")),
EvalFuncTuple(name="relevancy", func=context_query_relevancy_factory(question_field="question", context_fields=["context"])),
EvalFuncTuple(name="supported_by_context", func=percent_target_supported_by_context_factory(question_field="question", context_fields=["context"])),
]
def main():
handler = PareaAILangchainTracer()
question = "What is the population of New York City as of 2020?"
output = chain.invoke({"question": question}, config={"callbacks": [handler]})
# get parent trace id from the tracer
parent_trace_id = handler.get_parent_trace_id()
# build log component needed for evaluation metric functions
log = Log(inputs={"question": question, "context": CONTEXT}, output=output, target="8,804,190")
# parea provided helper function to run evaluation metrics in a thread to avoid blocking
run_evals_in_thread_and_log(trace_id=str(parent_trace_id), log=log, eval_funcs=EVALS, verbose=True)
# ###Eval Results###
# NamedEvaluationScore(name='matches_target', score=1.0)
# NamedEvaluationScore(name='relevancy', score=0.0053)
# NamedEvaluationScore(name='supported_by_context', score=1.0)
# View trace at: https://app.parea.ai/logs/detailed/{parent_trace_id}
```
## Tracing LangChain code together with other code
If you want to observe or test LangChain code and other code together, the `trace` decorator plays well with that and will associate any traces, LLM calls, and LangChain logs.
You can see a schematic example below:
```python langchain_inside_trace.py theme={null}
oai_client = OpenAI()
p = Parea(api_key=os.getenv("PAREA_API_KEY"))
handler = PareaAILangchainTracer()
p.wrap_openai_client(oai_client)
chain = ...
@trace
def main():
programming_language = (
oai_client.chat.completions.create(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Suggest one programming languages"}]).choices[0].message.content
)
return chain.invoke(
{"input": f"Write a Hello World program in {programming_language}."},
config={"callbacks": [handler]},
)
```
The full working example can be found below:
```python langchain_inside_trace.py theme={null}
import os
from dotenv import load_dotenv
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from openai import OpenAI
from parea import Parea, trace
from parea.utils.trace_integrations.langchain import PareaAILangchainTracer
load_dotenv()
oai_client = OpenAI()
p = Parea(api_key=os.getenv("PAREA_API_KEY"))
handler = PareaAILangchainTracer()
p.wrap_openai_client(oai_client)
llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"))
prompt = ChatPromptTemplate.from_messages([("user", "{input}")])
chain = prompt | llm | StrOutputParser()
@trace
def main():
programming_language = (
oai_client.chat.completions.create(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Suggest one programming languages"}]).choices[0].message.content
)
return chain.invoke(
{"input": f"Write a Hello World program in {programming_language}."},
config={"callbacks": [handler]},
)
if __name__ == "__main__":
print(main())
```