PAREA_API_KEY
as an environment variable, or in a .env file
trace
trace
decorator is used to trace a function and apply evaluation functions to its output.
It automatically attaches the current trace to the parent trace, if one exists, or sets it as the current trace.
This creates a nested trace structure, which can be viewed in the logs.
funcName
: This will be used as the name of the trace and is often the name of the function.func
- The function being traced.options
: An optional object that may contain the following properties:
metadata
: Any extra data that needs to be attached with the trace, which might help with debugging.endUserIdentifier
: The identifier for the end user that is using your application.tags
: Any tags that need to be attached with the trace.target
: An optional ground truth/expected output for the inputs and can be used by evaluation functions.evalFuncNames
: A list of names of evaluation functions, created in the Datasets tab, to evaluate on the output of the traced function. They will be applied non-blocking and asynchronously in the backend.accessOutputOfFunc
: An optional function that takes the output of the traced function and returns the value which should be used as output
of the function for evaluation functions.applyEvalFrac
: An optional parameter to specify the fraction of the time the evaluation functions should be applied. For example, if applyEvalFrac
is 0.5, the evaluation functions will be applied 50% of the time.deploymentId
: An optional parameter to specify the deployment ID of a prompt.target
: An optional parameter to specify the target of the trace.Experiment
Experiment
class is used to define an experiment of your LLM application. It is initialized the data to run the
experiment on (data
), and the entry point/function (func
). You can read more about running experiments
here.
run
experiment_stats
attribute. You can optionally specify the
name
of the experiment as an argument.