Runs not being associated with experiment after running evaluations

[elliecrawley]

Issue you'd like to raise.

Tracing Method: LangChain
Language: TypeScript
Host: Hosted (https://api.smith.langchain.com)
Other System Info:

• sdk version: 0.1.85
• runtime : python
• langchain_version: 0.2.7
• langchain_core_version: 0.2.16

Issue:

I am trying to run an evaluation. My target function selects a list of data items which it considers most relevant to the user input. My evaluator function uses the provided dataset to judge how relevant the targetFunctionOutput actually is.

    let targetFunctionOutput = await targetFunction('');
    let rawResults = await evaluate(() => {return targetFunctionOutput}, {
      data: dataset,
      summaryEvaluators: [relevant],
      experimentPrefix: `initial-validity-run-for-${dataset}`,
    });

This all works as expected. Both functions run as desired, output the expected data, add the trace to the correct tracing project, and create an experiment in the appropriate dataset. The issue is that the experiment is always empty, it never has any runs associated with it and therefore I am never able to view any of the analytics data or charts for my experiments.

Expected behaviour:

Upon concluding I should be able to see data about my experiment in the experiments area of my Langsmith dashboard.

Suggestion:

No response

[baskaryan]

@elliecrawley could you confirm if you're still seeing the issue with the latest sdk version, 0.3.4?

cc @jacoblee93

[elliecrawley]

Hi @jacoblee93 thanks so much for your quick reply but unfortunately that hasn't fixed it.