Refined Langsmith Preset

LangSmith integration setting that routes Claude Code traces to LangSmith for LLM observability, evaluation, and debugging.

When to Use This Setting

Apply this setting when you need to:

• Send Claude Code interaction traces to LangSmith for detailed LLM call observability and debugging
• Evaluate Claude Code response quality using LangSmith's evaluation framework and scoring tools
• Build a dataset of Claude Code interactions for fine-tuning prompts and improving workflow efficiency Consider alternatives when:
• You do not have a LangSmith account or your organization uses a different LLM observability platform
• Your security policies prohibit sending interaction traces to third-party observability services

Quick Start

Configuration

Copy
name: refined-langsmith-preset
type: setting
category: telemetry

Example Application

Copy
claude setting:apply refined-langsmith-preset

Example Output

Setting applied. Changes:
- LANGCHAIN_TRACING_V2: true
- LANGCHAIN_API_KEY: (from environment)
- LANGCHAIN_PROJECT: claude-code-traces
- trace_destination: LangSmith API
- observability_level: full trace capture

Core Concepts

LangSmith Integration Overview

Aspect	Details
Trace Capture	Records full interaction traces including prompts, responses, and tool calls
API Authentication	Uses LANGCHAIN_API_KEY for authenticated access to LangSmith API
Project Organization	Routes traces to a configurable LangSmith project for organized observability
Evaluation Support	Captured traces can be scored and evaluated using LangSmith evaluation tools
Dataset Building	Interactions are stored as datasets for prompt engineering and optimization

LangSmith Trace Architecture

+-------------------+     +---------------------+     +------------------+
| Claude Code       |---->| Trace Interceptor   |---->| LangSmith API    |
| interactions      |     | TRACING_V2=true     |     | authenticated    |
| tool calls        |     | serialize traces    |     | trace storage    |
+-------------------+     +---------------------+     +------------------+
                                   |                          |
                                   v                          v
                          +---------------------+    +------------------+
                          | Project Router      |    | LangSmith UI     |
                          | LANGCHAIN_PROJECT   |    | trace viewer     |
                          | -> project name     |    | evaluation tools |
                          +---------------------+    +------------------+

Configuration

Parameter	Type	Default	Description
LANGCHAIN_TRACING_V2	string	"true"	Enable LangChain v2 tracing for Claude Code interactions
LANGCHAIN_API_KEY	string	""	API key for authenticating with the LangSmith service
LANGCHAIN_PROJECT	string	"claude-code-traces"	LangSmith project name for organizing captured traces
LANGCHAIN_ENDPOINT	string	"https://api.smith.langchain.com"	LangSmith API endpoint URL
trace_sample_rate	float	1.0	Fraction of interactions to trace (1.0 = all, 0.1 = 10%)

Best Practices

1. Create a dedicated LangSmith project - Use a descriptive project name like "claude-code-dev" or "claude-code-production" to separate traces by environment. This prevents development noise from polluting production observability data.
2. Sample traces in high-volume environments - If you run Claude Code extensively, set trace_sample_rate to 0.1 or 0.2 to capture a representative subset without overwhelming your LangSmith storage quota.
3. Use traces for prompt optimization - Review captured traces in LangSmith to identify patterns where Claude Code responses could be improved. Build evaluation datasets from these traces to systematically test prompt changes.
4. Secure your API key - Store LANGCHAIN_API_KEY in your system keychain or a secrets manager rather than plain text environment files. The key provides full access to your LangSmith account.
5. Set up evaluation pipelines - Once traces are flowing, create LangSmith evaluators that automatically score response quality, latency, and tool usage patterns. This transforms passive observability into active quality improvement.

Common Issues

1. No traces appear in LangSmith - The LANGCHAIN_API_KEY may be invalid or the LANGCHAIN_TRACING_V2 variable may not be exported. Verify both variables are set and accessible in your Claude Code shell session.
2. Traces show in wrong project - The LANGCHAIN_PROJECT variable may be unset, causing traces to land in the default project. Explicitly set the project name to route traces to your intended destination.
3. High latency after enabling tracing - Trace serialization adds overhead to each interaction. If latency is noticeable, reduce trace_sample_rate to capture fewer traces or ensure your network connection to LangSmith is fast and stable.