LangSmith

LangSmith is the observability and testing platform developed alongside LangChain, providing the infrastructure needed to build, test, and monitor production LLM applications. As LangChain became the standard framework for LLM application development, LangSmith emerged as the complementary observability and quality assurance platform for LangChain-based applications.

LangSmith's tracing capability records the complete execution trace of LLM application runs, showing every prompt sent to models, every tool call made, every retrieval executed, and every output generated. This complete trace visibility is essential for understanding and debugging complex LLM application behavior.

The platform's dataset management enables creation of golden test sets that represent the expected behavior of LLM applications. These datasets can be used for automated testing when the application changes, providing a regression testing infrastructure for LLM applications.

LangSmith's evaluation framework allows systematic assessment of LLM outputs against defined criteria, including both automated evaluators and human evaluation workflows. The combination of automated and human evaluation provides comprehensive quality assessment for different aspects of LLM performance.

For production monitoring, LangSmith tracks application performance over time, identifying when quality degrades due to model changes, data drift, or prompt drift. These monitoring capabilities help maintain the quality of production LLM applications without requiring manual review of every output.