Label Studio

Multi-modal data labeling and annotation platform for agent traces, LLM evals, RLHF, computer vision, document AI, NLP, audio transcription, and more. The open source platform for data labeling, AI evaluation, and human-in-the-loop workflows. Connect observability tools to enable human-in-the-loop review Collect human preferences, corrections, rankings Create custom benchmarks and rubrics, side-by-side comparison Evaluate retrieval relevance, grade generated answers against source Detect objects on image, boxes, polygons, circular, and keypoints supported Label and track multiple objects frame-by-frame