Vals-Smith
Paid
AI Evaluation
Developer Tools
Vals-Smith is an AI-powered platform that transforms any public or private GitHub repository into a custom benchmark suite for evaluating LLMs and coding agents. By analyzing a repository's pull requests and test surface, it creates reproducible, task-based assessments that measure real-world performance on your specific codebase.
Automatic generation of custom benchmarks from public or private GitHub repositories
Evaluation of models and coding agents against real-world engineering tasks
Security-focused architecture ensuring code data remains private and isolated