Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.
Abstract: Unit testing is fundamental for software reliability, yet manual test construction is inefficient and often results in limited coverage. Existing automated tools struggle with complex ...