Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

Abstract: Pronunciation correction is crucial for Text-to-Speech (TTS) systems in production. Traditional methods, which rely on phoneme sequence manipulation, are often cumbersome and error-prone. To ...