renswickd/semantic-prompt-cache

This app leverages Semantic Caching to minimize inference latency and reduce API costs by reusing semantically similar prompt responses.

语言：Python · GitHub 星标：9 · Fork：0

话题：mistral-api · optimization · rag · semantic-caching · ttl-cache

在 GitHub 上查看原仓库