biraj21/llm-server-from-scratch

FastAPI server for locally serving Gemma 3 270M & OpenAI Whisper with batched inference and streaming support.

语言：HTML · GitHub 星标：9 · Fork：0

话题：asr · gemma · inference · llm · transformers · whisper

在 GitHub 上查看原仓库