When I first read about the release of Alpaca 7B (https://crfm.stanford.edu/2023/03/13/alpaca.html), Stanford’s self hosted chatgpt3 like large language model I started wondering what kind of cool things could be built with an offline version. Openai’s whisper is an amazing example of what can be accomplished on your own hardware, but there hasn’t been a great offline option. This reddit user has taken the initiative to create and release a dockerized version of Alpace that does just that! This looks really really good.
https://www.reddit.com/r/selfhosted/comments/11yvmqs/ive_been_working_on_serge_a_selfhosted/