I have this running at home on a used r630 (CPU only). oobabooga/automatic1111 for LLM/SD backends, vosk + mimic3 for tts/stt. A little bit of custom python to tie it all together. I certainly don’t have latency as low as theirs, but it’s definitely conversational when my sentences are short enough.
I am not joking lol but I do sometimes forget most people don’t live in this space the same way I do. I think people use these names because the programs themselves are forked often and the software names are very unspecific otherwise. I meant to imply that I was using the main branches of these softwares.
https://github.com/oobabooga/text-generation-webui
https://github.com/AUTOMATIC1111/stable-diffusion-webui