What kind of hardware do you need to run with comparable responsiveness to chatgpt?
Generally you need between $8-10,000 worth of equipment to get relative responsiveness from a self-hosted LLM.
What kind of hardware do you need to run with comparable responsiveness to chatgpt?
Generally you need between $8-10,000 worth of equipment to get relative responsiveness from a self-hosted LLM.
This is the point everyone downvoting me seems to be missing. OP wanted something comparable to the responsiveness of chat.chatgpt.com… Which is simply not possible without insane hardware. Like sure, if you don’t care about token generation you can install an LLM on incredibly underpowered hardware and it technically works, but that’s not at all what OP was asking for. They wanted a comparable experience. Which requires a lot of money.