Hacker Noon
WiCi Infer Runs 800B-Parameter AI Models on a Laptop by Streaming Them From the SSD | HackerNoon
WiCi Infer is a free desktop app that runs 800B-parameter AI models on Macs and PCs by streaming weights from the SSD. Here is how it works and how fast it is.