Or just.. download the books, and slap a search engine on top of them. It'll be like 10x smaller than the model. "Survival LLM" is absolutely a non-relevant use case.
When you’re living in an igloo trying to stay warm in the hollowed out belly of a cow having a few books around the place is probably more useful than some files on a laptop that ran out of battery just before it got morbidly damp. Plus you can burn them if you really need to.
On the other hand, the laptop struggling to power a good-enough LLM will probably be a great heat source for you in that igloo... for about 45 minutes.
As I said elsewhere, you can always burn the books! And eating a laptop is considerably crunchier and more fraught than chewing through some good old fashioned leather facings, horse glue and nice heavy 120gsm wove.
Yes I slightly mangled that one. But I’m sure there is a survival technique that involves crawling into the body cavity of a hollowed out cow. You probably wouldn’t want to do that in an igloo though. I’m not even sure you could get a cow into an igloo. If standard issue humans are this flawed it’s hardly surprising that top flight LLMs get things so wrong sometimes.
Recoll it's far smaller and it runs on a potato. It's a desktop search engine which can parse tons of files. Under GNU/Linux just install all the recommended dependencies for APT/RPM. Or just use Flatpak and to hell with transitive dependencies.
