> Mac or PC with 16–31 GB | IBM Granite 4.2 8B | 14 of 22 right | 6 GB
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
Or just.. download the books, and slap a search engine on top of them. It'll be like 10x smaller than the model. "Survival LLM" is absolutely a non-relevant use case.
When you’re living in an igloo trying to stay warm in the hollowed out belly of a cow having a few books around the place is probably more useful than some files on a laptop that ran out of battery just before it got morbidly damp. Plus you can burn them if you really need to.
On the other hand, the laptop struggling to power a good-enough LLM will probably be a great heat source for you in that igloo... for about 45 minutes.
As I said elsewhere, you can always burn the books! And eating a laptop is considerably crunchier and more fraught than chewing through some good old fashioned leather facings, horse glue and nice heavy 120gsm wove.
Yes I slightly mangled that one. But I’m sure there is a survival technique that involves crawling into the body cavity of a hollowed out cow. You probably wouldn’t want to do that in an igloo though. I’m not even sure you could get a cow into an igloo. If standard issue humans are this flawed it’s hardly surprising that top flight LLMs get things so wrong sometimes.
Recoll it's far smaller and it runs on a potato. It's a desktop search engine which can parse tons of files. Under GNU/Linux just install all the recommended dependencies for APT/RPM. Or just use Flatpak and to hell with transitive dependencies.
The advantage of a turn-by-turn flow (honestly the chat interface was genius) is that with a little human intervention you can get much better results out of even silly models. The model of choice is strange, though. Not a particularly good one.
Either it depends a lot on the model, or I’m bad at chat, or both, because in my experience once a chat starts going off the rails it’s pretty hard to correct the initial misconception. I find it works better to backtrack to just before the misconception and edit my previous response to head it off at the pass.
Definitely agreed that it's better to not have some text in the context than to have some that needs to be opposed. I suppose I meant steering prompts are a non-obvious innovation to LLM interaction.
People apparently like Granite for RAG specifically. I'm not aware of any evaluations with numbers though, so I'm not sure if it's a real difference or just vibes.
We had a set called something like “How to do anything in an emergency” which had instructions for this and more. Alongside lots of creative uses of potassium permanganate.
I’m pretty sure it also told you how to butcher a sheep.
I used to be obsessed with it as a kid. The spectre of nuclear war and a three volume handbook on how to survive the impending apocalypse with nothing more than a plastic bucket, a packet of potassium permanganate, some charcoal and string. The late 80s and early 90s were a wild time to be a kid.
Have you re-read it recently? Some of those books were fun to read but not really something to use as a field manual. Then again some of them were great! I had a book that started with explaining how a battery and a light globe worked, went through how to make a DC motor out of copper wire and a cork, and ended with a crystal radio and I think a radio homing beacon (the latter might have been in another book.)
I have not, sadly. But I’m pretty sure I also had the book you mention. My brother and I made our own motor and many other things that would now be considered entirely unsuitable for 10 year olds to be doing. Copper sulfate crystals seem like a great idea until you have to explain why the playroom carpet has melted.
I have a bunch of military field manuals and survival guides, but they all implicitly presume the existence and contactability of civilization sooner or later. Realistically, some pocket computer(s) and solar charger(s) will store far more information, be far more portable, and give you more carrying capacity for things like medical supplies, rope, tools, and other hard-to-substitute items.
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
Honestly, just buy some books.