How this open source LLM chatbot runner hit the gas on x86, Arm CPUs
Way to whip that LLaMA's ass
Applications
3 Apr 2024 | 3
Geez, give up!
x86 is a complete power hog - even Atom. This is a fundamental limitation of the x86 architecture. You can't make a gazelle by shaving an elephant's legs.
Power consumption is pretty much proportional to the number of transistors you have to flip to get the task done. All that pipelining etc that is required to get x86 speeds up requires a lot more transistors getting flipped (compared to a risc).
Right now even ARM-based systems are at the point where power consumption and thus battery life is the major constraint to adding more CPU power. x86 would just make that worse.
Sure, incremental process improvements will improve x86, but a rising tide floats all chips and ARM will score from that too. Thus ARM will always be some steps ahead in this game.
Biting the hand that feeds IT
