Gemma 3 4B QAT int4 quantized from bartowsky should barely fit in a 4GB Raspberry Pi, but without the vision encoder.

However the brand-new Gemma 3n E2B and E4B models might fit with vision.

Yep, the Gemma 3 1B would be 815MB, with enough margin for a longer prompt. Probably more realistic.

Nope, gemma3 and qwen3 exist of many sizes, including very small ones, that 4-bit quantized can run on very small systems. Qwen3-0.6B, 1.7B, ... imagine if you quantize those to 4 bit. But there is the space for the KV cache, if we don't want to limit the runs to very small prompts.