This is an older thinkpad I am using, 4 cores, 16G of ram. LLVM is now dependency you cannot get rid of. That is painfull, but in the morning, after several hours it is usually done. Rust is LLVM plus I am not sure what, really painfull. And Gentoo also forces many different arches and flags for some reason, use flags are masked, you cannot easily disable them.
That sounds similar to the hardware on my T460 (2 cores, 4 threads, 16G). Would you consider cloning the repo and building it with the same command as my other comment? I'm really intrigued if the issue is the hardware/platform or "just" the Gentoo configuration or something else that doesn't affect the default build process.