If we're talking about "which bottleneck is the biggest" memory size is traditionally the big one on consoles; there's a longstanding preference for a performance profile of small/fast memory, mid-tier CPU, and top-end dedicated graphics, because, as the saying goes, "graphics sell games," and in most games, the kind and quantity of assets in a scene are limited and the design will allow for them to be carefully streamed in as necessary, so it's more important to allow processing headroom. This also accounts for a difference in design style between PC games and their console equivalents, where PC stuff tends to incorporate deeper simulation aspects with more persistent data being tracked, because there's some extra room for that stuff, while console games are forced to be "lean" with most of their memory dedicated towards the assets while the stats and save data are relatively light.

Like everything else, this has changed as we've gotten closer and closer to photorealism and games that are glorified tech demos constitute less and less of the overall market; the current generation consoles have substantially changed their profile to generalize and be more like PCs - the 360 had only 512MB at a time where gaming PCs were going for 1-2GB, while Xbox One and PS4 are roughly in the same ballpark(5GB and 8GB) as current-spec gaming PCs.