The first time you server a page with a SSR-framework, it will be slow, and subsequently it will be fast when it is served from cache.
What I have seen is that, depending on the number of pages and frequency that each page is accessed, it may be necessary to pre-cache some or all of the server-side rendered pages to get good performance.
The use case I'm working with right now is eCommerce. Landing pages need to be served as quickly as possible to score highly on various page speed SEO metrics. I have seen a number of server-side implementations that need to have most of the common landing pages cached to score better than client-side rendered pages.
Many of these page renders require calls out to third-party APIs, which is responsible for some of the slowness. There are definitely optimizations that can be performed (e.g. caching the results of the backend APIs). But it's easier for less experienced developers to just cache the whole page with a reasonable TTL.
Also agreed that you won't be able to cache much (or as much) if the user is logged in. Logged in pages matter less for SEO, so we haven't focused too much on it.
What I have seen is that, depending on the number of pages and frequency that each page is accessed, it may be necessary to pre-cache some or all of the server-side rendered pages to get good performance.