It was clear it was technologically possible to do mass electronic dragnet surveillance for a long time, but NSA officials have lied thousands of times about the breadth and extent of their collection activities and people that like to trust the government(and authorities in general) gave them the benefit of the doubt. Now it's out of the bag though and the public is apathetic so it's likely to continue.

They probably just couldn't go as deep on as many targets but were able to watch a lot from a relatively high level. Back then there was a lot less noise to filter through, too. Mobile devices weren't ubiquitous... traffic was mostly e-mail. When the FBI was using Carnivore, Jaz drives were still a thing. If you had a 56k modem you the lucky guy on your local bulletin board.

So in retrospect tools like Carnivore sound primitive... at the time it was a pretty sharp scalpel.

Room 641A and MySpace both launched in 2003. Facebook became popular between 2004 and 2008. There were a few others in there; but in general the government programs scale relative to the amount of data made available through the Internet. In 1996 there just wasn't that much.