I have different machines running different linux distros, so I'm familiar with all the popular package managers (apt-get, yum, pacman), and I must say, Homebrew is really my favorite.
If homebrew is your favorite then you are probably a person who also loves to run Gentoo. For me, I moved away from gentoo in ~2005 because it just wasn't practical to be compiling everything from source all the time.
Binary packages are a wonderful thing and a package manager that doesn't support them out of the box isn't a usable solution for the things I do.
I tried Homebrew on OS X, but it didn't meet my needs. I'm not a fan of having to compile a new version of GCC from source every time I want to install a new library.
Homebrew also doesn't manage OS X itself, which is something I really prefer about apt-get.
The benefit of apt-get source is being able to read the correct version of the source code of any part of the operating system to debug a problem, not necessarily building from source.
I have different machines running different linux distros, so I'm familiar with all the popular package managers (apt-get, yum, pacman), and I must say, Homebrew is really my favorite.