I don't see why it would be difficult to go back in your late 20s. In my case I think if I had done that I would have had more maturity and self discipline.

Of course if by that time you are married and have kids and a house, etc.... yeah it would be exceedingly difficult to be a full time student for four years.

EDIT: changed "have a wife" to "are married"