I see it differently. I sit alone all day, working on my projects by myself. Come dinner time, it is a welcome change to get to chat a little with my coworkers.

Ultimately, I'd say it depends a lot on who your coworkers are. If you enjoy your coworkers, you will enjoy lunchtime with them and it won't be a chore. On the other hand, if they annoy you the second they open their mouth...