It was explained to me once that [at least in New Hampshire, USA] common law marriage has something to do with the principle of estoppel. Estoppel is the principal that if you do something repeatedly, you would be expected to do it in the future. In the case of common law, if you present yourself as married (for taxes for example) you can't later claim you are single to get out of paying alimony when split up.

Example: If an insurance company pays for flood damage, and they have another flood damage claim, they can't suddenly say they won't pay.