I've never understood that. I grew up in seemingly safe neighborhoods in suburban New Jersey and semi-rural Maryland, but leaving our door unlocked (even while we're at home) was unthinkable.
In our exceedingly rural area, doors are left unlocked very often. Not everyone, of course, and not all the time (most of the people I know do lock their doors at night or when they're on vacation, but otherwise leave them open). You're not crazy for doing it.
If someone wants to get in to steal something, they're going to get it done. That could mean a broken window, a broken door, or it could mean nothing broken at all. There's no-one within listening distance to hear a window break, so they're not going to get caught that way. But, your neighbors know your cars, and the sense of community commonly leads to a quick "hey there's a brown Buick outside your house, is that you guys?" call/text. There aren't very many "drive-by" robberies; houses are so far apart and all it takes is one person who sees a suspiciously slow driving car, or a car visiting multiple driveways, to call the police. Oh, and if you thought no one was home and you're wrong, there's a great chance they're armed, and they saw the car you used to get there. And the police are never busy, so they'll be there in minutes.
That seems a pretty common pattern for the semi-rural New England area I live in as well. I lock the house at night or if I'm headed out for an extended period but I don't really worry about it. If the house is all opened up on a summer day I'm not going to shut everything up if I'm going out for a bit and certainly not if I'm around my property somewhere.
During the time that you were home, but it was not immediately obvious from the outside - how many times have you had someone walk up to your door, and try the doorknob?
If the answer is never, then having your doors locked hasn't meaningfully increased your security (Only the perception thereof.)