What @pstuart said.
Plus most burglars in residential neighborhoods get in through windows. The best deadbolt in the world does nothing if the door is next to an unbarred window.
Most US homes have hackable automatic garage door openers already, and nobody thinks twice about it unless they know they personally are likely to be the individual target of stalkers or home invaders.
That said…probably the biggest general concern is techno punks in upper middle class neighborhoods. Bored teens with enough time and money to buy sniffer equipment and drive around looking for targets of opportunity. You want your network to be secure enough that you don’t look like an easy mark to them. But that’s pretty easy.
edited to add
@geko’s point about online accounts is also important, since one crack might expose thousands of accounts. But what can they do with them?
If you’re worried about physical barrier breach, they have to be at your house. So again, are you likely to be an individual target?
If you’re worried about remote techno punk attacks, there’s not much they can really do. Turn the thermostat up or down, maybe. Turn the lights off. Annoying, but how dangerous depends on your individual situation.
I’m quadriparetic. I don’t put anything on remote access that might cause a fire, Ever. (Not because of fears of malicious attack, just system error.) But that’s me.
The one piece of general advice I would have is never put a gas fueled appliance on a network that isn’t monitored by humans 24/7. That’s the one that could kill you if something goes wrong.