Re: How could you not notice?
Actually, you'd be surprised. If the bees aren't interested in your garden, they will exit their hive and make a bee-line (to coin a phrase) for their preferred source of pollen and nectar. You might not notice them at all if their doorway is above eye-level in a corner of the house that you rarely look at. Up under the eves is common ... One pencil sized hole on your soffit vent screen is all it takes.
 Many garden flowers have been bred to be pretty to humans, not bees.