I'd rather have the lower security device (rfid) open the front door.
My flat has a single key which opens both front door and the flat. If I lose it, I only have to buy new keys and a new lock *for my flat* - the front door is meant to be a minor hurdle, not a major security barrier.
Two keys is significantly worse than one key because I like to keep my pockets light. (I currently have a key ring with two door fobs for the office, a bike lock, and my flat.)