You have got to be kidding me ... do they care? Of course they care: they are WOMEN!!!
Any rejection is the end of the world to them, however thick their skin.
But that doesn't mean you should walk in, though, does it?
Apropos I once waited nearly an hour outside the door of a girl I wanted to see ... and the girl next door got more and more mad, as one punter after another knocked on her door, turned, and walked away. She'd already asked me to move away so I wouldn't scare anyone off ... by the fifth guy she was slamming the door quite hard ...
So far the most diplomatic comment I can think of is to say "sorry I was expecting someone else"
And as fjackass says, if you do it smartly, she will try to persuade you to come in anyway, may offer a lower price or some service incentive, and in those cases that I've accepted I've always had a good time. Like she has something to prove.
But don't fake it.
If you find your first reaction is a heaving stomach, just try not to let your horror show in your eyes as you recoil and back up the corridor ... won't go down well