I see both viewpoints. Drives me nuts when pedestrians walk right behind me when I'm backing out of a parking space. And nuts when drivers block my bikeway — especially when there was a win-win they chose not to take.
What's lacking in both cases is common courtesy. That often includes respectful communication, like waving someone by, or even just signalling your intentions.