Do you let random people in the street annoy or bother you?

Because me and my friend were walking behind a gang of lads and she wanted to walk slowly so we didn't cross them and I was like 'who cares just ignore them if they say anything' so we walked past them and they started saying stuff like 'can i have your number' and jeering so we just ignored them but after my friend was like 'look what you've done now' and she was really angry about them. I mean why do people take so much notice of people who they will never see again and let it get to them?