Should I try and stop doing this?

When I talk to people, usually girls, I have the habit of staring at their lips. I don't want to give them the wrong idea but it helps me understand what they're saying better. I'm not deaf or anything but I can't help it. And because I'm a lesbian it makes it tougher. Should I try and stop doing this because it could freak them out or is it not that big of a problem? Thanks in advance! And I posted it in the lgbt section because ya'll are awesome.