turn the tables - if their eyes drift to your hairline, and you are sensitive, your eyes should drift to their teeth while they are speaking. do it couple of times and see their reaction. just a thought.
In the corporate world senior managers will do this to me while i am speaking to them - it is a technique used to 'size you up' - do you get flustered or distracted easily if they are looking elsewhere, or do you simply overlook it unfazed. i have trained myself to ignore it, and on occassion, will look at their tie, teeth, collar, ear, etc. to have the same fun.