What really bothers me about this is that this guy knows you, even if indirectly. This, in my mind, crosses the line from disturbing to dangerous. I'd give serious thought to arming myself if I were you.

Anyway, glad to see you got the phone number and reported it to the police. Stay safe.