How does a long distant relationship like this work?

It never works. I have been there. The reason it does not work is because your work life and private life are both completely separate. You don't know who he's talking about where he is, and he is no longer a part of whats going on in your town. I know its sad. But better to end it now than later.