Well best thing you can do is to put all that crap out of your mind and not focus on it, because then you're going to start getting all clingy and weird because of your own internal fears, which will cause them to come true because you'll end up sabotaging your relationship yourself.
Best to just relax and stop worrying about it, things will happen as they do.
I am in a long distance relationship right now and I say it is totally worth it. My case is much different though. Met online and not in person yet. We are both 17. I'd say don't worry your heart too much. If he loves you and knows that you're good for him, he'll survive the distance. Love is greater than distance!! And if he can't survive the distance then he really isn't worth trying to get back.. You guy's still text, email, snapchat, skype. Skype is the best besides the texting and stuff. So many ways to keep you guys close despite the distance (: I don't know how well this reply helps but if you need someone to talk to or have any questions, I'm friendly and here!
I hate to say this, but generally the person freaking out, is the person who can not handle it. You keep saying that he can not handle a LD relationship, but there are two people here, and he sounds pretty sure that things will be ok. But you keep saying you don't know if he can handle a LD relationship. This is your freak out, you are the one worried about it, and you need to figure out why, and address your issues.
Two of my good friends are in a long-term relationship. He's about 3-4 hours away, and she attends my college. They made it work so far, and they still seem pretty happy! I think they Skype or call frequently too, and he comes down and visits for breaks and holidays.
In contrast, my ex and I lived in the same area after he graduated (I'm two years younger), and that didn't work out well at all. He wanted little/nothing to do with high school life, which I was still stuck in - and I didn't understand it until I got to college myself, a year after the breakup. :\
So I guess it just depends! I would definitely talk to him about this though. If you make him so happy as you claim, part of me doubts that he'd leave you or hurt you just because you're a drive away.
In my opinion, that isn't far enough away to warrant a break up. However, I'm not your boyfriend.
But I do believe that the internet, phones, ect exist and because of this, there is subsequently no excuse if you truly love the person. There is always a way to stay in contact even if there is a temporary lack of physical contact.