I'm in my first relationship too, so I know exactly what you're going through (it doesn't help that I'm a slightly awkward person). I'd just take it slow: trying to force it might just make it way more awkward.
I see how it would be a little uncomfortable. If you enjoy being "together" and really want to give it a try, just give it time. Maybe you guys will grow more comfortable with seeing each other that way soon. If not, you guys could always talk it out and decide to just stay friends.