Let's start with the fact that when you "probably" have a boyfriend. If you have a partner, you either have one or not, there's no "probably" option. He said, "You're a bastard," so he didn't go out with you for real... ...and then you're gonna have to make a comment. I advise you to either clarify the matter (maybe someone on his phone wanted to make a joke to him, etc.) or let it go.