How do I handle this situation?

Yes, just tell him! Don't bring up the fact that the kids aren't his "real" kids though. Just say, "I love how important your kids are to you and I like spending time with them, but I'd love to have a romantic date with just you and me, once in awhile. What do you think?" This is probably a conversation best had in person, but if you're not seeing each other too frequently, maybe you could have it over video chat.