In your second sentence the difference starts to express itself. With the "to form", the question makes more sense. How could someone know if they would like going out with someone unless they have already tried it? The infinitive form makes it more of a concept than an experience. In that same fashion, if two people are already dating, only the gerund form will work: Do you like going out with me?