We introduce interactive search queries for online communities (ISQ-problem) as a new research challenge. In online communities, users often try to leverage the combined community expertise to find answers for problems where neither a complete and accurate specification of the question nor the exploration of all possible answers is feasible. Some typical examples are identifying the species of an observed bird, disease diagnosis in a health-care community, finding the name of a nice restaurant where one ate several years ago during a vacation, etc. We formally define the ISQ-problem as a problem of minimizing user effort, while guaranteeing interactive system response time. To deal with uncertain and missing user responses, we develop a probability-based framework and show that it allows us to optimally rank possible answers. The framework is also the basis for an algorithm that decides about asking the right questions or presenting possible answers, such that it can home in on the right answer with minimal user effort. A core component of the algorithm is scalable technique for estimating the required probabilities while guaranteeing interactive response time even for very large problems.