This paper reports progress in developing a thesaurus for indexing images from diverse domains using non-specialist terminology. The foundation for the vocabulary is a series of research inquiries into user description of images in relation to different tasks. It discusses problems with current thesauri for indexing visual materials and the evolution of the vocabulary from a class project to a tool for annotation for the purpose of performing ground-truth judgments in a benchmarking system. Methods for adding terminology and structuring the thesaurus are discussed, as are considerations that relate to term specificity, and the size and structure of the thesaurus. Unique aspects of visual indexing include problems of scale in relation to meaning and the unbounded nature of visual ontology, requiring a flexible approach to developing a visual indexing vocabulary (VIV).