I have been testing the time to build a large project vs several small projects that amount to exactly the same code base. Only top-level constructs are indexed.
jvm memory: 1.2GB (is needed, with only 1G the times are 1.5x longer due to GC)
number of files: 1662
one big project: 9 minutes
~50 smaller projects: 6 minutes
I think the difference is significant and it asks for a question: is there any difference between the resulting index databases? If not, why does one of them go so much faster? With smaller projects, there is also less memory used.
I just realized that the small projects aren't referring to each other, could that make any difference? I'll test it, but tomorrow.