Officers have been searching a house at Pen-Y-Bryn, near Gelligaer.
Police said a man was assisting the force with its inquiries.
Gwent Police Det Ch Insp Nicky Brain said: "I'd like to reassure residents that this is an isolated case and we are not looking to speak to anyone further during this investigation."
Police in Caerphilly have launched an investigation into reports of historical sex offences and concealment of a birth.