The effect is only felt by children who have genes that make them vulnerable to alcohol, scientists found.
For these children, even being exposed to relatively small amounts of alcohol before birth could have a significant impact on intelligence at age eight.
It is already well known that heavy drinking in pregnancy can severely harm an unborn baby. But the picture is less clear with regard to light or moderate alcohol consumption. Some guidelines urge mothers-to-be to avoid alcohol completely, while others suggest a moderate intake is safe.
The new findings, published online in the journal Public Library of Science ONE, suggest that genes have a strong bearing on the effect of alcohol in the womb. The research forms part of the Avon Longitudinal Study of Parents and Children (Alspac), which enrolled more than 14,000 pregnant women in 1991 and 1992.
Scientists studied four genetic variants in more than 4,000 children that influence the body's ability to metabolise, or break down, alcohol. They also questioned mothers about their drinking habits while they were pregnant.
Test results showed a lowering of IQ for genetically susceptible children whose mothers consumed between one and six units of alcohol per week. A unit of alcohol is roughly equivalent to one small glass of wine, a pint of beer, or a single measure of spirits.
At eight-years-old, IQ was reduced by almost two points on average for each of the genetic variants a child possessed - but only if his or her mother drank while pregnant. Children whose mothers avoided alcohol completely were not affected, even if they had alcohol-sensitising genes.
Lead researcher Dr Ron Gray, from Oxford University, said: "This is a complex study but the message is simple: even moderate amounts of alcohol during pregnancy can have an effect on future child intelligence. So women have a good reason to choose to avoid alcohol when pregnant."
Colleague Dr Sarah Lewis, from the University of Bristol, said the research showed that levels of alcohol that are normally considered harmless can have an impact on childhood IQ. She added: "This is evidence that even at these moderate levels, alcohol is influencing foetal brain development."