Researchers also found a link between how often children were punished and their IQ.
They warned that children who were regularly smacked could suffer symptoms more closely associated with post-traumatic stress disorder, including becoming more fearful and easily startled.
Both have been linked to a lower IQ.
However, parents could be also more inclined to hit less intelligent children, they admit.
The findings are likely to reopen the debate over whether it is right for parents to be allowed to smack their children.
Last year a cross-party group of MPs failed in their attempts to introduce a ban, but campaigners insist that it is cruel and an ineffective way to admonish children for bad behaviour.
The study looked at more than 1,500 children split into two groups, the first aged between two and four and the second aged between five and nine.
Researchers recorded how often the children were smacked by their parents over four years and then tested their intelligence.
They found that the IQ of those in the younger age groups was an average of five points lower if they had been spanked than if they had not.
Similarly for the older children, the study found that their average IQ was 2.8 points lower if they were spanked.
Prof Murray Straus, from the University of New Hampshire, who led the study, said: "How often parents spanked made a difference. The more spanking the slower the development of a child's mental ability.
"Even small amounts of spanking made a difference."
The researchers also claim that average IQs are lower in countries where there are high levels of corporal punishment than in those where it has been banned.
The team, from the University of New Hampshire, are due to present their findings at the International Conference on Violence, Abuse and Trauma in San Diego today.