Government researchers tracked the progress of students in England who did not get the benchmark A* to C in the subjects in 2011.
Just over half continued to study the subjects but only 6.5% eventually got the grades in English and 7% in maths.
Schools Minister David Laws said "too many people were allowed to give up".
In 2011, 188,365 students did not achieve A* to C in English and 211,171 in maths.
The statistics, which the government says are "experimental", suggest that of these, almost half did not enter for any English or maths qualifications between the ages of 16 and 18.
A* to C in English and maths are crucial grades for teenagers, required by many employers and by colleges for access to further and higher education.
From this month, young people are required to remain in education or training until the age of 17, rising to 18 in 2016, and they must continue to study these two key subjects until they achieve "good" grades.
"To get the best start to life in modern Britain young people must have good qualifications in English and maths", said Mr Laws.
"Now anyone who fails to get a C in GCSE English or maths must continue studying those subjects."
Mr Laws said the government had invested Â£30m in training and recruiting more teachers for further education colleges, with bursaries and golden hellos "to attract and retain the best quality teachers".
"Our reforms are already bearing fruit with the latest figures showing thousands more 17-year-olds are taking GCSEs in those subjects giving them the vital skills they need to get a good job and get on in life," he said.
Shadow education secretary Tristram Hunt said the 2013 figures show a decline on the previous year, the first for which these figures were produced.
"David Cameron's government is failing to ensure that all young people leave education with a firm grasp of English and maths," said Mr Hunt.
"This just isn't good enough. Employers and universities are telling us that too many young people are leaving education without the numeracy and literacy skills that they need to succeed in further study and in the world of work. Yet far too many young people drop these important subjects at 16.
"Labour will ensure all young people will study English and maths to 18 so that we can keep pace with the accelerating progress being made in other successful education systems around the world. This is key to delivering a world-class education system in this country."
Joy Mercer, director of education policy at the Association of Colleges, warned against using the figures to draw conclusions about current behaviour "as they were gathered before the new policies on continuing to deliver maths and English to age 18 came into force".
"About 50% of students who fail to get the required C grade go to a further education college and are often those furthest from achieving it.
"Colleges are determined to help students develop the maths and English skills needed for employment, as they always have, but we think there is a place for a different post-16 GCSE qualification, recognised by employers, which could act as a passport to better employment."
Most students who fail to get good GCSEs in English and maths at 16, also fail to get them by 18, figures for 2013 suggest.