The Department for Education imposed the notice in November 2012 after Ofsted found vulnerable children in the area were being "left at potential risk of harm".
Walsall Council said the removal of the notice followed a further Ofsted inspection in July last year.
Council leader Mike Bird thanked the department for "its hard work".
"There have been difficult times in the past but we can put those behind us now and focus on the future," he said.
The original report criticised social worker assessments and performance management most severely.
It stated performance was "too variable" and there were particular concerns over risk assessments, with inspectors witnessing high-risk situations, incorrectly identified as low risk.
It went on to say that planning did not routinely identify risks, which meant interventions were often poorly focussed.
Pauline Pilkington, former head of children's services, went on to resign in light of Ofsted's findings.
An improvement notice on Walsall Council's children's services department has been lifted.