Dean Saunders, 25, died at HMP Chelmsford in Essex in January 2016.
He had been arrested the month before, after his father, Mark, was stabbed while trying to stop his son harming himself during a bout of paranoia.
Mark Saunders has called for the recommendations made by the ombudsman to be seen through.
On Friday, an inquest jury found Mr Saunders was "let down" by the prison and mental health services.
Live: For more on this and the latest Essex stories
Nigel Newcomen, the Prisons and Probation Ombudsman said Mr Saunders was at "high risk of suicide" when he arrived at Chelmsford and identified "a number of weaknesses" in the prison's handling of his case.
Mark Saunders, Dean's father, said the family had questioned his placement in prison from the outset.
"Obviously our concerns were that he needed help and medication, we needed to find out what was going on," said Mr Saunders.
"There was no proper medical structure there [in prison] to help him. We were lied to and mislead all the way through. We were devastated."
The ombudsman said those involved in Dean Saunders' care felt he should have been in a mental health facility rather than prison.
"I am also concerned that there appears to have been some confusion at Chelmsford about the process for transferring mentally ill prisoners to hospital, which meant that an opportunity to transfer Mr Saunders in December [of 2015] was missed.
"Sadly, the criminal justice system did too little to protect this very vulnerable man."
Speaking about the report, Mark Saunders said: "I can't walk away from this now without putting all the changes into place."
Care UK, the healthcare provider at HMP Chelmsford, announced it was to quit its contract at the prison.
The company said it had attempted to resolve issues at Chelmsford Prison but the level of resources the prison planned to make available were "insufficient".
The Ministry of Justice said after the inquest that it accepted there were "significant failings" in Mr Saunders' care.
It added "a number of measures to better support offenders suffering from mental health problems" had already been put in place.
The criminal justice system "did too little to protect" a vulnerable inmate who electrocuted himself in prison, the Prisons Ombudsman has found.