Why are you not sprawling?
My little brother can stuff a shot with a sprawl, and he's tiny. Strength and speed have way less to do with it then timing it properly, that's why Chuck is so hard to takedown.
Just do basic td defense excersise first. If you do 500 sprawls a day pretty soon you won't be weak and slow. It's good training, if not for the actual conditioning than certainly for the discipline.