Non VBA solution:
Select entire range with your data. Press CTRL+G and select Special...
Choose Blanks and press OK
You would get result as in image below. With selected cells type
= and press ↑ key. Press CTRL+ENTER
Now if you want you can copy result and make PasteSpecial-->Paste Values
Dim rng As Range, ar As Range
On Error Resume Next
Set rng = Range("A2:D20").SpecialCells(xlCellTypeBlanks)
On Error GoTo 0
If Not rng Is Nothing Then
rng.FormulaR1C1 = "=R[-1]C"
For Each ar In rng.Areas
ar.Value = ar.Value