No (unless you assume something like a lower curvature bound).
Consider two very long segments (say, of length 100) in $\mathbb R^3$ that meet orthogonally at their midpoints. Let $M$ be a smoothened boundary of an $\varepsilon$-neighborhood of the union of these segments. It is a Riemannian manifold invariant under a 90 degrees rotation. Let $A$ be a point of $M$ near one of the segments' endpoints and $B$, $C$, $D$ be its consecutive images under this rotation. Due to rotation invariance, one has $d(A,B)=d(B,C)=d(C,D)=d(D,A)$. And all distances between $A,B,C,D$ are greater than 50.
On the other hand, geodesic segments $AC$ and $BD$ go through the central part of the construction, namely an $O(\varepsilon)$ neighborhood of the center. Take $x$ and $y$ in that region and observe that $d(x,y)=O(\varepsilon)$.