VPN will be just as slow as (or slower than) connecting directly since all it does is add a layer of encryption. It’s still going across the internet with the resulting latency. Unless someone had dedicated (non-internet) connections between offices (with low latency).
