Average buggy speed: 8-12 km/h. Meaning: everything important must be within 10 miles (16 km). Church, school, market, family, blacksmith, doctor.
Consequences
Communities are small (30-40 families). Everyone knows everyone. Impossible to be anonymous. Impossible to 'drive away from problems' — you won't get far on a horse.
That's the design. Buggy isn't a limitation. Buggy is community architecture. If Amish drove cars, the community would collapse within a generation.