If I remember well, some studies pointed out that when the poles flipped it happened within a period of a couple of decades. So geologically speaking it's happened in a flash. But those two decades in particular will be unstable, magnetically speaking. My biggest concern would be the temporary loss (or weakening) of the earth's magnetic shield against the solar wind.I dread to think what could result if the poles do flip and only go halfway so somewhere around the equator becomes north and the other side becomes south. Now that would be interesting.
And yes, the almost unanimous consensus among geologist has been derived from geological records worldwide, and they do account for seismic activity and continental drift, among other things.