On Quora, I once answered a question about why Australian and NZ English sound more like “British English” (really, the questioner meant the accents of southeast England) than Canadian English does. At the time I answered, a lot of the other answers were along the lines of “Canada is sooooo diverse and Australia isn’t so that’s why”, even though in reality the populations of Canada and Australia are similarly diverse. The true answer is primarily that North American English diverged from SE England English earlier (in the 16th century) than the Southern Hemisphere dialects did (late 18th and early 19th centuries), with the “founder effect” helping to explain why later waves of immigration haven’t had the massive impact you might expect.
Basically, once the initial accent of a place is settled (within one generation), subsequent immigrants’ children adopt that accent for their own, instead of hybridising it with their parents’ accent. The accent will still evolve, undergoing natural change (which can be influenced by proximate languages and dialects, including immigrants’ languages), but no faster than any other accent or dialect would in a similar situation. Thus, there is no particular reason why English as spoken in Toronto, Melbourne or Auckland would change any faster than English as spoken in London.
The “founding population” of Anglophone Canada was, basically, loyalists who fled the thirteen colonies that declared independence and became the USA. (There were some Anglophone communities in Atlantic Canada before that – where accents are still often quite different from Standard Canadian English – but otherwise Anglophones in Canada were limited to a small number of administrators prior to US independence; the British didn’t see the point in settling when there were far more hospitable lands just to the south, although there was of course a large population of French-speakers.) Due to the extensive links between Canada and the US, Canadian English also hasn’t diverged too much from many northern US accents. In contrast, with the much poorer connections between North America and Britain between the 16th and 19th centuries, those two groups of accents diverged considerably before more regular contact was enabled.
Australia and New Zealand’s founding populations, in contrast, were established in the late 18th (AUS) and early 19th (NZ) centuries. The Australian founding population had a considerable Irish component, as well as many Scots, but the “founding accent” that was settled on was nonetheless much closer to the English of southeast England (perhaps because it was considered a more prestigious accent, but no one can really say for sure). New Zealand, particularly its South Island, had many Scots among its “founding population” who influenced the NZ accent. South African English was founded during the same era, with a similar founder population, which is why native English speakers from South Africa still speak with a similar (though obviously not identical) accent to Australians and New Zealanders today.
Despite the great distances involved, communication was actually much better between England and its Southern Hemisphere colonies than it had been centuries earlier with North America. Thus, while all the accents have diverged, each undergoing changes that have not necessarily been shared by others in the subgroup, they haven’t diverged as much as the North American accents have. People were also much more mobile within each territory; thus, Australian English lacks regional accents entirely (although some vocabulary is different), while North American English encompasses many.
Obviously it isn’t the case, strictly speaking, that Southern Hemisphereans “sound British” – rather, their accents are more closely related to the English of southeast and south-central England (which is already a group of accents) than Canadian English is. They’re no more closely related to other regional dialects within England, much less the rest of Britain or Ireland, though.