This has happened, right?
The Tories are the party of those areas which supported Brexit, and Labour/Lib Dem/Greens will fight between themselves for those areas which supported Remain. Whilst the Brexit war has been fought, and decided, the sociological, economic and moral issues against which it was fought remain important and divisive. Labour has lost large parts of its working class base and is become a more exclusively metropolitan and younger generation party. It is difficult to see how Starmer, or in fact anyone, could do anything to reverse this.
This is a bad lookout in the short to medium term, because although the two camps are broadly equal in numbers, there are more Brexit constituencies than Remain.
In the longer term, it could spell trouble for the Tory party, as their core voters are older and will shrink as a proportion of the population. Still, the Tories have always been quite good at reinventing themselves and some people become more reactionary as they get older.