Thanks guys.
After your comments I listened to the fronts only and found that, by crikey, there was just as much distortion there so I'll replace them 1st.
And yes, I have the factory Alpine 3CD in-dash unit. It's not 50W rms like I thought, hadn't swapped it out. Must of been dreaming.
Will upgrade the fronts first, then rears. Found some 6" speakers in the workshop, which will be 2nd hand so will put them in the rear, and get some new ones for the fronts.