1) You cannot remove all of the components from IE in XP. IE is imbedded in the operating system inextricably. One company
http://www.litepc.com/ has done a decent job in trying, but you can't get it all. If you want to reclaim some hard disk space, it might be worth a try, but when your computer has intermittant hiccups for no reason, you might ask yourself whose fault it is.
2) IE was a superb product in its day. It was built to be super useful and that's what makes it vulnerable. Microsoft has not kept pace by building a more competent browser so the best that can be done is to patch it or use a less vulnerable browser or an add-on browser that protects you from popups, phishing, and spoofing.
3) The ugly secret behind popups is not the browser but the user. Where you go on the net without taking precautions is irresponsible today. If you don't have anti-virus, anti-spyware, firewalls installed on your machine, don't blame the browser.
4) If you are going to do online gaming, file sharing with Kazaa and the like, you will get infected. If you are going to visit online casinos and adult rated sites, you are going to accumulate spyware and cookies that will not only track you but can make you susceptible to a variety of attacks.
As has been pointed out by those who have responded, there are tools that can block popups, but if you are experiencing the things you are writing about, you will have to use Ad-aware, Spybot Search and Destroy, HiJack this, and variety of tools to dig the stuff out that are hidden not only in your cookies, your Temporary Files in IE but also in your Registry. You should look into using a different browser though all of them have vulnerabilities. I guess the best advice is that if you cannot rid your machine of the stuff that you have accumulated from the net, it is best to reformat and reinstall. That will take care of the technical/mechanical side. The bigger problem will be thinking about where you want to surf and what the consequences of surfing there will be. The net is not as nice as it used to be.