I actually removed the Internet Explorer exe from my filesystem without incident. You can't have windows running when you do it. I have two operating systems on my computer, so I booted into linux, mounted my windows partition, and removed the executable. I have not had any problems.
Well, it's not only IE. It's every web browser. That's just what they do. After a while of leaving your browser open, it's starts to use more memory. Just close it and open it again and it should be fine. But it is relatively normal though.
Eh I can have only one or two tabs open and the memory be high, but I do see how more windows/tabs open would increase memory (I have a friend who insists on having 100+ tabs open so he can keep the music he's too lazy to download open. Crazy!)
Um. No. What ~NarcisistWaffle said, anyway. Firefox has its own cache and temp storage; it doesn't use IE's directory structure. Nor is IE the "general thing you need for your internet". That would be something called a TCP/IP stack, which on Windows is defined as part of the "Winsock API". While it's true that IE must call Winsock, Winsock is not part of IE. Any networking you do whatsoever, even on your LAN, must use Winsock.
I assume that what you mean by "memory" is really disk space. If so, then chances are, it comes from running Windows Update, which uses IE whether you want it to or not. Other Windows software that needs to render HTML might also use IE's libraries behind the scenes, and that could account for some of it too.