robjrussel, this is nothing like BSD, which is an utter pile of crap. You can completely rid your system of cydia and friends by running the following commands in terminal (but i really dont see why you'd want to do this, and i strongly advise against it in any case, but here goes
apt-get clean
apt-get remove apt (when it asks, respond with Yes, do as I say!)
Boom! you're done, and you have a phone thats infinitely less useful now :P But yes, as I said, although it IS uninstallable, so many apps rely on tools provided by Cydia/BSD that you really should keep one or the other installed, its not hurting anything and can only help if you run into trouble in the future (having remote shell access to your iPhone can be a lifesaver)