One of the easiest ways to start your own business is to take advantage of the freelance market. You can start slowly and just take a few freelance projects a month, keeping your full time job until you build up a clientele, or you can dive right in and work on the business full time, even hiring others to work for you.
Freelance businesses include a wide variety of options including Web & Programming, Design & Multimedia, Writing & Translation, Sales & Marketing, Admin Support, Engineering & Manufacturing, Finance & Management, and Legal. Freelancing is something you can do from your home with very little outlay to start. Another advantage to freelancing you can work from anywhere in the world, and have clients that are located all over the world.
If you plan to Freelance you should lay the groundwork well, know what you should charge and what taxes you will have to pay. For an article about freelancing see here.