Yeah, lots of good options, here. I'm thinking since your budget sounds a little tight you may want to gravitate towards the Lee stuff, at least for the time being, which would lead me to think you might want to start by looking at the Lee manual. Their products tend to be fairly ingenious and affordable, and make accurate ammo. Their presses tend to be great bargains, and work well. Do you have a bench or solid table top that you could drill holes in to mount one to? If so Lee probably makes a bench mounted press that you can afford, and you'll probably be better served here than the hand held press unless you don't intend to do a lot of reloading.
That said, a hand held press wouldn't be a bad place to start by any means. Neither would one of those Lee Classic Loader kits. They're so cheap! Just read through one of the manuals (particularly the Lee if you're thinking about the hand held presses or the Lee Classic Loader kits) before you start deciding here.
My suggestion, read through that manual first before you buy anything. Then start as simple (the more unsure you are, the more simple you should start) and come here whenever you have questions. Don't buy anything that isn't necessary to get the job done properly to start with. Then from there you'll probably have a better idea of what might work for your needs. I'm thinking start off with some kind of press, a set of dies, a way to manually measure powder (either a scale or a set of Lee dippers and a business card), a way to prime cases, some case lube if you're going to be reloading for rifles, some components, an inertia bullet puller to correct your mistakes, a pen and paper to keep track of what you're doing, some stickers to label your loaded ammo (bullet used, powder type and weight used, primer used, etc.), a flashlight to inspect your powder charges, and that's about it.
About reloading for dummies...sounds like you have a good plan putting safety as a high priority. It, via consistency and careful attention, and asking yourself (and others) questions about what you don't understand, should be top priority. You don't have to be a rocket scientist to be a great reloader, but you do have to be careful and treat the whole process with adequate respect. Putting a little thought and periodic reconsideration into intelligently organizing your workspace and workflow is not a bad thing, neither is adding redundant checks of what you're doing. Otherwise you may not be saving too much money if you're blowing up expensive firearms and paying expensive hospital or funeral expenses.
If you're determined to be safe and cautious, though, I encourage you to jump on in. It's a whole lot of fun, a relaxing hobby in its own right that is unique in that it gives you something "shooting related" to do without actually having to even leave your house! And it makes it a whole lot easier to afford to be able to shoot as much as you'd like.
