I did this recently, so if we are talking coarse fishing, at it's most basic you want:
Rod
Reel
Line (strong enough for the fish you want to catch)
Terminal tackle (floats, hooks, yadda yadda)
Disgourger/Forceps
Landing net (some lakes and clubs have a minimum size... but then again I've never seen anyone kicked out for having a smaller net)
Unhooking mat (some venues require it)
I went the route of picking up cheap stuff from ebay. You can get a perfectly serviceable reel with line for a tenner, about the same for a rod. There's bundles of terminal tackle on there too. It's not amazing kit, but it's good enough and the fish don't know how much your reel cost.
I really wanted to minimise the amount of money I put into a hobby I might get bored of, but now that I'm hooked again I can slowly start upgrading.
Most of the expense comes from the little bits and pieces of stuff that isn't essential but is good to have... like bank sticks, tackle box, rod bag, folding chair... not too expensive individually but they add up.
