To get started with your first website, you only need two things:
When I first started, I bought a cheap hosting plan from Bluehost ($255.96 for 3 years) which allowed me to host an unlimited number of websites. Of course, the tradeoff is that the overall performance was generally worse than more expensive packages.
I later migrated to Siteground which had a good deal and seemed to offer better performance, although I do hear that it's been going downhill in recent years.