I'm looking to build a website and was wondering what should I look for in a hosting provider? I thought buying the domain was all I needed to do. Where do I find hosting for my site? Can I just go with where I purchased the domain?
Firstly, there is no reason why your web hosting company has to be the same as the company where you registered your domain. These are very often two different companies chosen for very different reasons.
If you are looking to host a small site with some largely static content and don't plan to use a Content Management System (such as WordPress, Drupal, Joomla etc.) you can go with any company that you've checked out and is reasonably priced (Such as GoDaddy or Dreamhost).
If you are looking to use a Content Management System (CMS) you should ensure that your candidates meet the minimum server requirements for the CMS you will use.
All Content Management Systems require a backend database. Typically these are done with MySQL for php based systems. You'll need to ensure that the quantity, version and type of database offered through your package meets the CMS requirements.
If you are planning to use a .NET based site, you will want to use Microsoft's IIS server running on a Windows machine. If you are planning to use a php site, you're best served with a unix/linux system running Apache or nginx.
Different hosting companies have varying web administrative control panels. This could be more or less important to you but it is worth seeing what the control panel looks like. Some hosting providers require you to be an experienced server administrator and if you have that experience you can usually get comparable server specs for a lesser monthly charge.
The final consideration is the amount of server resources you will need. These will vary dramatically based on the number of users and complexity of the site and will likely change over time. For this reason you'll want to compare the RAM available and file space offered by web hosts to ensure it meets your needs now and into the future. While switching hosting providers isn't a major hassle it's nicer to choose a provider that has tiered service plans that you can move up to as your needs become greater.
One last tip. Create the hosting account in your own name and give your developer access. Some developers will arrange hosting for clients and pad the monthly hosting charge. Creating the account yourself will also give you control should your developer stop paying attention to your needs or disappear on you.
Hosting providers don't really affect SEO too much so long as they have solid up-times (over 99%) and they aren't dreadfully slow. If you are truly concerned about SEO for your site, make sure that you have a dedicated server along with a redundant server in a different location to minimize any chance of downtime with your site.