Have you checked out the Icelandic horses? If you've never been horseback riding before, the Icelandic horses are a good starting point since they are smaller than other horses.
I went on a half day tour with Reykjavík Riding Center last week. It was really sunny and warm and this was a really fun tour. The guy that runs the place was super friendly and chatty and always making jokes. After a bit of coffee we headed out for a short ride, only 1,5 hours on the horse. If it would've been much longer then I would probably have gotten quite stiff muscles the following day since it's been so long since I went horseback riding!
The group was small, they don't normally have more than 10 people in a group - but we were only 4. I went with my sister and in addition there was a Finnish couple, who were beginners. Since my sister and I are a little more advanced we got 2 tour guides with us and they split the group in two, so we could go faster with one guide. So we had our own private guide! And the horses were really beautiful, mine had an Icelandic name that was too hard for the tourists to pronounce so she was renamed Bráný (Brownie). My sister had a gorgeous white one. Mine was taller so I got to enjoy finally being taller than my sister!
The tour took us to Rauðhólar (Red hills) in Heiðmörk that are just outside Reykjavík, a really beautiful area. So it was a lovely sightseeing tour as well.
When we got back to the horseriding centre we were offered some spicy lobster soup and bread, that was really tasty. Since it was so warm outside it felt a bit odd having hot soup - but I'm sure it's very welcomed during the colder days!
The company offers a variety of other tours, such as a midnight riding tour - or a customized 3-9 day tour along the beaches of Snæfellsnes for the more experienced riders! I had so much fun last time that I think I might want to check out their midnight tour next time round, apparently that one includes some pieces of chocolate as well!
But if you want to check out the lovely Icelandic horses, this short taster tour is a good pick! :)