Many signs point to Feedburner closing soon as it's not being actively maintained anymore. Although it hasn't closed yet, users have complained about delays in email deliveries, and also loss of subscribers.
Therefore it might be wise to switch now.
Feedblitz has been around for a while, and they are a solid Feedburner alternative if you're looking for a service which sends out your posts automatically to your subscribers (based on your RSS feed), in a similar way as Feedburner does.
Feedblitz charges for the RSS- and Email parts:
See a full overview in Feedblitz' pricing table.
Switching from Feedburner to Feedblitz is made easy with a Feedburner migration guide.
Source: Feedburner-alternatives.com and Feedblitz.com
Feedpress is the to-go service if you are looking for a service to handle your RSS subscribers, esp. if you're a podcaster. Statistics are nice (see their features) and include Geolocation.
Feedpress also offers an RSS-to-Email solution, however you need to plug it into an external service.
Pricing: Feed analytics for 4 USD per month, podcasting solution for an additional 8 USD per month.
Source: Feedpress
Yes, you're looking for a Feedburner alternative, however if you are really looking for only free solutions then maybe you should consider sticking with Feedburner for now, and it covers both the email- as well as the RSS parts.
Mailchimp is a major newsletter provider and also offers a RSS-to-Email solution.
However, it doesn't cover the RSS part at all, so if you want to enable your visitors to follow your site via RSS, then this is no option for you (unless you're willing to not track your RSS subscribers).
Mailchimp is free up to 2,000 subscribers. If you have more than 2,000 subscribers then plans start at 10 USD per month.
Source: Mailchimp
SpecificFeeds is a free Feedburner alternative for both Email and RSS. It allows subscribers to filter the feed to only receive messages they want.
Source: Feedburner-Alterantives & SpecificFeeds
Aweber is a pure email newsletter solution, and it seems like they don't offer an RSS-to-Email solution, so posts don't get sent automatically based on your RSS feed, but you need to send separate mailings to your subscribers. Therefore only consider Aweber if you were thinking into putting more effort into your email newsletters anyway.
Pricing starts at 19 USD / month.
Source: Aweber
Feedio allows subscribers to follow not only certain blogs, but authors across many different blogs. They also position themselves as a possible replacement for Feedburner (however not tested yet).
Subscribe by Email is a plugin which you can install on your Wordpress site which allows your visitors to subscribe by email.
It doesn't offer any RSS tracking features, and also no migration guide from Feedburner, so if you already have Feedburner subscribers then this might not be the best option. However, if you're just starting out, then have a closer look.
Pricing: As part of a larger package it will set you back 49 USD every month.
Source: WPMUDEV
If you're using Wordpress and only need an Email solution, then consider using the Jetpack plugin. This might be a good option if you haven't started to collect emails in Feedburner yet. However, it doesn't provide any RSS solution.
If you don’t receive the email in the next 2 minutes please check your junk folder, and add [email protected] to your safe sender list.