This is a most complex matter due to the specifics of power generation (for example base power and peak power). There are two ways to look at the cost of electricity: One is the cost per kilowatt-hour as itemized on our electrical utility bill. This is the most relevant figure about the cost of electricity. The second way is the up-front capital costs involved in various power generation options. The cost per kilowatt-hour depends on the capital costs as well as the fuel, operation and maintenance costs.
A reasonably thorough study was carried out a few years back by PB Power for the Royal Academy of Engineering. It shows that, excluding any arbitrary carbon taxes, coal, gas and nuclear electricity is by far less expensive than non-conventional renewable sources. Interestingly enough, electricity generated by poultry-litter is competitive with wave power or offshore wind-farms. (Source)
Another 2008 study for the US Congress shows similar conclusions. It presents wind and solar costs without their required backup, assumed to be at about an additional 50% of the stand-alone cost. The interesting observation is that wind power is not only more expensive per kilowatt-hour, but has initial capital costs that are higher than nuclear or coal IGCC.
There is plenty of free wind and sun out there. The electricity they generate just happens to be very expensive.