At a practical level, your initial instinct was right. In space, nobody can feel you convect. Essentially all of the waste heat has to be disposed of via radiation, and this is orders of magnitude less effective than convection at the temperatures that any known material can withstand. Worse, you don't have to radiate "just" the waste heat. Eventually, all of your electric energy becomes heat, too. In other words, all of the energy you extract from your decay source will end up as heat that you have to get rid of via radiation. (Well, you could wind a giant spring, I suppose. But, a spring that large would be a better radiator than energy storage device.)
All that being said (and true!), you also have a very cold sink to reject waste heat to, whether you want to or not. The end result is that heating is as much an issue as cooling. Thus spacecraft usually wind up with blankets and radiators, both. :)
“RTGs use thermoelectric generators to convert heat from the radioactive material into electricity. Thermoelectric modules, though very reliable and long-lasting, are very inefficient; efficiencies above 10% have never been achieved and most RTGs have efficiencies between 3–7%”
Also interesting, the price of RTGs:
"Plutonium 238, which is used to power Radioactive ThermoGenerators such as the ones that power the Voyagers, Galileo, and Cassini probes has not been produced in the US for 25 years, but NASA and DOE have budgeted $50 million to restart production of about 2 Kg per year for 5 years…an average cost of about $5 million per Kg. That is expensive stuff!"
One SNAP-27 unit, the type used in Apollo missions, providing only around 70 W, had around 4 Kg, which means its Pu price alone would be at least around 20 million, if it would be produced now.
