That is the potential problem, if it comes out "nice 'n' warm" then you instantly start using it, taking it to even higher temperatures internally, that is when permanent damage might occur.
The electronic components are probably specified, in consumer grade equipment, to 70 or 85 degrees centigrade, but designed to say 150 degrees for military applications. In consumer kit, you might get an odd component that isn't that robust and it only takes one to fail ...
I appreciate you don't buy 'consumer grade' kit Colin and that probably also helps because they'll build to a higher standard. (e.g. not use 'hot melt glue' to stick things in - which softens when warm and makes it more susceptible to shake/drop damage)
So as long as you're sensible and don't do it too often, and/or allow some cooling time (whenever possible), even consumer kit probably won't fail (but don't sue me).
At cold temperatures, the main issue, engineering wise, is that a battery producing electricity is a chemical reaction and they slow down when cold, hence apparent fall in capacity. Keep spares, as I think Richard mentioned, well inside your clothing, next to your body for warmth. It is also probably one of the reasons that the battery is within the hand hold grip on most cameras, your hand shields it from losing heat so quickly - but that's just a theory, I have no idea whether there's any proof that, on a tripod, it loses capacity any quicker than my preference for handheld, but I do know I don't have half the issues with it going flat I read about here, even if out for 3 to 4 hours in sub-zero (centigrade) temperatures
A secondary issue is condensation; not when going into the cold, but when coming back into the warm. I was out in the garden a shooting couple of days ago, it was about zero (felt less with wind chill), when I needed to nip inside the house to do something, I left the camera outside so as not to 'fog' the filter indoors, allowing me to resume shooting instantly (not having to wait for fog to clear) - and yes, it did mist over when I came indoors eventually.
When operating at temperature extremes (high or low), unless the electronics is properly designed for it*, you may experience drift of accuracy of certain measurements and settings, but whether you'd make an association to the temperature or just put it down to 'the light on the day' is debatable; for example - if a series of shots comes out 1/2 a stop over or under exposed.
* this is again an area where one would hope a branded or semi-pro/professional item will perform better than cheaper alternatives
Just some thoughts of a nerdy ex-engineer - I call it "insight" (to how things work), you may well call it "boring as heck"
EDIT: sjgordon also makes a valid contribution on the effects of temperature on lubricants, which I suspect may also get thicker when cold
Cheers,