Getting it too hot, then cooling it back down will kill the texture and seems to dull the flavor also (or maybe you just don't taste chocolate chalk as well).
So protecting it from heat is more important to quality than protecting it from cold.
If you refrigerate it, it will still get some bloom on it. But it will warm back up and be edible and enjoyable.
Doesn't affect baking quality - you can still hack it up to make chocolate chip cookies and no one will know the difference.