Geological evidence indicate persistent tropical sea-ice cover in the deep past, often called Snowball Earth. Using a climate model, we show here that clouds substantially cool down the tropics and facilitate the advance of sea-ice into lower latitudes. We identify a critical threshold temperature close to 0 °C from where cooling down the Earth is accelerated. This value can be used as a constraint on Earth's sensitivity to CO₂, as recent cold paleoclimates never entered Snowball Earth.