That makes sense, I am both tall and reasonably lean and I do very badly in the cold. At the other end I can live in heat that will kill most people. If you look at genetic adaption in different parts of the world, both current and past, folks who survive in very cold climates are physically compact and reasonably rotund, they store more body fat for both insulation and energy reserve. People who have a genetic history from temperate to hot climates are usually lean and long, people who come from a genetic history of cold climates are a different adaption.
Similar thing with skin colour, people from cold climates are often very fair and can absorb vitamin D even in low light conditions where folks from hot climates with dark skin often suffer from Rickets (vitamin D deficiency) if they live in a cold climate. At the other end dark skinned people can survive in hot climates and high sunlight and absorb enough vitamin D where fair skinned people get badly sunburnt and are high risks to skin cancer.