Japanese homes are built for summer
That's right. Japanese summers are very warm and humid, leaving you no escape from the heat. Aside from that, mold and mildew is a big problem in Japan, causing respiratory and health problems in severe cases. During the old times the option for most Japanese carpenters was simple, "during the winter you can always put on more clothes but there's no way to escape heat and humidity." That is why Japanese homes are built with plenty of ventilation, open windows and means to let the air circulate and cool down a house.
One room at a Time
The idea that many Japanese have is to keep yourself warm over keeping a whole house warm. In old times people had one hearth in a central place called an irori (いろり). This hearth would also be used to cook and smoke food. It even helped protect the house itself by drying out the wood with its heat thus preventing rot, fungus and wood disease. Thanks to the heat of the irori many homes have been beautifully preserved. If you see an irori it usually has a fish decoration somewhere, symbolically protecting the house against the fire of the hearth.
This idea of "one room at a time" is still visible in Japanese homes today with the use of appliances like space heaters and the kotatsu.
The idea of "Warm yourself first"
As mentioned before, the principle is that you warm yourself before you start warming an area. From an economical point of view this is very smart indeed. But for most Japanese they don't have any choice because their homes aren't built to preserve the heat from an airconditioner for long. Back when people wore kimono daily they wore a hanten during winter. This is a coat similar to a haori and consist of many fluffy layers of cotton for warmth. Families would huddle up next to the hearth and drink warm tea or eat a hotpot with their hanten on. You can still buy these warm jackets today.
People on the go have options other than hanten and hearths, opting for warm layers and hot packs called kairo (懐炉,カイロ, literally means "pocket hearth"). The first form of kairo were simple tin cases. Special coal pieces would be lit and inserted in the case and people would bring them around in their kimono. Nowadays, you are more likely to find disposable kairo packs at any convenience store or supermarket. They become small sources of heat the moment you open the package. You can opt for the sticky kind, to stick on your clothing, or the non-sticky kind for holding in your hands. There are even versions to put in your shoes.
Keeping the warmth
The key to surviving Japanese winter is to create as many hot spots in your home as possible to ensure you're not in a cold space for too long. Soak in a warm bath or onsen. Bring out a space heater to warm your bedroom, wear a hanten while holding a hot pack when going to the living room, then immediately slip under the kotatsu to enjoy a hot pot and go to sleep with your electric blanket. Now you're ready to survive until spring comes.