You can lose a tremendous amount of heat through your uncovered head, so wear a hat, cap, or hood.
Your feet get cold first. Wear warm, waterproof boots or shoes. Insulate them with warm socks and keep them dry.
Because of the large surface area to volume, your hands are also more susceptible to cold. Gloves or mittens should be worn before the hands become cold. Choose mittens over gloves, because the fingers can warm each other and the mitten decreases the exposed surface area.