Hi. To a very large extent, an individual's skin color is as a result of their skin pigmentation (a material in the body that changes color of reflected light as a result of wavelength selective absorption) called the **melanin** which can be traced from their genetics makeup of both parents. Overtime, the skin color could change due to the exposure to the sun.
Talking about what could destroy the skin color, I think what we eat, the soaps, creams and how well we take care of our skin plays a major role in making it healthy or destroying it. Been careless about the skin by exposing it to harsh weathers like the sun, using harmful creams and soaps could lead to a permanent destruction of it.