First heat is the enemy of leather...maybe the worst enemy. Sunlight generates heat. To the extent that you can control the heat generated when an object is left in the sun (really?!) the damage may be mitigated...somewhat. But of course darker shoes will absorb and retain heat more readily than lighter shoes. Heat volatilizes oils and makes them either fugitive or in cases of extreme heat can actually cook the leather.
As far as causing the colour to fade, I suspect it is the ultraviolet rays that are breaking down the finish and changing the molecular structure of any dyestuff that has been applied...before making and/or after. To the extent that these finishes are anything but superficial, I would worry that damage was being done to the grain as well.
Re: elk and moose...and deer. These leathers are used but because of the way in which the fiber mat of these hides is structured the leather tends to be very coarse, the hair follicles wide apart and very large, and the leather extremely soft, spongy and very stretchy. For some people...who, presumably, would really rather be wearing sneakers or felt slippers...such softness and stretchiness is the appeal.
That said, I suspect most reputable shoemakers would tell you that they don't like working with these leathers and that the suitability to task or function is really limited.
Thanks, DWFII. As always, a detailed and comprehensive explanation. Good reasons not to bake your footwear in the sun to accelerate "patina".