surely the only thing the gloss map should affect with regard to fresnel, is the distance the reflection travels (ie. the falloff)? for example, metal fresnel seems to be a fairly tight, short falloff, while a cardboard box would give a much broader falloff.