Ray didn't think there was anything particularly wrong with his hat, unless Asher just had something against them in general. Which wouldn't really make sense, considering his comment in the summer.

"Somebody's fickle. Thought you wanted me to wear them, lest you get bedazzled?" The closer Ray got, the more he could see. "Oh, eugh, eughhh, what is she doing?! Why is she licking that?! That is disgusting!"

If women did that right after birth and men saw, humans would have been extinct long ago.