Do you use a dauber brush for cream/conditioner or a cloth? I use a dauber/welt brush for creams. I couldn't imagine why I never washed them before. I noticed my horsehair brush was a little "harder" or "rougher" than I would have liked. I could feel some hairs being clumped together just by running my hand over it, particularly in the middle of the brush.
Simply put, the bristles get stuck together and turn hard because of the dried cream, especially on a welt brush.
I've only used cloth for cream/conditioner. I apply wax/cream on welt using old toothbrush. My daubers/welt brush are, literally, used to brush the welts and hard to get places.