The term for it is “micro-aggression.”
{audio}http://cpa.ds.npr.org/krcb/audio/2014/11/NBR_11-18-14_Micro_aggressions.mp3{/audio}

Much of the time, suggests Santa Rosa psychologist Dr. Marco Lopez, casual “micro-aggressions” are more thoughtless than intentionally disparaging.
{audio}http://cpa.ds.npr.org/krcb/audio/2014/11/from_0.mp3{/audio}
But the person on the receiving end of such remarks, Lopez continues, is going to recognize their hurtful nature, even if the speaker doesn’t.
{audio}http://cpa.ds.npr.org/krcb/audio/2014/11/received.mp3{/audio}
One active online forum for discussion of this issue is The Microaggressions Project. You can also see a photo gallery of examples here.