This may quickly evolve into a mini rant, so fair warning.
To the individual that justifies talking shit about someone because "that's my brother": that is exactly why you don't talk shit about that person, you ignoramus. It's not your place to tear them down, in fact, a family is supposed to be there to back you up and stand up for you when other people do that shit.
So when I ask if you would want someone to say those things to your significant other you cannot justify it by saying "That's different, she's my wife" because it's not, in fact, it is exactly the same thing. You wouldn't let my boyfriend talk shit to your woman about you. I will not let you talk shit to me about him. End of story.