Fiance doesnt stand up for me?

Everytime my fiance goes to see his daughter her grandmother is always saying "you should've stayed with my daughter...i told him I feel disrespected by that and he should tell them he didnt want to hear that and that they need to respect his relationship. he responds you want me to be on there bad side..Um quite frankly I could care less..Am I wrong to feel the way I do