It's a fair criticism to be honest. But, I can't think of a reasonable alternative. When I support trans-individuals, usually I argue from the brain-body mapping perspective as 1) It's not based on feelings as we know that there are people whose brain mapping does not correspond with their body mapping, and 2) There's a lot of evidence to support that perspective.
A lot of people aren't just going to get "What it feels like to a man/woman.", and personally, while I identify as a man solely for my parts, I just simply lack what it means not because it supposedly matches me, but because I have no feelings associated with gender. And here's the kicker, I wouldn't understand dysphoria when called the wrong gender either. For example, my mother has always addresses me as a male because I was born with a penis and all that jazz, but recently a few months ago, in a endearing way, she told me that I was like a daughter to her, which I take to mean that I give off that energy to her. Didn't really felt off or anything either.
As a Dem, I'm going to say that I don't support what Bill Clinton did given their power level difference. But, I do acknowledge what Trump did was way worse, and I will always compare him negatively to whatever you come up with.