I think a perfectly acceptable line to draw is "Is it reasonable to expect a large majority of the people on this instance would want this other instance blocked?" If the answer is yes, block them. If somebody has a problem with that, move to a different instance.
I don't really understand what the problem is.