The. fuck.
Please ignore those people, who clearly have a more than somewhat skewed sense of reality, and write what you want to write. I have yet to read anything of yours that is anything less than good (if not awesome!).
I am considering rereading "Back again, Harry?" and really would love to see it continued but please do not let the crazy people make you miserable, even if that means that you have to leave the story unfinished.