The answer is: Never. And if the neighbor's objections were the only thing I'd received, I wouldn't have thought twice about issuing the permit. My only concern, as I tried to explain before, is that when I receive a 'lawyer letter', I am not so presumptuous as to think that I understand the law as well as the attorney (my gut feeling is that this is just a bluff and carries no real weight, but at the same time, I want another attorney - ours - to review the letter and the circumstances and give me some guidance). "Cover my a$$?" - That would be one way of putting it, yes.
"Keep your job" - I like to think that is always a secondary (or lower) concern, but at the same time, I would really hate to lose it over something as insignificant and non-building-code-related as a blankety-blank fence!
In short (and refer to my 'handle' on this board, as well, which suggests my relative 'newness' to this): I make a very deliberate effort to not go about my job thinking I know everything, which I would hope you would appreciate, given the general theme of many of your posts regarding code officials (with which, I many times agree, by the way). In this case, I do not intend to indefinitely delay the issuance of this permit, I simply want to verify, through expert opinion, in accordance with the following:
104.4 Inspections.
The building official shall make all of the required inspections, or the building official shall have the authority to accept reports of inspection by approved agencies or individuals. Reports of such inspections shall be in writing and be certified by a responsible officer of such approved agency or by the responsible individual. The building official is authorized to engage such expert opinion as deemed necessary to report upon unusual technical issues that arise, subject to the approval of the appointing authority.
I consider receiving an urgently-worded 'lawyer letter' to be an 'unusual technical issue' - I can't recall a similar occurrence in the recent past.