If you do get too close then he will call for you to "Halt!", but will not intiate dialogue. It will be up to you to initiate dialogue, to which his standard response is, "You have no business here, so keep walking.", you can ask him, "What is this place?", but he will curtly reply "That's privileged information. Now move along."
If you have progressed the quest Missing In Action far enough, the dialogue option "I'm here for Thorald Gray-Mane." is made available, which results in, "Are you joking? Even if we had a prisoner by that name, the only way he'd be released is by order of the Imperial Legion. Now I suggest you turn around and go back the way you came."
The related quest has the option to try, but cannot actually be peacefully solved. There is dialogue for him to have accepted an Imperial Legion Decree, and he would have escorted you to the prisoner, and released him.
If you attack him, he will become hostile so does other Northwatch guards, archers, and mages.