You wouldn't normally have capital letters there. I'd write it your department's chairman. You'd usually only have capitals in a job title.
"In considering the use of grammar as a corrective of what are called ‘ungrammatical’ expressions, it must be borne in mind that the rules of grammar have no value except as statements of facts: whatever is in general use in a language is for that very reason grammatically correct."
[Henry Sweet, 1891.]
I agree with Alan, but if you've been told that the phrase is "incorrect", there may be another reason. Some people have got the idea that only people (and perhaps animals) can "possess" things, so only they can have a possessive apostrophe.
This is utter rubbish, of course. We can certainly say - and write - that Wednesday's weather was fine, the Earth's atmosphere is polluted, and a diamond's sparkle is inimitable.
I'd say that not only should "Department" have a lower case "d", but "Chairman" should have a lower case "c". Both "department" and "chairman" are being used as a common nouns (as opposed to proper nouns), so neither of them merits a capital initial letter.
... your department's chairman ... ... my chairman ... ... the previous chairman ...
But: Mr Chairman (as a form of address) Dear Chairman (e.g. on a letter to the chairman) Greetings, Chairman.
(The apostrophe's correct, as Dave has already stated.)