That's perfectly OK, Marie, but stilted. Although it would be fine for academic prose, it's not the sort of sentence most people would utter. Perhaps a more idiomatic phrasing would be There are men I speak to who don't care about such things.
"In considering the use of grammar as a corrective of what are called ‘ungrammatical’ expressions, it must be borne in mind that the rules of grammar have no value except as statements of facts: whatever is in general use in a language is for that very reason grammatically correct."
[Henry Sweet, 1891.]
Marie, try to stop thinking "correct" and start thinking "appropriate" - it will be far more useful if you truly want to understand English. It's almost impossible to comment intelligently on this sentence unless it is put into some kind of context.
"He who dares not offend cannot be honest" Thomas Paine