You have to be careful with what you say and who you say it to. No ethnic group that I know of would like to be called "brown". There are no "brown people" or "browns" in America.
The term "black" is used more frequently to describe a person of African heritage, but the most accepted term is "African-American".
"Brunette" is only used in reference to hair.