Unattractive people, on the other hand, are seen as unappealing even if they smile broadly because the underlying structure of the face is what others recognise, not the expression.
But expressions can still make a small difference as a signal of whether someone is worth pursuing as a mate, the researchers found.
Dr Ed Morrison, of the Department of Psychology at the University of Portsmouth, wanted to study whether attractiveness was more to do with how you look or how you present yourself.
He found facial attractiveness remained steadfast in other people's minds no matter what expression a person displays.
Dr Morrison said: "The hard tissues of the face are unchangeable and it seems people, both men and women, can tell if the underlying structure of the face is attractive or not no matter what expression a person has on their face.
"Evolutionary theory has long suggested facial attractiveness is one of the key cues of someone's biological quality and that humans prefer an attractive face when choosing a mate."
The study did find small differences in how others rated a person's attractiveness based on the expression on their face, but the variations were limited and only about half as important as variations between the people themselves.
Also, happy expressions were no more attractive than neutral expressions.
Dr Morrison said: "People do make decisions about the attractiveness of others based on their facial expression, but this has less to do with deciding if they are attractive and much more to do with deciding if someone is worth pursuing as a mate.
"For example, disgust or contempt are a clear signal to not pursue, while smiling or neutral expressions are likely to mean mating effort will be rewarded."
The study was conducted by asking 128 men and women to rate the attractiveness of 30 men and women, each of whom was photographed expressing happiness, neutral, sadness, surprise, fear, anger and disgust.
Both men and women rated attractiveness of each face relatively equally, indicating the sex of the model and the sex of the person rating them played no part in their decision.
Dr Morrison said: "Our findings are consistent with the idea men and women see attractiveness in the same way.
"This is probably because people have to be able to evaluate the attractiveness of the opposite sex as potential mates and the attractiveness of people of the same sex as competitors."
The research is published in the Journal of Nonverbal Behavior.