A person's physical appearance is simply what they look like superficially. In the modern world, following the invention first of photography and then of the internet, it is scarcely conceivable that a person might become famous without their physical appearance being generally known and recognised. However, in other eras of history, this was very possible or even unavoidable.
Before the advent of photography, people's likenesses were recorded in paintings or sculptures. These could often show inaccuracies, either deliberately because the artist idealised the subject, or inadvertently because of a simple lack of skill. Even if they were accurate, onlookers would have to visit or be shown the work in order to see the likeness. Famously, Henry VIII of England married his fourth wife Ann of Cleves having only seen her portrait, and was extremely disappointed when he met her in person!
In even earlier times, even portraits were rare. Before the Renaissance, painting techniques were often rudimentary or ineffective (so works did not last) and the techniques of artistic observation had not advanced to produce true likenesses, save in exceptional cases. Notice how representations of saints, for example, are stylised so that the saints are often identified from the colors of their robes or by the objects they are carrying rather than their physical features.
It seems, given the impossibility of knowing the true appearances one had not personally met, that physical appearance was far less important in earlier times than it is today. Notice that the Bible rarely describes people physically, except where their physical attributes are important to the story, such as Goliath's great size, Samson's hair or Esau's fur-covered skin. For the most part, people are allowed to be depicted by their actions and their words, so that physical appearance is not considered. The authors of the Bible simply did not consider it at all important.
In fact, God as described in the Old Testament is notable for His reluctance to be seen. Idols are forbidden, and even Moses is only permitted to see God's back, not His face. Of course God could awe people with His appearance, but again he prefers to be known through words and deeds and not superficialities. Contrast this with other (false) gods such as Baal, whose followers made idols and graven images wherever they could, but whose deeds were paltry or non-existent, as proved by the prophet Elijah and recounted in 1 Kings 18.
In the modern world, however, judgments of those in the public eye are becoming ever more superficial - contrary to the scriptural example. President William Howard Taft was a greatly obese man who was nonetheless able to be elected to high office and make not inconsiderable achievements; but it in the modern world it seems that any perceived physical defect would count against any candidate. Certainly it seems that politicians generally are becoming younger and more fresh-faced; the recent British Prime Ministers David Cameron and Tony Blair were both well below the historical average age for a Prime Minister, as indeed was John Major, while older men such as Michael Howard and David Davis have been marginalised and even ridiculed despite their superior experience.