This is actually not a simple question, but neither do the answers give an accurate picture of historical practice during the common practice period.
The Beethoven Fifth is a good example: we regard it as a "C-minor symphony" because its first movement is in C minor. Within that movement, C minor is, as we often say, the tonal center: it's the key to which we ultimately relate all the other harmonic events within the movement. The second theme (as it's frequently but imprecisely called) appears first in E-flat major, the "relative major" of C minor; that is, the major key counterpart to C minor whose scale contains the same notes (or "pitch classes") as the natural minor form of the C minor scale. When, toward the end of the movement, the second theme reappears, it does so in C major, the "parallel major" of C minor. Nevertheless, despite visits to other keys, some of which Beethoven establishes quite convincing, the movement begins and ends in C minor: it's the gravitational field that pulls everything towards it.
At the multi-movement level, the symphony's key is also important. According to the convention in multi-movement works like symphonies, string quartets, and the like, the minuet or scherzo--the 3/4-time dance(-like) movement that normally appears in the second or third position--also is in the tonic key; and, in the case of a minor key symphony, the middle section of them movement (called the "Trio") will appear in the parallel major key. This is what happens in the Beethoven Fifth. (Another good example of this is the Menuetto of Mozart's 40th.)
As a C-minor symphony, of course, we should expect that the symphony to end in that key. Yet that's not how Beethoven ends this one: rather, he ends in a famously triumphant C-major (emphasized at the end by an obsessive affirmation of the key). Without our expectation of the C-major ending, we'd have a very different piece on our hands, wouldn't we?
The bottom line is that the tonic key of a large-scale piece like a symphony is of vital importance for how we understand what it's about. While Beethoven Fifth is relatively straightforward, there are other pieces (like Beethoven's Seventh) where the composer does some wild, unexpected, clever things with the key relationships. But in all cases, at least until the early twentieth century, the stated key of a symphony is almost always a vital reference point.