Kettle of fish
This expression is usually part of the phrases 'a fine kettle of fish', 'a pretty kettle of fish' etc, which mean 'a muddle or awkward state of affairs'.
The phrase 'a different/another/whole-new kettle of fish' has a separate meaning, which is, 'an alternative; a different thing altogether'.
Being English, I have a close association with kettles; essential equipment for a custom in which the English can still claim world dominance, the making of 'a nice cup of tea'. As you may have realised, the expression 'a kettle of fish' doesn't refer to tea-kettles but to the long saucepans that have been used for centuries to poach whole salmon, namely fish-kettles.
The noun 'kettle of fish' is listed by several reference works as dating from 1745, although the earliest actual citation of the term in print that I can find is in Thomas Newte's A Tour in England and Scotland in 1785:
"It is customary for the gentlemen who live near the Tweed to entertain their neighbours and friends with a Fete Champetre, which they call giving 'a kettle of fish'. Tents or marquees are pitched near the flowery banks of the river... a fire is kindled, and live salmon thrown into boiling kettles."
Incidentally, the two meanings of the verb 'poach', which are 'steal game' and 'cook by simmering in water', both derive from the French 'pocher', which means 'put in a bag'. The 'stealing game' meaning is pretty obviously connected with putting game into bags. The 'cook by boiling' is less intuitive and derives from the poaching of eggs, in which the egg white forms a pocket for the yolk. So, if a ne'er-do-well were to illegally take a salmon from a river and boil it for his tea, he would be poaching in both senses.
The French term fête-champêtre, meaning 'rural feast', was still in use at the 1780s to describe outdoor meals. The word 'picnic' (also French - 'pique-nique') was introduced around that date but wasn't widely used until a century or so later.
There's no obvious reason why a humdrum item of kitchen equipment was singled out as the source of a phrase meaning 'muddle or mess'. It may be an allusion to the confusion of bones, head and skin that is left in fish-kettles after the fish has been eaten. In all likelihood there wasn't any specific connection between the saucepans and muddle. It seems that 'kettle of fish' was picked (who knows why) as a synonym for 'state of affairs', or simply 'thing' and then various prefixes added to convey meaning. The earliest uses of the phrase, which apparently are examples of the 'muddle' meaning come from the English novelist Henry Fielding. In The History of the Adventures of Joseph Andrews, 1742, he writes:
"'Here's a pretty kettle of fish', cries Mrs. Tow-wouse."
and, in The History of Tom Jones, 1749:
"Fine doings at my house! A rare kettle of fish I have discovered at last."
Should there be any doubt about the meaning of the expression in Fielding's uses of it, there's an unambiguous explanation of the phrase in Francis Grose’s Dictionary of the Vulgar Tongue, 1811:
"When a person has perplexed his affairs in general, or any particular business, he is said to have made a fine kettle of fish of it."
'A different kettle of fish' is much later in origin than 'a pretty kettle of fish' and is known only since the 1920s. It's quite pleasing that, as far as etymology goes, 'a different kettle of fish' is a different kettle of fish.