Richard Rutt (1925-2011) was the former bishop of Leicester in the U.K. "The Knitting Bishop," as Serenity Knitting called him, was taught to knit by his grandfather, the village blacksmith. Rutt was a missionary in Korea for 20 years, where he lacked yarn for knitting and took up Asian history instead. Once back in England, he started knitting again and even knitted his own miter (the pointy hat bishops wear). His book A History of Hand Knitting was published in 1987.
A History of Hand Knitting has many interesting tidbits of history in it. Did you know there are methods of making fabric structures that are more ancient than knitting: nalbinding and sprang? If your friends think you are geeky for knitting, wait until you tell them you've taken up nalbinding.
Then there's the legend about shipwrecked sailors from the Spanish
Armada teaching the people of Fair Isle (off the coast of Scotland) how
to knit with more than one color at a time. It's unlikely that the
technique was learned from sailors, but still interesting to wonder what
passed between the 17 native families and the 300 sailors of the ship
El Gran Grifon who were, in fact, stuck on Fair Isle for 6 weeks.
I also liked hearing about all the literary references to knitting in literature: Shakespeare's Gentlemen of Verona, Tolstoy's War and Peace, Dickens' A Tale of Two Cities, Thackeray's Vanity Fair, Daniel deFoe's The Life and Adventures of Captain Robert Singleton, Adam Smith's The Wealth of Nations, Elizabeth Gaskell's Cranford, and Virginia Woolf's To the Lighthouse. Knitting appears in art, as well: check out the Knitting Madonnas set of photos on Flickr, some of which also appear in the book.