The first question that might come to mind is "Wait, don't kittens live only on milk?" Kittens are meant to only drink cat milk, because its formula is designed for cats.
Lactose, the sugar in much dairy, including cow milk, is bad for cats. Giving a kitten or a cat milk from a cow is a very bad idea, and will usually result in a sick cat with diarrhea.
Even when kittens don't have mothers from being orphaned or fostered, cow's milk shouldn't be given because it makes them sick. They actually need special kitten formula.
After four weeks, kittens begin to wean and stop drinking mother's milk. From this point on, it's not even nutritionally necessary to give your cat milk, even milk from a cat. A bowl of water serves well.
Though not all cats are lactose intolerant, milk should usually only be given as a treat, if ever. But if you're still intent on giving your cat milk, some pet stores sell special lactose free cat milk, which is a much better treat than ordinary cow's milk. Still, water is always better for hydration.