It definitely seems as though he is getting some sort of satisfaction from this, whether from the attention you give to his volume or just pure joy of doing the activity.
When my kids were younger, at that age, the time outs, etc didn't work either. What I found that does work is first, not to pay too much attention to the issue to where he knows its driving you mad.
At that age, they are learning to push boundaries and buttons. That doesn't mean you cannot tell him that he needs to lower his voice. Saying something like, when you talk or sing too loud, it really hurts my ears.
Usually, kids that age don't want to hurt their parents in any way so that might help. When my kids would do physical things that would hurt me such as kicking and screaming during a tantrum, I would calmly, firmly but sadly tell them that they are hurting me, it doesn't feel good and they are hurting my feelings.
I would definitely reassure them that you think they sing beautifully, ahem lol, but you wish you could enjoy it better if it was a little quieter. This is to not put the child down to make them think they are good at what they clearly enjoy doing. They'll find out when they are older, by peers, if they stink at it or not lol.
Another thing that works for children of that age and even a little younger or older is distraction as a behavior changing method. When children do not respond to other forms of discipline, the distraction technique tends to work very well.
When the child is doing something inappropriate as far as you're concerned, divert or distract his attention to something else, in this case, an interesting quieter game or activity and enjoy it with him.
Try games that require thought such as the game memory, where you actually have to be quiet to think of where the matching pieces are. Or any other thinking games.
Make sure when offering your diversion/distraction, you make it sound fun and appealing with enthusiasm so they get excited to join in. Other games that require some quiet thought, are building games such as legos, building blocks because they have to focus on what pieces go where, what pieces fit together, how to create a structure of some sort.
Thinking games are great that way. Praise them for using their indoor voices of course while playing nicely.
Also, reading is a great way for you both to sit and enjoy , hopefully semi quiet time, depending on the book. Picture books also if he isn't reading yet.
One key thing to remember is to catch them "being good" or doing the behavior that you'd like them to do. When he is absorbed in a thinking game, for example, try saying, "I love how you are using your indoor voice while playing this game!" , cause chances are they aren't going to be silent lol. There will still be chatter. Remind them that during these sorts of activities that they need to lower their voices in order to focus and concentrate to play the game, activity, etc correctly.
Also, there is nothing wrong with asking if they'd like to sit and watch a favorite movie or t.v. show to take their minds off the singing for a while. There are some great dvd's out that help relax children as they are watching. Also, dvd's that teach children about indoor and outdoor voices. Barney was always my sons fave, that's how we got him into the indoor voices. Play a whispering game and see who can be quiet or whisper the longest and maybe have a reward for the winner.
Drawing is also usually a quiet activity as well as coloring.
I hope you find some of this helpful and good luck. He may be the next American Idol someday :) As always, positive reinforcement works better than negative attention. Catch him listening to what you ask him to do. Take Care!