While using the microphone in front of the speaker, it makes a howling sound. Why is it so? How can I avoid that without reducing the sound?
This is called feedback.
Put simply, the microphone hears some sound. It sends it to the amp. The amp sends it to the speaker. Some time has passed. The microphone hears the sound from the speaker (now louder), sends it to the amp, and round-and-round it goes, getting louder each time.
After a few goes around the loop, you reach internal limits and resonances in the mic, amp and speaker, which work together to distort the original sound into the howl you hear.
To avoid this, you need to ensure that the microphone "hears" as little sound from the speaker as possible.
Turning the volume down is one simple solution, but of course often you don't want to do that. Moving the microphone further away from the speaker also helps.
A more useful solution, is the knowledge that many speakers and mics are directional. Speakers send sound in a certain direction. Mics described as "directional" pick up sound from a certain direction.
So, point speakers away from mics. Point mics away from speakers. If you have control over EQ, put a notch at the frequency that's causing feedback, in the speakers that are causing problems.
Some mics are decribed as "omnidirectional" - they pick up sound from all directions. These are good for recording ambient sound. They are less useful for live amplification because they make it difficult to control feedback.
One very common approach is to make the stage area much quieter than the audience area, by putting the PA speakers in front of the band. The band won't really hear much from the PA speakers because they are pointing away from them. The band hear themselves through monitor speakers, which are quieter than the PA, and which can be EQ'd to prevent feedback.