Is there a quantitative method in monitoring trades to reduce the possibility of correlated trades?
The most straightforward approach is to develop a covariance matrix to ensure that you are not overweighting to the same factors or bets in your trading. The covariance matrix can be built off of a factor model, for example, or you can construct a covariance matrix based on your prediction signals if you have multiple models. In this way you can understand the ex-ante historical correlation of your trades.
Note that there is a considerable amount of art in designing a covariance matrix (See Fabozzi).
Without understanding more about your approach it's hard to be more helpful than the above approach.
Historical correlation isn't as useful as you might think. Like volatility, correlation is not constant. During times of stress, it is common for the correlation of many different assets to increase/change.
As examples, look at the data for August 2007, the last quarter of 2008, and May 6, 2010. Any trading scheme that minimized correlation before those periods probably had a much different affect during those periods.
Edit 1 (05/10/2011) ===========================
I've bumped into all sorts of problems with this issue, with the estimation itself involving large errors. If you dig around, you'll find several papers with important improvements.