I have coded some AI for connect-4. I would like to adjust the weights in certain evaluation functions. I have limited time and hardware so my question is this: Is it very bad with respect to quality of the evaluation to perform the "training" and adjustment of weights based on results obtained from using lower-depth searches than those that will be used when the AI has to perform its best later in a situation where it has more time and hence can search the game tree at deeper levels ?
Well, if you have limited time, you have to adjust the weights using a lower depth search. This is very similar to the idea of temporal difference learning that is used to make games like backgammon. That is, you might want to use the idea of reinforcement learning. Temporal difference is a flavour of reinforcement learning.