I'm studying McEliece and Multivariate Public Key cryptographic systems. The main problem here is the huge key size.
Is there any restriction in using lossless compression algorithms to fix this problem?
The problem is that the key is essentially random data and thus cannot be meaningfully compressed. Several variations of the McEliece cryptosystem have tried modifying it to produce public keys with special structure which are compressable. However, all such systems have been broken (as far as I know), and it seems that in most cases, adding special structure like that makes it more vulnerable to attack.