on your server (the one you control) and create a disposable user just for accepting transfered files. Make the shell for this disposable user "scponly". Now you can safely use scp or sftp from the remote server in a cron job using an identity file with no passphrase.
Of course, you must be very careful about how you handle the files that have been uploaded to your server because they could contain malicious data. You should do some kind of verification on the files before you use them in a trusted manner.