Thanks for trying out!
For the first issue:
great to know that you figured out. This is an temp issue because recent update to pytorch 2.11 and we will resolve it soon. Or maybe use any commit before #34644 - 2111997f96f33b118ec0c562cc2df5681862cff3 as temp workaround.
For second issue:
I noticed that you're using 2*B60, so total device memory is <48GB and google/gemma-4-26B-A4B-it has total 51.6GB in weight. Please try TP=4