What does 'i1' in title means ?

#1
by kalashshah19 - opened

What is its full form and meaning? And how is it diff from mradermacher/Trinity-Nano-Base-GGUF ?

Trinity-Nano-Base-GGUF contain the static and Trinity-Nano-Base-i1-GGUF the weighted/imatrix quants. For static quants everything is quantized the same way while for weighted/imatrix quants an importance matrix is commutated based on around 164352 token measurements so see what ports of the model are important. During quantization the important parts of the model are then quantized in higher precision than unimportant parts of the model. This allows weighted/imatrix to be much closer to the original model than static quants. If unsure always choose weighted/imatrix quants as they far exceed the quality of static quants in every use-case imaginable. I recommend you take a look at the quality column on https://hf.tst.eu/model#Trinity-Nano-Base-i1-GGUF. Use the drop-down in the header of the column to play around with different quality metric. That should give you a great idea how the quality differs between the quants we offer.

Sign up or log in to comment