| | --- |
| | license: apache-2.0 |
| | library_name: transformers |
| | tags: |
| | - code |
| | - granite |
| | - llama-cpp |
| | - gguf-my-repo |
| | base_model: ibm-granite/granite-20b-code-base |
| | datasets: |
| | - bigcode/commitpackft |
| | - TIGER-Lab/MathInstruct |
| | - meta-math/MetaMathQA |
| | - glaiveai/glaive-code-assistant-v3 |
| | - glaive-function-calling-v2 |
| | - bugdaryan/sql-create-context-instruction |
| | - garage-bAInd/Open-Platypus |
| | - nvidia/HelpSteer |
| | metrics: |
| | - code_eval |
| | pipeline_tag: text-generation |
| | inference: true |
| | model-index: |
| | - name: granite-20b-code-instruct |
| | results: |
| | - task: |
| | type: text-generation |
| | dataset: |
| | name: HumanEvalSynthesis(Python) |
| | type: bigcode/humanevalpack |
| | metrics: |
| | - type: pass@1 |
| | value: 60.4 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 53.7 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 58.5 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 42.1 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 45.7 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 42.7 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 44.5 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 42.7 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 49.4 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 32.3 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 42.1 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 18.3 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 43.9 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 43.9 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 45.7 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 41.5 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 41.5 |
| | name: pass@1 |
| | - type: pass@1 |
| | value: 29.9 |
| | name: pass@1 |
| | --- |
| | |
| | # mayank-mishra/granite-20b-code-instruct-Q4_K_M-GGUF |
| | This model was converted to GGUF format from [`ibm-granite/granite-20b-code-instruct`](https://huggingface.co/ibm-granite/granite-20b-code-instruct) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space. |
| | Refer to the [original model card](https://huggingface.co/ibm-granite/granite-20b-code-instruct) for more details on the model. |
| | ## Use with llama.cpp |
| |
|
| | Install llama.cpp through brew. |
| |
|
| | ```bash |
| | brew install ggerganov/ggerganov/llama.cpp |
| | ``` |
| | Invoke the llama.cpp server or the CLI. |
| |
|
| | CLI: |
| |
|
| | ```bash |
| | llama-cli --hf-repo mayank-mishra/granite-20b-code-instruct-Q4_K_M-GGUF --model granite-20b-code-instruct.Q4_K_M.gguf -p "The meaning to life and the universe is" |
| | ``` |
| |
|
| | Server: |
| |
|
| | ```bash |
| | llama-server --hf-repo mayank-mishra/granite-20b-code-instruct-Q4_K_M-GGUF --model granite-20b-code-instruct.Q4_K_M.gguf -c 2048 |
| | ``` |
| |
|
| | Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well. |
| |
|
| | ``` |
| | git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m granite-20b-code-instruct.Q4_K_M.gguf -n 128 |
| | ``` |
| |
|
