Instructions to use vinamrag-code/llama-cpp-python-cuda-wheels with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use vinamrag-code/llama-cpp-python-cuda-wheels with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="vinamrag-code/llama-cpp-python-cuda-wheels", filename="{{GGUF_FILE}}", )output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
File size: 134 Bytes
a6acce9 | 1 2 3 4 | version https://git-lfs.github.com/spec/v1
oid sha256:162d81a2a9458dc3fdd6fbcb0040485f1743336c19720e0f87112e60d1e6105e
size 271311101
|