Editing Openai/694057b6-101c-8007-9a65-c40578c7252d (section)

==== What you get: ====
* High-level <code>Llama</code> class for local inference (text + chat-style prompting). GitHub<ref>{{cite web|title=GitHub|url=https://github.com/abetlen/llama-cpp-python|publisher=github.com|access-date=2025-12-16}}</ref>
* OpenAI-like API surface (so it feels like chat.completions, etc.). PyPI<ref>{{cite web|title=PyPI|url=https://pypi.org/project/llama-cpp-python/|publisher=pypi.org|access-date=2025-12-16}}</ref>
* Embeddings support. PyPI<ref>{{cite web|title=PyPI|url=https://pypi.org/project/llama-cpp-python/|publisher=pypi.org|access-date=2025-12-16}}</ref>
* Low-level access to llama.cpp’s C API via <code>ctypes</code> if you want finer control. PyPI<ref>{{cite web|title=PyPI|url=https://pypi.org/project/llama-cpp-python/|publisher=pypi.org|access-date=2025-12-16}}</ref>
* Integrations: works with LangChain and LlamaIndex. PyPI<ref>{{cite web|title=PyPI|url=https://pypi.org/project/llama-cpp-python/|publisher=pypi.org|access-date=2025-12-16}}</ref>
* Also advertised in the ecosystem/docs: function calling support and vision API support (when model/build supports it). PyPI<ref>{{cite web|title=PyPI|url=https://pypi.org/project/llama-cpp-python/|publisher=pypi.org|access-date=2025-12-16}}</ref>

Tiny example (in-process):

<syntaxhighlight lang="python">from llama_cpp import Llama

llm = Llama(model_path="model.gguf", n_ctx=4096)
out = llm("Write a haiku about GPUs.")
print(out["choices"][0]["text"])

</syntaxhighlight>

(Exact knobs vary by your build/model.)