📖 About This Model

This model is google/functiongemma-270m-it converted to GGUF format for use with llama.cpp, Ollama, LM Studio, and other compatible inference engines.

🚀 Quick Start

Option 1: Python (llama-cpp-python)

from llama_cpp import Llama

# Load the model
llm = Llama.from_pretrained(
    repo_id="QuantLLM/functiongemma-270m-it-4bit-gguf",
    filename="functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf",
)

# Generate text
output = llm(
    "Write a short story about a robot learning to paint:",
    max_tokens=256,
    echo=True
)
print(output["choices"][0]["text"])

Option 2: Ollama

# Download the model
huggingface-cli download QuantLLM/functiongemma-270m-it-4bit-gguf functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf --local-dir .

# Create Modelfile
echo 'FROM ./functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf' > Modelfile

# Import to Ollama
ollama create functiongemma-270m-it-4bit-gguf -f Modelfile

# Chat with the model
ollama run functiongemma-270m-it-4bit-gguf

Option 3: LM Studio

1. Download the .gguf file from the Files tab above
2. Open LM Studio → My Models → Add Model
3. Select the downloaded file
4. Start chatting!

Option 4: llama.cpp CLI

# Download
huggingface-cli download QuantLLM/functiongemma-270m-it-4bit-gguf functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf --local-dir .

# Run inference
./llama-cli -m functiongemma-270m-it-4bit-gguf.Q4_K_M.gguf -p "Hello! " -n 128

📊 Model Details

📦 Quantization Details

This model uses Q4_K_M quantization:

All Available GGUF Quantizations

🚀 Created with QuantLLM

Convert any model to GGUF, ONNX, or MLX in one line!

from quantllm import turbo

# Load any HuggingFace model
model = turbo("google/functiongemma-270m-it")

# Export to any format
model.export("gguf", quantization="Q4_K_M")

# Push to HuggingFace
model.push("your-repo", format="gguf")

📚 Documentation · 🐛 Report Issue · 💡 Request Feature