Hugging Peers

Llama-3.1-8B-Instruct-ROCMFP4_FAST

☆ Save
by alst10 cc-by-4.0
74 downloads 0 likes on Hugging Face updated

Model card

Llama-3.1-8B-Instruct-ROCMFP4_FAST - ROCmFPX Quantized

This model was dynamically quantized and generated using the ROCmFPX My Repo Space

Usage Instructions with ROCmFPX llama.cpp

To run this model, you must use the specialized ROCmFPX fork of llama.cpp by charlie12345. Standard llama.cpp releases do not currently support ROCmFPX quantization formats.

# 1. Clone the ROCmFPX repository
git clone --depth 1 [https://github.com/charlie12345/ROCmFPX.git](https://github.com/charlie12345/ROCmFPX.git)
cd ROCmFPX

# 2. Build the project
cmake -B build -G Ninja -DGGML_CUDA=OFF -DGGML_NATIVE=OFF
cmake --build build --target llama-cli -j

# 3. Run the model
./build/bin/llama-cli -m Llama-3.1-8B-Instruct-Q4_0_ROCMFP4_FAST.gguf -p "You are a helpful assistant. Hello!" -n 128

License

This model is distributed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Please adhere to the original license constraints of the base model and provide appropriate attribution when sharing.