Hugging Peers

vulkan-matmul

☆ Save
by elyanlabs agpl-3.0
0 downloads 0 likes on Hugging Face updated

Model card

Source snapshot of github.com/Scottcjn/vulkan-matmul, also on GitLab with full history. Code only: nothing here runs on Hugging Face infrastructure.

vulkan-matmul

GPU matmul offload server prototypes for POWER8-oriented inference setups.

This repository contains two server paths:

  • a Vulkan server for AMD GPUs, with a POWER8-focused build flow
  • a ROCm / rocBLAS server for HIP-capable AMD GPUs

It also includes POWER8 support headers and small client / benchmark helpers for validating the offload path.

What Is Here

File Purpose
server.cpp Vulkan matmul server
rocm_matmul_server.cpp ROCm / rocBLAS matmul server
build_power8.sh Vulkan build helper for POWER8 systems
rocm_build.sh ROCm build helper for HIP-capable AMD GPUs
test_client.py basic client smoke test
vk_bench.py benchmark helper for the Vulkan server
powerpc/ POWER8 integration and support code
BCOS.md BCOS certification details

Quick Start

Vulkan path on POWER8

./build_power8.sh
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.ppc64le.json ./build/vulkan_matmul_server 8097
python3 test_client.py

ROCm path

./rocm_build.sh gfx1012
./rocm_matmul_server 8098

Then point your inference host at the ROCm server:

env GGML_GPU_OFFLOAD_ROCM=1 \
    GGML_ROCM_MATMUL_HOST=<server_ip> \
    GGML_ROCM_MATMUL_PORT=8098 \
    GGML_ROCM_MIN_M=2 \
    llama-cli -m model.gguf -ngl 0 -t 1 -p 'prompt' -n 20

Build Notes

  • CMakeLists.txt is used for the Vulkan server build
  • build_power8.sh installs the Vulkan-side build dependencies if needed
  • rocm_build.sh expects ROCm and hipcc under /opt/rocm unless ROCM_PATH is overridden
  • vk_bench.py assumes the Vulkan server is reachable on port 8097 by default

Related Docs

License

See the contribution guide for contribution licensing terms.