ReaderLM-v2 Q4_K_M for Jetson

This repository mirrors the tested GGUF Q4_K_M conversion by mradermacher, from https://huggingface.co/mradermacher/ReaderLM-v2-GGUF at revision 08a86568ae97ff37fbc22688ac919be29f5bca3e. Original model: Jina AI ReaderLM-v2. No weight bytes were changed.

The file SHA-256 is d5bf01392bc58fc49f15d9441b16969da388c6601b9c192d4a6a1d8f31be2512.

Tested on AGX Orin JetPack 6.2 with pinned llama.cpp 970b51ce7c9ba5fff0726c5e9994f1c2f0e4eedc: Q4_K_M weights, f16 KV, Flash Attention, CUDA Graph, 32K total input/output capacity, one slot, --cache-ram 0. A synthetic short HTML extraction decoded about 81 tokens/s. This is a warm workload measurement, not an accuracy guarantee or full-page throughput limit.

ReaderLM is specialized for HTML-to-Markdown and schema-guided JSON extraction. It is not a general-purpose chat recommendation. The deployed 32K limit is smaller than the upstream model maximum.

License / 许可证

CC-BY-NC-4.0. Preserve Jina AI attribution and the original license. 非商业许可；商业使用请向 Jina AI 获取授权。See NOTICE.md.