Hugging Peers

cvescore-deberta

☆ Save
by angronain text-classification mit
0 downloads 1 likes on Hugging Face updated

Model card

CVEscore: CVSS v3.1 vector prediction from CVE descriptions

Fine-tuned microsoft/deberta-v3-base with 8 classification heads (AV, AC, PR, UI, S, C, I, A). The base score and severity are computed from the predicted vector with the official CVSS v3.1 formula.

Input format

Prefix the description with who will score the CVE, as in training:

  • [NVD] <description> predicts the vector NVD would give.
  • [CNA: <org>] <description> predicts the vector that CNA would give. <org> is the domain of the CNA's email without the TLD (security@wordfence.com -> wordfence), or cisa-adp for CISA's ADP.

Remove CVSS vectors and scores quoted in the description first (strip_cvss in src/cvss_text.py): the model was trained without them.

With every CVE prompted as [NVD], exact match is 0.393 overall (0.552 on NVD-scored CVEs). Tagging each CVE with its real scorer raises CNA-scored exact match from 0.339 to 0.438. Ablation: https://github.com/atomic-crafter/CVEscore/blob/master/results/ablation.md

Training data and split

NVD CVE JSON 2.0 feeds, CVEs with a CVSS v3.1 vector (NVD score preferred, CNA score otherwise). Time-based split: train on CVEs published before 2024-10-01, validate up to the end of 2024, test on CVEs published from 2025 onwards. No random split, so no leakage from the future.

Test results (123,459 CVEs published in 2025+, each tagged with its scorer)

Metric Value
Exact vector match 0.467
Severity accuracy 0.691
Base score MAE 0.860
Inference speed 1.01 ms/CVE (RTX 5080, batched)

Per-metric macro F1: AV 0.782, AC 0.707, PR 0.746, UI 0.903, S 0.851, C 0.819, I 0.819, A 0.796

Intended use

Triage aid: pre-score new CVEs before official scoring is available, and route low-confidence predictions (product of the 8 heads' top probabilities) to a human analyst. Not a replacement for analyst scoring.

Limitations

  • English descriptions only; vague descriptions ("unspecified vulnerability") are scored poorly.
  • NVD and vendor (CNA) scores disagree on most CVEs scored by both (18% exact agreement in 2025+), which caps achievable accuracy.
  • CVSS v3.1 only (no v4.0).

Usage

Code, training pipeline and the CVSSModel class: https://github.com/atomic-crafter/CVEscore

from model import CVSSModel, predict_proba   # src/model.py in the GitHub repo
model, tokenizer, config = CVSSModel.load("path/to/this/repo", device="cuda")