Hugging Peers

Distil-Whisper

☆ Save
by Qualcomm-AI-Hub mit
0 downloads 0 likes on ModelScope updated

Model card

Distil-Whisper: Optimized for Qualcomm Devices

Distil-Whisper Small English is a distilled version of Whisper Small, optimized for fast and efficient automatic speech recognition.

This is based on the implementation of Distil-Whisper found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.45, ONNX Runtime 1.27.1 Download
QNN_DLC float Universal QAIRT 2.45 Download
TFLITE float Universal Download

For more device-specific assets and performance metrics, visit Distil-Whisper on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Distil-Whisper on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.speech_recognition

Model Stats:

  • Input resolution: 80x3000 (30 seconds audio)
  • Max decoded sequence length: 200 tokens
  • Model checkpoint: distil-whisper/distil-small.en
  • Model size (decoder) (float): 450MB
  • Model size (encoder) (float): 332 MB
  • Number of parameters (decoder): 211M
  • Number of parameters (encoder): 166M

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder ONNX float Snapdragon® X2 Elite 5.276 ms 40 - 40 MB NPU
decoder ONNX float Snapdragon® X Elite 10.889 ms 179 - 179 MB NPU
decoder ONNX float Snapdragon® 8 Gen 3 Mobile 8.876 ms 2 - 491 MB NPU
decoder ONNX float Snapdragon® 8 Gen 1 Mobile 18.272 ms 32 - 383 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ IQ-8275 13.158 ms 40 - 83 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 11.708 ms 19 - 40 MB NPU
decoder ONNX float Qualcomm® QCS8450 18.272 ms 32 - 383 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ IQ-9075 12.824 ms 40 - 82 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ IQ-X7181 10.889 ms 179 - 179 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ Q-8750 7.306 ms 16 - 572 MB NPU
decoder ONNX float Snapdragon® 8 Elite Mobile 7.306 ms 16 - 572 MB NPU
decoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 5.726 ms 12 - 556 MB NPU
decoder QNN_DLC float Snapdragon® X2 Elite 5.991 ms 40 - 40 MB NPU
decoder QNN_DLC float Snapdragon® X Elite 11.331 ms 40 - 40 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 8.717 ms 40 - 642 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 18.251 ms 28 - 330 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 12.863 ms 40 - 87 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 11.56 ms 40 - 42 MB NPU
decoder QNN_DLC float Qualcomm® SA8775P 12.874 ms 19 - 511 MB NPU
decoder QNN_DLC float Qualcomm® SA8650P 12.874 ms 19 - 511 MB NPU
decoder QNN_DLC float Qualcomm® SA8255P 12.874 ms 19 - 511 MB NPU
decoder QNN_DLC float Qualcomm® QCS8450 18.251 ms 28 - 330 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 12.693 ms 40 - 86 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ IQ-X7181 11.331 ms 40 - 40 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 7.303 ms 0 - 543 MB NPU
decoder QNN_DLC float Qualcomm® SA7255P 19.239 ms 27 - 524 MB NPU
decoder QNN_DLC float Qualcomm® SA8295P 14.051 ms 19 - 261 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Mobile 7.303 ms 0 - 543 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 5.919 ms 4 - 505 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 3 Mobile 8.584 ms 4 - 747 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 1 Mobile 18.207 ms 5 - 465 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ IQ-8275 12.94 ms 0 - 266 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 11.661 ms 5 - 7 MB NPU
decoder TFLITE float Qualcomm® QCS8450 18.207 ms 5 - 465 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ IQ-9075 12.91 ms 0 - 265 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ Q-8750 7.226 ms 4 - 573 MB NPU
decoder TFLITE float Qualcomm® SA7255P 19.195 ms 5 - 538 MB NPU
decoder TFLITE float Qualcomm® SA8295P 13.913 ms 5 - 297 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Mobile 7.226 ms 4 - 573 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 5.834 ms 4 - 575 MB NPU
encoder ONNX float Snapdragon® X2 Elite 60.166 ms 133 - 133 MB NPU
encoder ONNX float Snapdragon® X Elite 133.451 ms 238 - 238 MB NPU
encoder ONNX float Snapdragon® 8 Gen 3 Mobile 97.357 ms 80 - 1160 MB NPU
encoder ONNX float Snapdragon® 8 Gen 1 Mobile 268.749 ms 72 - 1015 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ IQ-8275 162.058 ms 74 - 79 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 129.181 ms 0 - 258 MB NPU
encoder ONNX float Qualcomm® QCS8450 268.749 ms 72 - 1015 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ IQ-9075 154.309 ms 78 - 82 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ IQ-X7181 133.451 ms 238 - 238 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ Q-8750 71.537 ms 77 - 786 MB NPU
encoder ONNX float Snapdragon® 8 Elite Mobile 71.537 ms 77 - 786 MB NPU
encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 58.002 ms 47 - 781 MB NPU
encoder QNN_DLC float Snapdragon® X2 Elite 71.597 ms 1 - 1 MB NPU
encoder QNN_DLC float Snapdragon® X Elite 139.313 ms 1 - 1 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 96.446 ms 0 - 963 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 269.376 ms 0 - 822 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 161.46 ms 1 - 39 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 135.577 ms 0 - 883 MB NPU
encoder QNN_DLC float Qualcomm® SA8775P 153.553 ms 1 - 687 MB NPU
encoder QNN_DLC float Qualcomm® SA8650P 153.553 ms 1 - 687 MB NPU
encoder QNN_DLC float Qualcomm® SA8255P 153.553 ms 1 - 687 MB NPU
encoder QNN_DLC float Qualcomm® QCS8450 269.376 ms 0 - 822 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 153.958 ms 1 - 39 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-X7181 139.313 ms 1 - 1 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 71.034 ms 1 - 690 MB NPU
encoder QNN_DLC float Qualcomm® SA7255P 437.78 ms 1 - 696 MB NPU
encoder QNN_DLC float Qualcomm® SA8295P 193.158 ms 1 - 610 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Mobile 71.034 ms 1 - 690 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 57.856 ms 0 - 717 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 476.714 ms 40 - 186 MB GPU
encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 842.466 ms 41 - 194 MB GPU
encoder TFLITE float Qualcomm® Dragonwing™ IQ-8275 2986.821 ms 0 - 39 MB GPU
encoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 654.625 ms 0 - 317 MB GPU
encoder TFLITE float Qualcomm® QCS8450 842.466 ms 41 - 194 MB GPU
encoder TFLITE float Qualcomm® Dragonwing™ IQ-9075 1303.187 ms 0 - 39 MB GPU
encoder TFLITE float Qualcomm® Dragonwing™ Q-8750 410.329 ms 24 - 62 MB GPU
encoder TFLITE float Qualcomm® SA7255P 3120.053 ms 38 - 83 MB GPU
encoder TFLITE float Qualcomm® SA8295P 668.731 ms 38 - 82 MB GPU
encoder TFLITE float Snapdragon® 8 Elite Mobile 410.329 ms 24 - 62 MB GPU
encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 367.016 ms 21 - 59 MB GPU

License

  • The license for the original implementation of Distil-Whisper can be found here.

References

Community