# Alibaba's LLM Qwen2-0.5B for VOXL 2

Source: https://forum.modalai.com/topic/4373/alibaba-s-llm-qwen2-0-5b-for-voxl-2
Category: VOXL 2 and VOXL 2 Mini (https://forum.modalai.com/category/26/voxl-2-and-voxl-2-mini)
Posted: 2025-04-24 04:17:37 UTC by Suvasis M
Replies: 0 · Views: 1514

## Suvasis M · 2025-04-24 04:17:37 UTC

hi, 
Has anyone tried Qwen2-0.5B  for the VOXL 2?

Here's are some requirements and compatibility:
1. RAM Requirements for Qwen2-0.5B

>> Base model (FP16): ~1GB of RAM
>> Quantized to INT8: ~500MB of RAM
>> Quantized to INT4: ~250MB of RAM

VOXL 2 Specifications

>> The VOXL 2 has approximately 4GB of RAM available
>> It uses a Qualcomm QRB5165 processor
>> Has AI acceleration capabilities via the Hexagon DSP
>> Primarily designed for computer vision tasks

Compatibility Assessment
Qwen2-0.5B could potentially run on the VOXL 2 with quantization to INT4 or INT8, the memory footprint would be manageable.

>> Inference speed would likely be slow, perhaps 1-2 seconds per token

Question:

Has ModalAI explored using TensorFlow Lite with the Hexagon delegate to run compact language models like Qwen2-0.5B on the VOXL 2 platform? I'm interested in whether you've experimented with leveraging the Qualcomm GPU for language model inference, even though I understand the VOXL 2 is primarily optimized for computer vision workloads. Have you conducted any experiments or performance testing with small LLMs on this hardware?
Thanks.
suvasis