Report on real machine with AMD Ryzen 5 and AMD Radeon GPU (6600)

Machine spec

Performance
==> Summary: AMD GPU consistently yield 300% better performance compared to AMD Ryzen CPU (which is really good already)

• Tiny Llama 1.1B

  • AMD GPU
    
  • AMD Ryzen CPU
    

• Mistral 7B Q4

  • AMD GPU
    
  • AMD Ryzen CPU
    

Updated:

• vulkan enabled: should be a boolean value located in the root of settings.json.
• gpus_in_use: reuse, and it should be shared between Cuda & Vulkan settings while resetting when switching between them.
• Do not use the VULKAN_DEVICE_INDEX variable; instead, use GGML_VULKAN_DEVICE. Only support using one selected GPU at a time, and both Nvidia and AMD GPUs should work.
• Handle crash on app launch due to undefined value (settings.json > vulkan)
• nvidia-smi yield error when vulkan enabled (wrong device index)
• Wrong Linux vulkan executable binary download url

In progress:

• GPU devices list

Update:

• GPU devices list

NOTES: There is a limitation with the current approach; GPU VRAM information is not retrieved. Will investigate further.

The screenshots are for demo purposes; please see the AMD performance above.

Known issue:
Enable Vulkan support on unsupported GPU device would cause this error. The GPU list would be empty in App > Settings.

Experimental feature: 0.4.7