path: root/docs/LoaderDebugging.md

<!-- markdownlint-disable MD041 -->
[![Khronos Vulkan][1]][2]

[1]: https://vulkan.lunarg.com/img/Vulkan_100px_Dec16.png "https://www.khronos.org/vulkan/"
[2]: https://www.khronos.org/vulkan/

# Debugging The Vulkan Desktop Loader
[![Creative Commons][3]][4]

<!-- Copyright &copy; 2015-2022 LunarG, Inc. -->

[3]: https://i.creativecommons.org/l/by-nd/4.0/88x31.png "Creative Commons License"
[4]: https://creativecommons.org/licenses/by-nd/4.0/
## Table of Contents

- [Debugging Issues](#debugging-issues)
- [Loader Logging](#loader-logging)
- [Debugging Possible Layer Issues](#debugging-possible-layer-issues)
  - [Enable Layer Logging](#enable-layer-logging)
  - [Disable Layers](#disable-layers)
  - [Selectively Re-enable Layers](#selectively-re-enable-layers)
- [Debugging Possible Driver Issues](#debugging-possible-driver-issues)
  - [Enable Driver Logging](#enable-driver-logging)
  - [Selectively Enable Specific Drivers](#selectively-enable-specific-drivers)

## Debugging Issues

If your application is crashing or behaving weirdly, the loader provides
several mechanisms for you to debug the issues.

**NOTE**: This functionality is all specific to the desktop Vulkan loader and
does not work for the Android loader.

## Loader Logging

The Vulkan desktop loader has added logging functionality that can be enabled by
using the `VK_LOADER_DEBUG` environment variable.
The results will be output to the standard output, but will also be passed to
any `VK_EXT_debug_utils` messengers present as well.
The variable can be set to a comma-delimited list of debug level options which
include:

  * error&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Report any errors encountered by
    the loader
  * warn&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Report any warnings encountered by
    the loader
  * info&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Report info-level
    messages generated by the loader
  * debug&nbsp;&nbsp;&nbsp;&nbsp;Report debug-level messages generated by the
    loader
  * layer&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Report all layer-specific messages
    generated by the loader
  * driver&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Report all driver-specific messages
    generated by the loader
  * all&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Report all
    messages generated by the loader (includes all of the above)

If you're not sure where the issue comes from, at least set it to output all
messages through the "info" level:

```
set VK_LOADER_DEBUG=error,warn,info
```

Then, you can search the list for any errors or warnings that might provide a
hint at why you're seeing issues.

For more info on enabling loader logging, refer to the
[Enable Loader Debug Layer Output](LoaderApplicationInterface.md#enable-loader-debug-layer-output)
and the
[Table of Debug Environment Variables](LoaderInterfaceArchitecture.md#table-of-debug-environment-variables)
below.

## Debugging Possible Layer Issues

### Enable Layer Logging

If you suspect a layer issue, set the loader logging to specifically output
layer messages in addition to warnings and errors:

```
set VK_LOADER_DEBUG=error,warn,layer
```

Most important layer messages should go out with error or warning levels set,
but this will provide more layer-specific info as well such as:
  * What layers are found
  * Where they were found
  * If they are implicit, what environment variables can be used to disable them
  * If there is any incompatibility with a given layer, this could include:
    * The layer library file (.so/.dll) wasn't found
    * The layer library is the wrong bit-depth for the executing application
      (i.e. 32-bit vs 64-bit)
    * The layer itself doesn't support the application desired version of Vulkan
  * If any environment variables are disabling any layers

For example, the output of the loader looking for implicit layers may look like
the following:

```
LAYER: Searching for layer manifest files
LAYER:  In following folders:
LAYER:   /home/${USER}/.config/vulkan/implicit_layer.d
LAYER:   /etc/xdg/vulkan/implicit_layer.d
LAYER:   /usr/local/etc/vulkan/implicit_layer.d
LAYER:   /etc/vulkan/implicit_layer.d
LAYER:   /home/${USER}/.local/share/vulkan/implicit_layer.d
LAYER:   /home/${USER}/.local/share/flatpak/exports/share/vulkan/implicit_layer.d
LAYER:   /var/lib/flatpak/exports/share/vulkan/implicit_layer.d
LAYER:   /usr/local/share/vulkan/implicit_layer.d
LAYER:   /usr/share/vulkan/implicit_layer.d
LAYER:  Found the following files:
LAYER:   /home/${USER}/.local/share/vulkan/implicit_layer.d/renderdoc_capture.json
LAYER:   /home/${USER}/.local/share/vulkan/implicit_layer.d/steamfossilize_i386.json
LAYER:   /home/${USER}/.local/share/vulkan/implicit_layer.d/steamfossilize_x86_64.json
LAYER:   /home/${USER}/.local/share/vulkan/implicit_layer.d/steamoverlay_i386.json
LAYER:   /home/${USER}/.local/share/vulkan/implicit_layer.d/steamoverlay_x86_64.json
LAYER:   /usr/share/vulkan/implicit_layer.d/nvidia_layers.json
LAYER:   /usr/share/vulkan/implicit_layer.d/VkLayer_MESA_device_select.json
```

Then, the loading of layer libraries is reported similar to this:

```
LAYER | DEBUG: Loading layer library libVkLayer_khronos_validation.so
LAYER | INFO: Insert instance layer VK_LAYER_KHRONOS_validation (libVkLayer_khronos_validation.so)
LAYER | DEBUG: Loading layer library libVkLayer_MESA_device_select.so
LAYER | INFO: Insert instance layer VK_LAYER_MESA_device_select (libVkLayer_MESA_device_select.so)
```

Finally, when the Vulkan instance is created, you can see the full instance
call-chain from a functional standpoint with output like this:

```
LAYER: vkCreateInstance layer callstack setup to:
LAYER:  <Application>
LAYER:    ||
LAYER:  <Loader>
LAYER:    ||
LAYER:  VK_LAYER_MESA_device_select
LAYER:      Type: Implicit
LAYER:         Disable Env Var:  NODEVICE_SELECT
LAYER:      Manifest: /usr/share/vulkan/implicit_layer.d/VkLayer_MESA_device_select.json
LAYER:      Library:  libVkLayer_MESA_device_select.so
LAYER:    ||
LAYER:  VK_LAYER_KHRONOS_validation
LAYER:      Type: Explicit
LAYER:      Manifest: /usr/share/vulkan/explicit_layer.d/VkLayer_khronos_validation.json
LAYER:      Library:  libVkLayer_khronos_validation.so
LAYER:    ||
LAYER:  <Drivers>
```

In this scenario, two layers were used (the same two that were loaded earlier):
* `VK_LAYER_MESA_device_select`
* `VK_LAYER_KHRONOS_validation`

This information now shows us that the `VK_LAYER_MESA_device_select` is loaded
first, followed by `VK_LAYER_KHRONOS_validation` which will then continue into
any available drivers.
It also shows that `VK_LAYER_MESA_device_select` is an implicit layer which
implies that it wasn't directly enabled by the application.
On the other hand, `VK_LAYER_KHRONOS_validation` is shown as an explicit layer
which indicates that it was likely enabled by the application.

### Disable Layers

**NOTE:** This functionality is only available with Loaders built with version
1.3.yyyy of the Vulkan headers and later.

Sometimes, implicit layers can cause issues with an application.
Because of this, the next step is to try to disable one or more of the listed
implicit layers.
You can use the filtering environment variables
(`VK_LOADER_LAYERS_ENABLE` and `VK_LOADER_LAYERS_DISABLE`) to selectively enable
or disable various layers.
If you're not sure what to do, try disabling all implicit layers manually by
setting `VK_LOADER_LAYERS_DISABLE` to '~implicit~'.

```
  set VK_LOADER_LAYERS_DISABLE=~implicit~
```

This will disable all implicit layers and the loader will report any disabled
layers to the logging output when layer logging is enabled in the following way:

```
WARNING | LAYER:  Implicit layer "VK_LAYER_MESA_device_select" forced disabled because name matches filter of env var 'VK_LOADER_LAYERS_DISABLE'.
WARNING | LAYER:  Implicit layer "VK_LAYER_AMD_switchable_graphics_64" forced disabled because name matches filter of env var 'VK_LOADER_LAYERS_DISABLE'.
WARNING | LAYER:  Implicit layer "VK_LAYER_Twitch_Overlay" forced disabled because name matches filter of env var 'VK_LOADER_LAYERS_DISABLE'.
```

### Selectively Re-enable Layers

**NOTE:** This functionality is only available with Loaders built with version
1.3.yyyy of the Vulkan headers and later.

When trying to diagnose problems caused by layers, it is useful to first disable
all layers and re-enable each layer individually.
If the problem reappears, then it is immediately clear which layer is the source
of the issue.

For example, from the above given list of disabled layers, let’s selectively
re-enable one:

```
set VK_LOADER_LAYERS_DISABLE=~implicit~
set VK_LOADER_LAYERS_ENABLE=*AMD*
```

This would keep both the "VK_LAYER_MESA_device_select" and
"VK_LAYER_Twitch_Overlay" layers disabled, while enabling the
"VK_LAYER_AMD_switchable_graphics_64" layer.
If everything continues to work, then the evidence seems to suggest the issue is
likely not related to the AMD layer.
This would lead to enabling one other layer and trying again:

```
set VK_LOADER_LAYERS_DISABLE=~implicit~
set VK_LOADER_LAYERS_ENABLE=*AMD*,*twitch*
```

And so forth.

For more info on how to use the filtering environment variables, refer to the
[Layer Filtering](LoaderLayerInterface.md#layer-filtering) section of the
[LoaderLayerInterface](LoaderLayerInterface.md) document.


## Debugging Possible Driver Issues

### Enable Driver Logging

**NOTE:** This functionality is only available with Loaders built with version
1.3.yyyy of the Vulkan headers and later.

If you suspect a driver issue, set the loader logging to specifically output
driver messages:

```
set VK_LOADER_DEBUG=error,warn,driver
```

Most important driver messages should go out with error or warning levels set,
but this will provide more driver-specific info as well such as:
  * What drivers are found
  * Where they were found
  * If there is any incompatibility with a given driver
  * If any environment variables are disabling any of the drivers

For example, the output of the loader looking for drivers on a Linux system may
look like the following (NOTE: additional spaces have been removed from the
output for easier reading):

```
DRIVER: Searching for driver manifest files
DRIVER:    In following folders:
DRIVER:       /home/$(USER)/.config/vulkan/icd.d
DRIVER:       /etc/xdg/vulkan/icd.d
DRIVER:       /etc/vulkan/icd.d
DRIVER:       /home/$(USER)/.local/share/vulkan/icd.d
DRIVER:       /home/$(USER)/.local/share/flatpak/exports/share/vulkan/icd.d
DRIVER:       /var/lib/flatpak/exports/share/vulkan/icd.d
DRIVER:       /usr/local/share/vulkan/icd.d
DRIVER:       /usr/share/vulkan/icd.d
DRIVER:    Found the following files:
DRIVER:       /usr/share/vulkan/icd.d/intel_icd.x86_64.json
DRIVER:       /usr/share/vulkan/icd.d/lvp_icd.x86_64.json
DRIVER:       /usr/share/vulkan/icd.d/radeon_icd.x86_64.json
DRIVER:       /usr/share/vulkan/icd.d/lvp_icd.i686.json
DRIVER:       /usr/share/vulkan/icd.d/radeon_icd.i686.json
DRIVER:       /usr/share/vulkan/icd.d/intel_icd.i686.json
DRIVER:       /usr/share/vulkan/icd.d/nvidia_icd.json
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/intel_icd.x86_64.json, version "1.0.0"
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/lvp_icd.x86_64.json, version "1.0.0"
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/radeon_icd.x86_64.json, version "1.0.0"
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/lvp_icd.i686.json, version "1.0.0"
DRIVER: Requested driver /usr/lib/libvulkan_lvp.so was wrong bit-type. Ignoring this JSON
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/radeon_icd.i686.json, version "1.0.0"
DRIVER: Requested driver /usr/lib/libvulkan_radeon.so was wrong bit-type. Ignoring this JSON
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/intel_icd.i686.json, version "1.0.0"
DRIVER: Requested driver /usr/lib/libvulkan_intel.so was wrong bit-type. Ignoring this JSON
DRIVER: Found ICD manifest file /usr/share/vulkan/icd.d/nvidia_icd.json, version "1.0.0"
```

Then when the application selects the device to use, you will see the Vulkan
device call chain reported in the following way (NOTE: additional spaces have
been removed from the output for easier reading):

```
DRIVER: vkCreateDevice layer callstack setup to:
DRIVER:    <Application>
DRIVER:      ||
DRIVER:    <Loader>
DRIVER:      ||
DRIVER:    <Device>
DRIVER:        Using "Intel(R) UHD Graphics 630 (CFL GT2)" with driver: "/usr/lib64/libvulkan_intel.so"
```


### Selectively Enable Specific Drivers

**NOTE:** This functionality is only available with Loaders built with version
1.3.yyyy of the Vulkan headers and later.

You can now use the filtering environment variables
(`VK_LOADER_DRIVERS_SELECT` and `VK_LOADER_DRIVERS_DISABLE`) to control what
drivers the loader will attempt to load.
For drivers, the string globs passed into the above environment variables will
be compared against the driver JSON file name since there is no driver name
known to the loader until much later in the Vulkan initialization process.

For example, to disable all drivers except Nvidia you could do the following:

```
set VK_LOADER_DRIVERS_DISABLE=*
set VK_LOADER_DRIVERS_SELECT=*nvidia*
```

The loader outputs messages like the following when the environment variables
are used:

```
WARNING | DRIVER: Driver "intel_icd.x86_64.json" ignored because not selected by env var 'VK_LOADER_DRIVERS_SELECT'
WARNING | DRIVER: Driver "radeon_icd.x86_64.json" ignored because it was disabled by env var 'VK_LOADER_DRIVERS_DISABLE'
```

For more info on how to use the filtering environment variables, refer to the
[Driver Filtering](LoaderDriverInterface.md#driver-filtering) section of the
[LoaderDriverInterface](LoaderDriverInterface.md) document.