godot: [Tracker] Vulkan error unable to create swapchain (device lost) happening randomly
Godot version
4.0 beta14 and earlier and likely later
System information
Most users are on Windows, some on Linux, variety of GPUs (intel, nVidia and AMD)
Issue description
I am opening this tracker issue as a central place to track a bug that seems to be plaguing a variety of users. The bug occurs quite randomly, but appears to be most often connecting with complex shaders and/or updating uniforms.
The bug leads to a freeze or crash when updating swap buffers along with the following errors:
E 0:00:01:0015 VulkanContext::swap_buffers: Condition "err" is true. Returning: ERR_CANT_CREATE
<C++ Source> drivers\vulkan\vulkan_context.cpp:2299 @ VulkanContext::swap_buffers()
E 0:00:01:0016 VulkanContext::prepare_buffers: Vulkan: Did not create swapchain successfully.
<C++ Error> Condition "err != VK_SUCCESS" is true. Breaking.
<C++ Source> drivers\vulkan\vulkan_context.cpp:2222 @ VulkanContext::prepare_buffers()
For some users these errors spam the console while the engine itself is frozen.
This issue is very difficult to debug as it is near impossible to reproduce reliably for people other than the affected users.
- [ ] https://github.com/godotengine/godot/issues/71453
- [ ] https://github.com/godotengine/godot/issues/70946
- [ ] https://github.com/godotengine/godot/issues/69855
- [ ] https://github.com/godotengine/godot/issues/69046 (may not be connected as the issue went away with a driver update)
- [ ] https://github.com/godotengine/godot/issues/67404
- [ ] https://github.com/godotengine/godot/issues/59699
- [ ] https://github.com/godotengine/godot/issues/71841
- [ ] https://github.com/godotengine/godot/issues/60755
- [ ] https://github.com/godotengine/godot/issues/60979
- [ ] https://github.com/godotengine/godot/issues/52112
- [ ] https://github.com/godotengine/godot/issues/56856
- [ ] https://github.com/godotengine/godot/issues/74499
- [ ] https://github.com/godotengine/godot/issues/74554
- [ ] https://github.com/godotengine/godot/issues/74700
- [ ] https://github.com/godotengine/godot/issues/75659
Steps to reproduce
Steps to reproduce can be found in individual reports
Minimal reproduction project
See individual reports
About this issue
- Original URL
- State: open
- Created a year ago
- Reactions: 15
- Comments: 137 (47 by maintainers)
Commits related to this issue
- Fix massive validation errors when enabling TAA + MSAA TAA + MSAA would make Godot request unnecessary flags for an MSAA velocity texture. flags that were not even actually needed. This was causing:... — committed to darksylinc/godot by darksylinc 9 months ago
- Fix massive validation errors when enabling TAA + MSAA TAA + MSAA would make Godot request unnecessary flags for an MSAA velocity texture. flags that were not even actually needed. This was causing:... — committed to darksylinc/godot by darksylinc 9 months ago
- Fix massive validation errors when enabling TAA + MSAA TAA + MSAA would make Godot request unnecessary flags for an MSAA velocity texture. flags that were not even actually needed. This was causing:... — committed to SaNeOr/godot by darksylinc 9 months ago
- Fix massive validation errors when enabling TAA + MSAA TAA + MSAA would make Godot request unnecessary flags for an MSAA velocity texture. flags that were not even actually needed. This was causing:... — committed to Occalepsus/godot by darksylinc 9 months ago
- Fix massive validation errors when enabling TAA + MSAA TAA + MSAA would make Godot request unnecessary flags for an MSAA velocity texture. flags that were not even actually needed. This was causing:... — committed to mandryskowski/godot by darksylinc 9 months ago
- Fix massive validation errors when enabling TAA + MSAA TAA + MSAA would make Godot request unnecessary flags for an MSAA velocity texture. flags that were not even actually needed. This was causing:... — committed to YuriSizov/godot by darksylinc 9 months ago
At this point I am becoming increasingly confident that the issue comes from improper synchronization in the RenderingDevice. It seems like the issue is more common on weaker hardware which would make GPU tasks take longer. This would also explain why for some users the issue only happens when they add slightly more complexity to a particular shader.
We probably need to do a careful audit of how we use synchronization primitives in the RenderingDevice to ensure that nothing is slipping through
We’ve made a build of Halls of Torment using Godot v4.2.dev4 and released it on an experimental opt-in branch on Steam. We’ve asked the folks who have encountered these issues to keep us posted.
Sorry it took so long for us to do this. A lot of things have happened at once lately preventing us to do this earlier.
This is what I was looking for! This error tells us that we are writing to the swapchain image after handing control of the image to Vulkan. I’ll see if I can pinpoint why this is happening
Should we force the engine to close in this situation, rather than trying to recreate the swapchain endlessly? As far as I know, there is no way to recover from this situation (not until the renderer can be restarted at run-time).
Also, the error message should be tweaked to indicate most likely causes (bugs in graphics drivers).
If this helps,
Turning
Single Window Modeon, seemed to have resolved the issue. I haven’t had it crash at all for over 8 hours now.To turn it on:
Editor Settings -> Interface -> Editor -> Single Window Mode. If this works for others, the bug can be narrowed down by a lot.I compiled the Acyclic Command Graph PR #84976, and for me it fixes the problem, I no longer have a crash when opening an empty project, and the editor seems to work fine.
So far it is working perfectly. 5 minutes into a game that normally crashes after the first few seconds so I think you did it
We have a potential fix for this issue!
Could those of you who can reproduce this issue reliably please test out https://github.com/godotengine/godot/pull/80566 and update us with your findings?
Cc @haraldwer @pLawitzki
I think I have found a way to reproduce this issue. I run the MRP from https://github.com/godotengine/godot/issues/71929#issuecomment-1526795518. Then I unplug my laptop from power, close the editor, restart godot, and then it crashes while loading the project. At this point it seems fairly reproducible, but time will tell.
Windows 10, Intel XE graphics.
I had to do dual boot windows onto my linux laptop to do this, so it will take some time before I have the development environment set up enough to properly work on investigating further.
As mentioned in #60755, I can help as I have 100% reproducible case (summary: launch editor, put computer to sleep, wake it up later, try to use any UI component => I got a “winner”)
Feel free to give me some debug code to run, patched version of Godot with additional debug code, anything which could help. I’m still not convinced it could solve all cases users have encountered with all OSes & hardware involved but it could narrow it down at least.
For the record I have some older version of Visual Studio Community (latest is 2019) installed and never tried yet to compiled Godot from source, but I may give it a try if it could help.
I have this issue every time I launch my game on my laptop. What is somewhat reassuring is that I don’t have the issue in the 4.3 dev5 snapshot. I hope it’s stable soon, try it on your projects!
Also probably worth nothing both the graph and the synchronization fixes are part of
masternow, so it’s probably worth just tryingmasterand see what you get.Left my computer running whille I was off to clean to make sure and 2,180,000 frames later at about 160fps (so about 4 hours) and we’re still going strong. safe to safe this isssue has been completely resolved for the A770.
#81775 seems to work perfectly 👍
Just for the record, the issue not longer happened to me (yet, even using 3D scene) on Windows 10, Nvidia driver 425.31 and Godot v4.2.dev3.official [013e8e3af].
I followed your instructions, here are my findings:
First test:
scons platform=windows target=editor dev_build=yes dev_mode=yes verbose=false warnings=allgodot.windows.editor.dev.x86_64.exe --gpu-validationVulkan Configurator running Did not crash 😕 4 attempts for a total of ca 30 min with the editor running Log was scrambledUnrelated but interesting things in the log:
WARNING: VALIDATION - Message Id Number: 1810669668 | Message Id Name: UNASSIGNED-BestPractices-vkCmdBeginRenderPass-ClearValueWithoutLoadOpClear Validation Warning: [ UNASSIGNED-BestPractices-vkCmdBeginRenderPass-ClearValueWithoutLoadOpClear ] Object 0: handle = 0x27dea44ff90, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x6beca064 | This render pass does not have VkRenderPassCreateInfo.pAttachments->loadOp == VK_ATTACHMENT_LOAD_OP_CLEAR but VkRenderPassBeginInfo.clearValueCount > 0. VkRenderPassBeginInfo.pClearValues will be ignored and no attachments will be cleared. Objects - 1 Object[0] - VK_OBJECT_TYPE_DEVICE, Handle 2739824557968 at: VulkanContext::_debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:264)
ERROR: VALIDATION - Message Id Number: 929810911 | Message Id Name: SYNC-HAZARD-WRITE-AFTER-READ Validation Error: [ SYNC-HAZARD-WRITE-AFTER-READ ] Object 0: handle = 0x89f8ef0000001071, name = VoxelGI LightProbe Data, type = VK_OBJECT_TYPE_IMAGE; | MessageID = 0x376bc9df | vkCmdPipelineBarrier: Hazard WRITE_AFTER_READ for image barrier 0 VkImage 0x89f8ef0000001071[VoxelGI LightProbe Data]. Access info (usage: SYNC_IMAGE_LAYOUT_TRANSITION, prior_usage: SYNC_COMPUTE_SHADER_SHADER_SAMPLED_READ, read_barriers: VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT|VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT, command: vkCmdDispatchIndirect, seq_no: 211, reset_no: 1). Objects - 1 Object[0] - VK_OBJECT_TYPE_IMAGE, Handle -8504785113010139023, Name “VoxelGI LightProbe Data” at: VulkanContext::_debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:267)
ERROR: VALIDATION - Message Id Number: 929810911 | Message Id Name: SYNC-HAZARD-WRITE-AFTER-READ Validation Error: [ SYNC-HAZARD-WRITE-AFTER-READ ] Object 0: handle = 0x4156c7000000119c, type = VK_OBJECT_TYPE_RENDER_PASS; | MessageID = 0x376bc9df | vkCmdBeginRenderPass: Hazard WRITE_AFTER_READ in subpass 0 for attachment 2 image layout transition (old_layout: VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, new_layout: VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL). Access info (usage: SYNC_IMAGE_LAYOUT_TRANSITION, prior_usage: SYNC_COMPUTE_SHADER_SHADER_SAMPLED_READ, read_barriers: VK_PIPELINE_STAGE_2_DRAW_INDIRECT_BIT|VK_PIPELINE_STAGE_2_VERTEX_SHADER_BIT|VK_PIPELINE_STAGE_2_TESSELLATION_CONTROL_SHADER_BIT|VK_PIPELINE_STAGE_2_TESSELLATION_EVALUATION_SHADER_BIT|VK_PIPELINE_STAGE_2_GEOMETRY_SHADER_BIT|VK_PIPELINE_STAGE_2_FRAGMENT_SHADER_BIT|VK_PIPELINE_STAGE_2_EARLY_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_LATE_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_2_COLOR_ATTACHMENT_OUTPUT_BIT|VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT|VK_PIPELINE_STAGE_2_BOTTOM_OF_PIPE_BIT|VK_PIPELINE_STAGE_2_TASK_SHADER_BIT_EXT|VK_PIPELINE_STAGE_2_MESH_SHADER_BIT_EXT|VK_PIPELINE_STAGE_2_RAY_TRACING_SHADER_BIT_KHR|VK_PIPELINE_STAGE_2_FRAGMENT_SHADING_RATE_ATTACHMENT_BIT_KHR|VK_PIPELINE_STAGE_2_TRANSFORM_FEEDBACK_BIT_EXT|VK_PIPELINE_STAGE_2_COPY_BIT|VK_PIPELINE_STAGE_2_RESOLVE_BIT|VK_PIPELINE_STAGE_2_BLIT_BIT|VK_PIPELINE_STAGE_2_CLEAR_BIT|VK_PIPELINE_STAGE_2_INDEX_INPUT_BIT|VK_PIPELINE_STAGE_2_VERTEX_ATTRIBUTE_INPUT_BIT, command: vkCmdDispatch, seq_no: 617, reset_no: 1). Objects - 1 Object[0] - VK_OBJECT_TYPE_RENDER_PASS, Handle 4708169263276364188 at: VulkanContext::_debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:267)
ERROR: VALIDATION - Message Id Number: -1539028524 | Message Id Name: VUID-vkCmdDrawIndexed-None-02699 Validation Error: [ VUID-vkCmdDrawIndexed-None-02699 ] Object 0: handle = 0x5ed5200000000fcc, name = RID:8010114007066, type = VK_OBJECT_TYPE_DESCRIPTOR_SET; | MessageID = 0xa44449d4 | vkCmdDrawIndexed: Descriptor set VkDescriptorSet 0x5ed5200000000fcc[RID:8010114007066] Image layout specified at vkCmdBindDescriptorSets time doesn’t match actual image layout at time descriptor is used. See previous error callback for specific details. The Vulkan spec states: Descriptors in each bound descriptor set, specified via vkCmdBindDescriptorSets, must be valid as described by descriptor validity if they are statically used by a bound shader (https://vulkan.lunarg.com/doc/view/1.3.250.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdDrawIndexed-None-02699) Objects - 1 Object[0] - VK_OBJECT_TYPE_DESCRIPTOR_SET, Handle 6833403193976688588, Name “RID:8010114007066” at: VulkanContext::_debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:267)
ERROR: VALIDATION - Message Id Number: -564812795 | Message Id Name: VUID-VkDescriptorImageInfo-imageLayout-00344 Validation Error: [ VUID-VkDescriptorImageInfo-imageLayout-00344 ] Object 0: handle = 0x285fe88ab60, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0xde55a405 | vkCmdDrawIndexed: Cannot use VkImage 0x8bac20000000ce5[RID:5179730559158] (layer=0 mip=0) with specific layout VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL that doesn’t match the previous known layout VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL. The Vulkan spec states: imageLayout must match the actual VkImageLayout of each subresource accessible from imageView at the time this descriptor is accessed as defined by the image layout matching rules (https://vulkan.lunarg.com/doc/view/1.3.250.0/windows/1.3-extensions/vkspec.html#VUID-VkDescriptorImageInfo-imageLayout-00344) Objects - 1 Object[0] - VK_OBJECT_TYPE_COMMAND_BUFFER, Handle 2774524275552 at: VulkanContext::_debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:267)
ERROR: VALIDATION - Message Id Number: -1735852301 | Message Id Name: VUID-vkCmdDrawIndexed-None-02697 Validation Error: [ VUID-vkCmdDrawIndexed-None-02697 ] Object 0: handle = 0x1ae36124450, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x9888fef3 | vkCmdDrawIndexed(): VkPipeline 0x5ae80f0000000f57[RID:9028021256219] uses set#1 but that set is not bound. The Vulkan spec states: For each set n that is statically used by a bound shader, a descriptor set must have been bound to n at the same pipeline bind point, with a VkPipelineLayout that is compatible for set n, with the VkPipelineLayout used to create the current VkPipeline, as described in Pipeline Layout Compatibility (https://vulkan.lunarg.com/doc/view/1.3.250.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdDrawIndexed-None-02697) Objects - 1 Object[0] - VK_OBJECT_TYPE_COMMAND_BUFFER, Handle 1847743104080 Command Buffer Labels - 33 Label[0] - Render 3D Transparent Pass{ 1, 1, 1, 1 } Label[1] - Render 3D Transparent Pass{ 1, 1, 1, 1 } Label[2] - Render 3D Transparent Pass{ 1, 1, 1, 1 } Label[3] - Copy depth texture{ 1, 1, 1, 1 } Label[4] - Draw Sky{ 1, 1, 1, 1 } Label[5] - Render Opaque Pass{ 1, 1, 1, 1 } Label[6] - Bake Light Cluster{ 1, 1, 1, 1 } Label[7] - Gather Samples{ 1, 1, 1, 1 } Label[8] - Process Screen Space Indirect Lighting{ 1, 1, 1, 1 } Label[9] - Process Screen Space Indirect Lighting{ 1, 1, 1, 1 } Label[10] - Process Screen Space Indirect Lighting{ 1, 1, 1, 1 } Label[11] - Gather Samples{ 1, 1, 1, 1 } Label[12] - Process Screen Space Ambient Occlusion{ 1, 1, 1, 1 } Label[13] - Process Screen Space Ambient Occlusion{ 1, 1, 1, 1 } Label[14] - Process Screen Space Ambient Occlusion{ 1, 1, 1, 1 } Label[15] - Downsample Depth{ 1, 1, 1, 1 } Label[16] - Shadow Render{ 1, 1, 1, 1 } Label[17] - GI Render{ 1, 1, 1, 1 } Label[18] - Shadow Setup{ 1, 1, 1, 1 } Label[19] - SDFGI Store Probes{ 1, 1, 1, 1 } Label[20] - Render Depth Pre-Pass{ 1, 1, 1, 1 } Label[21] - SDFGI Update Probes{ 1, 1, 1, 1 } Label[22] - Setup Sky{ 1, 1, 1, 1 } Label[23] - Render Setup{ 1, 1, 1, 1 } Label[24] - SDFGI Update dynamic Light{ 1, 1, 1, 1 } Label[25] - SDFGI Pre-Process Cascade{ 1, 1, 1, 1 } Label[26] - Render SDFGI Voxel{ 1, 1, 1, 1 } Label[27] - SDFGI Pre-Process Cascade{ 1, 1, 1, 1 } Label[28] - Render SDFGI Voxel{ 1, 1, 1, 1 } Label[29] - SDFGI Pre-Process Cascade{ 1, 1, 1, 1 } Label[30] - Render SDFGI Voxel{ 1, 1, 1, 1 } Label[31] - SDFGI Pre-Process Cascade{ 1, 1, 1, 1 } Label[32] - Render SDFGI Voxel{ 1, 1, 1, 1 } at: VulkanContext::_debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:267)
Second test:
scons vsproj=yes --directory="C:\Dev\godot" platform=windows target=editor progress=no -j15(nodev_build)godot.windows.editor.x86_64.exe --gpu-validationVulkan Configurator running No crash 😕Third test: Same build as second (no
dev_build) Without--gpu-validationWithout Vulkan Configurator Crash occurs Log:After playing around with configurations and Vulkan Configurator settings I’ve found that rate seems related to my FPS and GPU usage, higher -> more frequent
Edit: I was finally able to reproduce with
dev_buildwith all the Vulkan Configurator settings enabled. Here are the logs:The error about using the same texture in framebuffer attachment and a uniform (the first and last error) is caused by me, and I don’t think it’s related to the crash as it is spammed all the time and the crashes were happening before I added the code that caused that message.
Hi,
Paul from Chasing Carrots here. The Vulkan swapchain error is the most common crash report we’re receiving from our Halls of Torment players. It’s been mentioned, that the error could potentially be caused by some overlay injecting failing code, like the Steam overlay for example. Since we can’t go without the Steam overlay in our game, all of our players run the application with its injected code. Yet, only a small fraction seems to experience the problem.
If there’s any way we can assist in this issue, we would be more than happy to do so. We could ask for logs or details about hardware configurations etc. from players who encounter this on a more regular basis. We could also merge pull requests into our own Godot branch and let affected players test a potential fix (on a separate Steam beta branch for example). If there’s anything else we could help out with, please let us know.
I can reproduce this spam:
and crash reliably on 1050Ti by not having the editor update for 25-30s and then doing so. However, If the editor is set to
update_continuously, I am unable to crash Godot in this fashion. I’ve used it as a temporary solution for now and I’d urge others to try as well.I’m getting the same error (Manjaro Linux, RTX 3060). At first I thought it was completely random, but for me it’s more likely to happen the more objects are visible (working on a game with a pretty large map). So I played with the settings and found that increasing “Max Clustered Elements” in the project settings seems to fix it. So maybe one possible cause for the crash is when it hits that limit?
I encountered this issue but I believe it was the software I was running.
When running Godot_v4.0.1-stable_mono_win64 was consistently getting:
within 30 seconds or so of launching.
Switching to “Compatibility” render stopped this behavior.
Interestingly, entirely disconnecting my other monitors increased the time until this happened consistently by about a minute.
I was stopping various processes and realized the swapchain error stopped occuring after I killed Rivatuner. I could then consistently get this error to occur by simply starting Rivatuner and then rebooting godot in forward rendering mode.
Digging further the specific setting that seemed to trigger the consistent swap issue was:
(Note: toggling this option while godot is running results in an abrupt crash)
For those encountering this issue: Maybe there is some app that is trying to draw an overlay over godot or similarly access / inject into your instance?
Hopefully this is at least a little helpful
For me this reproduces 100% reliably when I leave Godot on when I put my laptop to sleep. I get in an endless loop:
OS: Linux (Arch Linux, up to date) GPU: NVidia (NVIDIA Corporation GA104GLM [RTX A3000 Mobile])