The Vulkan spec says:
Parameters to the command requesting a deferred operation may be accessed by the implementation at any time until the deferred operation enters the
complete state. Pointer parameters must not be modified (e.g. reallocated/freed).

This fixes a regression in Doom Eternal with ray tracing enabled with drivers
that actually support deferred operations (e.g. nvidia, amdvlk).