We don't want to send WM_CANCELMODE or set the foreground window to the
desktop when a virtual desktop loses focus. It has its own focus independent
of X.