Allow us to avoid grabbing a temporary reference. This becomes a problem with
the next patch, where we would otherwise grab a reference while a texture is
being destroyed, and hence destroy it twice.