This operation takes 2-5 seconds on every build, but doesn't actually need to
run most of the time.  Now we cache it and see if it needs a rebuild (fast)
before actually rebuilding (slow).