If you delete a source file, we want to reflect that in the build container.  We
only use --delete going that one way as we don't want to accidentally delete
files in the user's source tree.