- add output to console=ttS0 for run in cloud

Either /etc/hasher-priv/system or /etc/hasher-priv/user.d/$USER
must contain at least "allowed_mountpoints=/proc" for mkimage
to work for mkimage-profiles; thanks Daniil Golovanov for
providing feedback indicating the lack of the corresponding
checks.

The sort key field value has been off-by-one apparently,
"6" corresponds to "capacity" field of df -PT output
while the "available" one was meant:

Filesystem     Type  1024-blocks     Used Available Capacity Mounted on
/dev/sda9      ext4     15350768 12799972   1747980      88% /
tmpfs          tmpfs     1967204    27396   1939808       2% /tmp

1              2               3        4         5       6

This lapse has been five years old, looks like it's only
been masked by lower available space limit *and* filesystem
ordering by type so far.  It didn't hit me even now, rather
inspected the code while implementing volumes-profile-starterkit...

Yet another age old bug: `sfdisk -l' is mimicking what
a person does by hand but the script is rather interested
in what `sfdisk -g' provides, that is, geometry.

And it's stupid enough to only grok C locale.

Looks like there's a race condition somewhere: this script
will fail to clean up after itself when considerable background
load is inflicted upon the host it's running on (e.g. LA ~16
on a 8-core, single-disk system).

Note that this commit is NOT enough to win that race
but just a step in the right direction...

It's unwise to skip the common storage driver,
and vagrant (which uses virtualbox by default)
is likely to crash and burn as well.

The check introduced by commit d7689f30 while rewriting tar2vm
(which presumed x86) was subtly broken: it checked for *host*
binary before preparing *chroot* configuration file for it.

Wonder how many build servers run lilo over here that this BUG
has managed to evade attention for almost two years...

Forward-port of b84182c5cb684afbd30ed1f9e5b6f89f55b4b53f
commit extended to handle /boot as well.

Thanks glebfm@ for spotting that it's = instead of +=
as it goes in all the other places; I remember no good
reason to overwrite the potentially preexisting contents.

Seeing tagged/base+rescue~ in build.log isn't particularly
heart-warming; while other editors but the one leaving tilda
marked backups and .sw* swap files might exist let's do this
step at this time.

Wonder what changed though, this used not to happen before.

Really no need to test and sleep when there's sync mode.

Factor of 2 is not enough for kernel upgrade due to filesystem overhead.
Add BOOT_SIZE_FACTOR on top of that.

A trivial overlook fixed up.

Overview of the changes:
- ARM support: separate ext2 /boot, no LILO
- avoid race condition with devmapper
- trap ERR so that -e in shebang doesn't result in extra cleanup hassle
- configurable root filesystem type (ext4 by default)
- jumps through parted hoops

Details:

1. LILO is x86-specific while the rest of the script can be used
   to prepare e.g. Marvell ArmadaXP or CuBox images; we can generally
   count on uboot supporting ext2 for relatively sane platforms but
   not ext4 that would be a better root filesystem performance-wise.

2. Apparently /dev/mapper/loopXpY can be still missing at the time
   when kpartx returns and pop up a bit later... sit there, wait
   and check for it.

3. If something went wrong with any command of the script it would bail out
   due to -e in shebang; it is now better to clean up the loopback device
   and its mappings in this situation either.

4. One size doesn't fit all, really.

5. The parted sizing was sloppy as in broken, now it's just half insane.
   Someone's decision to stick units and auto-alignment knobs into
   a single one was apparently hilarious...

   http://www.gnu.org/software/parted/manual/parted.html#unit

Manual loop/dm cleanup is described in documentation just in case.

/boot size meter is suboptimal in terms of additional I/O incurred,
will be most likely rewritten to make use of advance "du -s".

The issue at hand is the ability to accomodate boot sector
payload at the start of the filesystem's underlying block device.

XFS doesn't spare that space.

Thanks vsu@ for the reminder, by the way.

Now this is ugly: instead of commoditizing the repetitive code
the result ended up working differently by creating several
repositories for the target subdirs instead of the single one
for the generated subprofile as a whole.

This results in .disk/profile.tgz being basically useless
in every image since c4311108.

The (funny but somewhat confusing) problem manifests itself as

  E: Couldn't find package Binary

during a build run in the profile where a tagged packagelist
referenced by a specific target being built is open with vim
(which results in .FILE.sw? temporary file lying aside).

The issue (#28002) resulting in vm image build error reading

  Syntax error at or above line 5 in file '/etc/lilo-loop.conf'

was caused by fdisk-2.22 changing its "-l" option output format
to drop the very mention of the long irrelevant crap named "CHS".

The problem is, however, that we still need that crap to piggyback
a loop device's fake geometry to lilo while installing it there.

Reported by icesik@.

See http://www.opennet.ru/openforum/vsluhforumID3/86239.html#1
for a query that has led to this one; in particular,
- xdm dropped (won't log in root and there are no users yet);
- network is brought up and configured via DHCP by default;
- apt-get works out-of-box;
- default image size is twice the chroot size.

There were heaps of "if type -t git" there already;
it wasn't an unintentional mishap but rather a moderate
copy-paste to get the use cases, and now these seem to
have essentially settled.

So time to scrap some dups.

NB: the scripts in the generated profile can't rely on
the contents of the metaprofile (these need to be able
to work in standalone case either), so a bit of crap
still lurks there.

No need to just go ahead and source files in potentially
unavailable directories, really.

"df -l" is going to hurt (or at least spam) if it's a diskless node;
"df -P" helps maintain output parseability.  So be it.

Finally tired of expr(1) quirks,
and even ash(1) does $((arithmetic-expansion))!

Yes, mkimage-profiles is now able to build VM disk images.
So far the support is pretty basic:

- a single hard drive image with a single partition/FS
- only stock root password is configurable
- LILO is hardwired as a bootloader

The resulting images tend to boot under qemu/kvm though.

Please see doc/vm.txt for the warning regarding additional
privileges and setup required.  This was started back in
February but I still hoped to avoid sudo/privileged helper
(and libguestfs is almost as undistributable as can be)...

Thanks:

- http://blog.quinthar.com/2008/07/building-1gb-bootable-qemu-image-using.html
- Alexey Morarash who reworked that as https://github.com/tuxofil/linsygen
- led@, legion@, vitty@, aen@ for providing advice and inspiration

glebfm@ asked what to do with new package lists: whether these
belong to features, or to distributions themselves.  This question
is actually open and up for discussion but there are guidelines
that can and should be written down already; and so they were.

Added pkgdups utility reference as well.

The output was still somewhat ragged in 80x24 terminal window
with fmt(1) which wasn't anticipating the word length difference
subsequent column(1) would have to cope with later on.

Thanks Loic Cattani for his shell columnizer implementation:
https://github.com/Arko/Columnize

The purpose is being able to examine particular target interdependency
graph for a given image having been configured to avoid convoluted
dependencies (loops in particular).

The implementation is based on SHELL hook hint by John Graham-Cumming:
http://cmcrossroads.com/ask-mr-make/6535-tracing-rule-execution-in-gnu-make

As was found out by Vladimir Karpinsky (thanks for patience!),
the autochosen directory might still have too restrictive mount
options -- nodev and/or noexec.  Hopefully the diags are a bit
better and faster by now.

It happens that if the host environment isn't particularly
tuned up for package builds already then bin/mktmpdir might
come up with a directory outside hasher-allowed prefix list;
now that's a shame and not a Christmas gift, clearly.

Thanks Vladimir Karpinsky for pointing this problem out too.

The fallback case of building in a brother directory moved
from the last line of code to the first one becoming more
explicit along the way.

Support for slash-containing argument (being a tmpdir name
template prefix) has been added.

The bin/pkgdups.sh script comes from m-p-d in considerably optimized
form and is to be used with the pkglist files of interest passed
as its arguments to produce a "hall of duplicates" among those.

The tagged lists received some updates along the rescue image lines,
most of those are actually inspired by http://rescuecd.pld-linux.org/
and to lesser extent a few articles on rescue/recovery/forensics
software -- so some newcomers are even employed already.

With not-that-recent mkimage-profiles development,
it's no longer apparent that at least a gigabyte
of free space is required to build something useful
(at least for the tests, like syslinux.iso).

In short, the guesser cutoff margin is now 256M.

src/dst tags might have been empty confusing tags2lists;
the current implementation is more robust (along with
slightly better debug within bin/tags2lists itself).

pushd/popd spam tamed too (replaced by nice log messages).

Just in case the build.log will be inobvious, and it's easy to diagnose
automatically.  Thanks Andrey Stroganov for this use case.

Thanks for improving the initial implementation go to raorn@ for kind
commit lynch and to Yuri Bushmelev for actually suggesting something
more concise.

BTW the "1024" magic number was taken out of thin air:
the "no free space" errors are most likely to happen while
forming/populating a chroot (apt/rpm errors out) and chroots are
roughly two orders of magnitude heftier than a megabyte.

- more thorough build stages inline doc
- quieter BUILDDIR instantiation
- branding-altlinux-sisyphus by default
- prettify installer selection somewhat
- mkimage #24669 fixed upstream
- sub.in/stage1/modules: +drm (as in m-p-d, for KMS)
- tags2lists: generalize a bit

tags2lists was tweaked to work on the more natural word boundaries
than explicit "_", and the lists were renamed accordingly.

Dropped the borrowed pkglists by the way.

- syslinux: menu.c32 works
- features: initial draft
  + memtest
- mv pkg/ pkg.in/
  + server-base: include openssh-blacklist by default
- mv isodata/ copy/
- README: changed to utf8, added an URL

These feel quite interesting but not that obvious yet...

As not every package list is useful for every distro,
only the needed lists should be copied for a particular one.

- bin/tags2lists
- pkg/lists/tagged/