[Angstrom-devel] [oe] [RFC] initramfs support for linux-handhelds-2.6

[Paul Sokolovsky]

Hello,


So, changes discussed below, has been pushed to .dev. They are in turn
part of the plan outlined in
http://lists.linuxtogo.org/pipermail/angstrom-distro-devel/2008-February/001776.html

So, from now on, the linux-handhelds-2.6 kernel for angstrom-2008.1
in .dev must be built with the following commands:

ANGSTROM_MODE=uclibc IMAGES_FSTYPES=cpio.gz bitbake \
	initramfs-bootmenu-image 
ANGSTROM_MODE=glibc bitbake virtual/kernel

I.e., before going to build the kernel itself, initramfs-bootmenu-image
for uclibc (!) must be built.

I would like to ask all linux-handhelds-2.6 device maintainers/testers
to test the resulting kernel, so we can push these changes to the
stable branch soon.

Thanks,
Paul


On Mon, 25 Feb 2008 15:57:52 +0200
Paul Sokolovsky <pmiscml at gmail.com> wrote:

> Hello,
> 
> On Mon, 25 Feb 2008 14:03:55 +0100
> Koen Kooi <koen at dominion.kabel.utwente.nl> wrote:
> 
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > Paul Sokolovsky schreef:
> > | Hello,
> > |
> > | On Mon, 25 Feb 2008 10:15:11 +0100
> > | Koen Kooi
> > <koen at dominion.kabel.utwente.nl> wrote:
> > |
> > |> -----BEGIN PGP SIGNED MESSAGE-----
> > |> Hash: SHA1
> > |>
> > |> Paul Sokolovsky schreef:
> > |> | Hello,
> > |> |
> > |> | Failing to add generic and fully automated
> > kernel-with-initramfs |> build, | I have patch to add external
> > initramfs to kernel for |> linux-hh. I'd | still like to establish
> > conventions for other kernel |> recipes to follow, | so here're the
> > patch for suggestion. The idea is |> simple: |
> > |> | 1. KERNEL_INITRAMFS_PATH is set to the full path of cpio.gz
> > image to |> | use as initramfs. Empty value (default) means don't
> > use initramfs. |> If | file doesn't exist, build fails with an error
> > message (content |> is left | to specific recipe - there
> > unfortunately doesn't seem to be |> a way to | sensibly provide
> > detailed one in generic case). |> |
> > |> | 2. KERNEL_INITRAMFS_PATH is considered a distro realm, like for
> > |> example | kernel PREFERRED_VERSION. Below, angstrom distro config
> > |> sets it.
> > |>
> > |> | +KERNEL_INITRAMFS_PATH =
> > |>
> > "${TMPDIR}/deploy/uclibc/images/${MACHINE}/initramfs-bootmenu-image-${MACHINE}.cpio.gz"
> > |>
> > |> | +	if [ -n "${KERNEL_INITRAMFS_PATH}" -a
> > "${ANGSTROM_MODE}" |> == "glibc" ]; then
> > |>
> > |> A number of things:
> > |>
> > |> 1) Make the initramfs generation a seperate function (or task)
> > that |> can be called at will, instead of always being called in
> > |> do_compile_prepend
> > |
> > | Initramfs *generation* is a separate task. With this scheme, it is
> > | completely external, binary blob drop. Except by setting initramfs
> > path | to point to OE's deploy dir, one can use previous OE build
> > result. So, | for our case, it would be:
> > |
> > | ANGSTROM_MODE=uclibc IMAGE_FSTYPES="cpio.gz" bitbake \
> > | 	initramfs-bootmenu-image
> > | bitbake x11-image
> > |
> > | and everything will magically work.
> > |
> > |
> > |> 2) Keep it simple and keep the kernel and
> > |> initramfs in the same libc realm, so a uclibc based initramfs
> > needs a |> uclibc kernel (I known there is no such thing, but you
> > get the idea). |
> > | Keeping it separate is exactly *the simple* thing. Trying to do it
> > | otherwise calls for hard-to-resolve-in-good-way complications.
> > Again, | there's recursion in the job of building kernel with
> > initramfs: |
> > | To build kernel, you need to have initramfs. To build initramfs,
> > you | need modules. To build modules, you need to build kernel.
> > |
> > | You saw one solution for breaking this vicious cycle - the
> > recursion | was lashed into OE's own blackbox where it was kept
> > under control under | given, real-world achievable circumstances.
> > That was rejected. |
> > | Current solution splits cycle the other way: 1) "uclibc" kernel is
> > never | built with initramfs; 2) that means that its build finishes
> > without | recursion, giving out modules; 3) uclibc initramfs is
> > built with that | modules; 4) then "glibc" kernel recipes pulls it
> > automatically(!) from | uclibc deploy dir, builds kernel with it,
> > and more importantly | *packages* it w/o any dirty magic.
> > 
> > So:
> > 
> > ANGSTROM_MODE=glibc bitbake virtual/kernel
> > 
> > Will build a kernel without initramfs and:
> > 
> > ANGSTROM_MODE=uclibc bitbake virtual/kernel
> > ANGSTROM_MODE=uclibc bitbake initramfs-foo-image
> > ANGSTROM_MODE=glibc bitbake virtual/kernel
> > 
> > Will build one with initramfs, right?
> 
> No, vice-versa.
> 
> ANGSTROM_MODE=uclibc bitbake virtual/kernel - builds w/o initramfs
> (by-product, never used, built only for modules)
> ANGSTROM_MODE=glibc bitbake virtual/kernel - builds production kernel,
> with initramfs
> 
> 
> > If so, I want to propose something different:
> > 
> > ANGSTROM_MODE=uclibc bitbake virtual/kernel
> > ANGSTROM_MODE=uclibc bitbake initramfs-foo-image
> > ANGSTROM_MODE=uclibc bitbake initramfs-bar-image
> > ANGSTROM_MODE=uclibc bitbake initramfs-qux-image
> > ANGSTROM_MODE=glibc  bitbake virtual/kernel
> > 
> > The virtual/kernel recipe (e.g. linux-handhelds-2.6_2.6.21-hh21.bb)
> > looks in the KERNEL_INITRAMFS_DIR (e.g.
> > ${TMPDIR}/deploy/initramfs/${MACHINE}/) for initramfs*cpio.gz and
> > iterates over those creating
> > 
> > zImage.bin (no initramfs)
> > zImage-initramfs-foo.bin
> > zImage-initramfs-bar.bin
> > zImage-initramfs-qux.bin
> 
> Or, speaking in other words, it picks up random, non-deterministic
> stuff and creates something from it? -1 from me.
> 
> > 
> > in ${DEPLOYDIR_IMAGE} and
> > 
> > kernel-image_2.6.21_h4000.ipk
> > kernel-initramfs-image-foo_2.6.21_h4000.ipk
> > kernel-initramfs-image-bar_2.6.21_h4000.ipk
> > kernel-initramfs-image-qux_2.6.21_h4000.ipk
> > 
> > in ${DEPLOYDIR_IPK}.
> 
> virtual/kernel recipe cannot create packages on its whim. Packaging
> stuff is flesh-deep area of OE, which works well and transparent when
> used as it is intended, but is not ready for such tricks. Most
> discussion with Richard on the previous try evolved actually around
> this, not even bitbake recursion. It ended with him saying something
> in the spirit of "I know a magic way, but I won't tell ya", but I
> didn't insist as by that time I gathered that OE simply not ready for
> such tricks, and any implementation of that would be a pretty dirty
> hack.
> 
> > That way people wanting a kernel without initramfs (for whatever
> > reasons) get one, and people wanting a kernel with initramfs get one
> > as well.
> 
> Well, just compare all the procedure above with what I've written in
> the original message: people who want kernel with initramfs, set
> KERNEL_INITRAMFS_PATH to the exact initramfs image to include, and get
> properly packaged kernel without any residue in deploy dirs (or
> alternatively error if the image they expect to be used is not
> available). People who don't want builtin initramfs, just don't set
> KERNEL_INITRAMFS_PATH (what is the default), and also get exactly what
> they want and not something else in addition.
> 
> Finally, people who want both, or threesome, call for rather different
> build configs, so, to not get themselves confused, they just split it
> up into different DISTROs.
> 
> > The big gaping hole in this a mechanism that select which kernel
> > package to install (for machines with kernel in rootfs), but that
> > shouldn't be too hard.
> 
> Big gaping, yeah.
> 
> > 
> > And this scenario is generic enough for general usage, so we can do
> > this for 'all' kernels in OE, instead of a specific 'family'.
> 
> Well, mine obviously too - that do_compile_prepend can be moved to
> kernel.bbclass and all recipes will enjoy that functionality without
> changes.
> 
> > 
> > regards,
> > 
> > Koen
> > 
> > 
> > |> That way you can just fish it out of DEPLOY_DIR_IMAGE
> > |
> > | The main problem is not kernel image, but kernel package.
> > |
> > |> 3) Add an initramfs-<kernel-name_version>.bb that does
> > |>
> > |> ~  require <kernel-name_version>.bb
> > |> ~  FILESPATH = "/path/magic/whatever"
> > |> ~  do_compile_prepend() { do_initramfs}
> > |>
> > |> This involves some extra build time ('uclibc' kernel build and a
> > |> 'uclibc' initramfs kernel build), but is more transparent and
> > less |> error prone than your plan. I think it even avoids the
> > dependency |> problems :)
> > |
> > | Packaging.
> > |
> > | Besides, it's conceptually not very correct. For a given
> > | configuration (distro+machine), there used to be one kernel,
> > right? | Now, there're two, one of which is not supposed to be
> > used, nor it's | even working (now that we rely on initramfs, a
> > kernel w/o it is dead | beef). One way to ensure transparency is to
> > not confuse people with | giving out build results which are not
> > intended to be there. My patch | works exactly that way - if
> > Angstrom currently really supports glibc as | "production" config,
> > and thus "glibc" kernel is production one, then it | will be always
> > built with initramfs, and non-initramfs (== broken) | kernel won't
> > be built at all, to not make all this prone to errors like | it
> > getting into rootfs, picked up by users, etc. |
> > | Yes, that's not too general. But as I told, the correct way to
> > solve | this is to have zImage vs modules split for kernel recipe.
> > (Currently | tried splits - user level vs OE balckbox; uclibc vs
> > glibc - are just | hacks around the issue).
> > |
> > |> regards,
> > |>
> > |> Koen
> > |>
> > | []
> > |
> > 
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.5 (Darwin)
> > 
> > iD8DBQFHwry7MkyGM64RGpERAgp2AJ4qQGU212X2cU1VuT7YuuTM13Ro+QCeLNcZ
> > YfKyN8LUcuhnutmRqVdF7Ec=
> > =SzBL
> > -----END PGP SIGNATURE-----
> > 
> > 
> > _______________________________________________
> > Angstrom-distro-devel mailing list
> > Angstrom-distro-devel at linuxtogo.org
> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/angstrom-distro-devel
> 
> 
> 
> -- 
> Best regards,
>  Paul                          mailto:pmiscml at gmail.com



-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com