[oe] OE recipe tree quality

[Esben Haabendal]

On Fri, Jul 30, 2010 at 10:31 AM, Graeme Gregory <dp at xora.org.uk> wrote:
>  On 30/07/10 09:22, Koen Kooi wrote:
>> On 30-07-10 09:21, Esben Haabendal wrote:
>> > On Thu, Jul 29, 2010 at 2:07 PM, Philip Balister
>> <philip at balister.org> wrote:
>> >>
>> >>
>> >> On 07/29/2010 05:45 AM, Koen Kooi wrote:
>> >>>
>> >>> -----BEGIN PGP SIGNED MESSAGE-----
>> >>> Hash: SHA1
>> >>>
>> >>> On 29-07-10 10:50, Frans Meulenbroeks wrote:
>> >>>>
>> >>>> Dear all,
>> >>>>
>> >>>> Given the discussions on quality that sometimes pop up (and also
>> >>>> triggered
>> >>>> by Robert's message), I decided to kick off a bitbake -k world.
>> >>>
>> >>> Could you first explain to me why 'bitbake world' is a good way to
>> >>> measure quality?
>> >>>
>> >>> I would think that building something like console-image and
>> looking at
>> >>> the following would be a much better metric:
>> >>>
>> >>> * does it build?
>> >>> * are all the rootfs types working?
>> >>> * does the image do what it is supposed to do?
>> >>> * Are all the licenses of the output packages correct?
>> >>> * Do the output packages have any spurious deps?
>> >>> * Is the content of the output packages correct?
>> >>> * Are there any known CVEs in the resulting packages?
>> >>> * Did packaged-staging do its job?
>> >>> * What kind of QA errors and warnings were raised?
>> >>> * Did all recipes pass recipe_sanity?
>> >>> * Did all recipes conform to oe-stylize.py?
>> >>>
>> >>> etc
>> >>>
>> >>> I would actually advocate removing the 'world' feature from bitbake/OE
>> >>> to stop people from wasting time on looking at bitbake world and have
>> >>> them fix actual problems.
>> >>
>> >> bitbake world seems to be the source of pointless listserv
>> discussions. Does
>> >> it serve any purpose?
>>
>> > Pointless or not really depends on how you look at quality.
>>
>> > If you look at it as you, Koen and other OE long-timers, yes, it looks
>> > rather pointless to have bitbake world.
>> > But for those of us who have a different view on what quality is, then
>> > bitbake world serves a purpose.
>>
>> As Thomas points out, as soon as you start blacklisting things (which
>> actually increases quality), bitbake world doesn't work anymore.
>> That alone should be enough to kill it.
> Time to jump in the cage here.

Welcome :-)

> "Quality" is achieved by comparing a set of known specifications against
> a known data set. In the software case this means we need a good set of
> specifications which we are testing against. We also need to know in
> detail what we are testing against this set of specifications.

That is by no means the definitive definition of "Quality" in software.