Ryan Hill (psykil) wrote,
Ryan Hill

the importance of test suites

(this isn't directed at any one person or group or any recent incident, this
has been bugging me for years)

I have one simple request.  When you make a non-trivial change to an ebuild -
a patch, a version bump, anything that can effect the behaviour of the
package - please run the test suite.  If it fails, fix it.  Or restrict it.
Or even make it non-fatal if there's no other choice.  If you can reproduce
failures outside of portage, report them upstream.  Failures indicate either
a broken package or a broken test suite and either way it's in your best
interests to get them fixed.

Remember that for anyone running FEATURES=test a failure breaks the build[i].
You wouldn't commit something that doesn't compile (hopefully :P), so why
is this any different?  There is no point in even having test suites if
everyone just disables them in frustration because every third package fails[ii].

I apologize for the rant, but when I'm working for gcc-porting I rely heavily
on tests to catch runtime issues.  And every release cycle I end up
spending way too much time trying to figure out why a test is failing, only
to find that there's been an bug open about it for two years with no

[i] I know about test-fail-continue, but I've found that it just causes me
file fewer bug reports because they don't annoy me as much. ;)

[ii] a neat trick to disable tests on a per-package basis:  masking the
test USE flag of a package will disable tests even if the package has
no test flag

$ cat /etc/portage/profile/package.use.mask
dev-db/virtuoso-server                  test
dev-java/commons-cli                    test
dev-libs/boost                          test
dev-libs/ppl                            test

Tags: gentoo

  • lto USE flag removal

    I've gotten a couple emails already asking why I removed LTO support from GCC today. Short version: I didn't. It is now unconditionally enabled…

  • Stupid Portage Tricks - DOC_SYMLINKS_DIR

    Ever have a package that installs HTML documentation into /usr/share/doc that you want to create a local bookmark to? For example, I find it useful…

  • Back again

    Finally back online. Between graduating from SIAST in the field of Geomatics Technology, moving to Yorkton, starting back at work for Altus, getting…

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.