Extending the Buildbot

From Open Source@Seneca

Jump to: navigation, search

Contents

Description

This project is a catch-all for Buildbot development done here. The Buildbot is an automated build system written in python. It is used on the Mozilla Seneca Cluster.

Leader

Ben Hearsum (bhearsum)

Contributor(s)

News

January 2007

16:32, 1 January 2007 (EST)

  • I've updated the original Build Commenting patch to remove the starPath argument I originally had. Looking back on it it was a pretty ugly solution. There is now a 'C' beside the 'Build NN' when a Build has a comment.

16:50, 6 January 2007 (EST)

  • I have created an NSIS installer for Buildbot. Version 0.1 has been released. See below for more details.

19:31, 7 January 2007 (EST)

  • Version 0.2 of my Buildbot installer has been released. The only change is that it now generates a real, working buildbot.tac file.

13:56, 10 January 2007 (EST)

  • I released a patch to fix a non-ascii character bug in the BonsaiPoller. See the Downloads section.

November 2006

21:57, 3 November 2006 (EST)

  • I finished by bonsaipoller test cases today! I ended up doing a complete re-write of the BonsaiPoller. More information in my [1]

22:07, 3 November 2006 (EST)

  • Here is my new BonsaiPoller patch. This patch includes the revision patch from Rob and all of my test cases.

09:23, 21 November 2006 (EST)

  • New Bonsai polling patch has been applied to the trunk.

Roadmap

Short Term

Tinderbox Quickparse Queries

Rob Helmer would like to start migrating build testing to a Buildbot setup rather than having it run on Tinderbox. This involves parsing the Tinderbox quickparse page and kicking off builds when one is done. The patch itself will be similar to the BonsaiPoller one with a few minor changes. Rob is discussing a possible change to the Change object to make this implementation cleaner.

Rob Helmer has written this patch.

Tinderbox Module

The current patch is complete already but there should be a set of test cases to go along with it to help prevent regressions.

Buildbot Try Scheduler

This is something that many people would really like to see working. I'm not sure whether this feature is still at alpha or beta stage but I haven't been able to get it to work yet. It's a fairly high priority due to the possible payoffs once it is working. This item moves to medium or long term if it turns out it's alpha/beta.

Medium Term

Build Commenting Patch

The current build commenting patch is fully functional but not a great patch. It adds an extra option to the Waterfall called 'starPath', which is supposed to hold the full path to a icon that shows up when a comment is added. It's an ugly hack and should not be there. There is currently no other way to add an image so this option should be removed. There still must be a way to know if there is a comment on a build but perhaps in a text-only format. Any other clean method of doing this is acceptable as well.

The other questionable thing is a modification to the I*Status objects to allow manual calls to their serialization methods (saveYourself()). I don't know enough about Buildbot as a whole to know if this is a good solution so until Brian reviews the patch I don't know if it acceptable.

Brian still has not gave his feedback on the patch so once the starPath option has been ripped out he should be pinged about reviewing it again.

Generalized Build Output

Right now all of Buildbot's output methods are query the Status objects themselves and send out whatever information they need to. If data was sent out by something like an RSS feed or XML page or SOAP, etc. it would be very easy and quick to parse it and add new mediums of displaying it. For example, if an RSS feed was available the Generalization of the Joga Extension team could easily add support for displaying Buildbot output to their extension.

After a generalized output method is implemented the next step would be to move all existing output methods (Waterfall, e-mail, ircbot, etc) to use it instead of the Status objects. This would be a fairly major overall to Buildbot architecture. Before undertaking this task Brian Warner (the Buildbot maintainer) should be contacting and agreeable to it -- so all the hard work doesn't happen for nothing.


Long Term

Drive Mozilla builds with Buildbot

Rob Helmer would eventually like to drive builds with Buildbot instead of a Tinderbox server.


Ongoing

Waterfall Display CSS

Currently on the Mozilla@Seneca cluster we are using a buildbot.css borrowed from someone else's Waterfall display. This is a low priority item but all new waterfall.css' will be accepted. If they are clean and likable it is possible that they could be distributed in a standard Buildbot distribution.

Tasks

Task Details Priority Contributors Status Link(s)
Tinderbox Notification The Mozilla Tinderbox helps developers and users know when a source tree is stable. For Buildbot to be really useful for doing test builds it will need to be able to update the Tinderbox. The Tinderbox recieves information via formatted e-mails. Low Initial test cases written. I feel that there should be more though.
Build Commenting The Buildbot waterfall display is very similar to the Tinderbox display page. One feature currently exclusive to the Tinderbox is the ability to comment on a build, also called "starring". When implemented it should be possible to select one or more builds for annotation. When a comment is added an icon will be displayed in the cell and the comment added to the head of the log. This comment should be persistent even after the build(s) it affects are no longer visible on the front page. Low Patch has been submitted to Brian. Waiting for his review of it.
New CSS file for the Waterfall display There are a few waterfall.css files around but none of them are all too great. One that does more table borders would be great. Also, pretty colours. Ongoing * Liz Liz contributed a style, thanks Liz! waterfall.css.liz
Generalized Build output Different methods of reporting build results (Waterfall display, mail notification, irc bot, etc) all work in different ways. The development of a generic reporting system, maybe done with SOAP or RSS, would make the development of new build reporting systems much easier. This is something that will be useful to the Generalization of Joga extension team. Low 0% complete
Tinderbox quickparse querying Query a Tinderbox quickparse page (example) and run tests when a build has finished. Rob Helmer suggested this and may be implementing it. More detailed description here Done Rob Helmer Patch is complete.
Test a Buildbot TryScheduler with MailNotifier. Low Ben will probably do this Getting errors from CVS. Need to e-mail the mailing list again.
Add filtering by e-mail address for Bonsai Currently the only way to filter possible builds with a BonsaiPoller is by Branch. Since many projects check-in to the same branch it would be convenient to filter by other methods such as e-mail address. Done Ben Finished.
Buildbot Installer Installing Buildbot on Windows is a bit of a task -- especially for slave installs. An installer that installs all dependencies and creates the '.tac' file in the case of a slave needs to be created. Bonus points for setting up Buildbot as a Windows service. Medium Ben 0.2 released. WIP still.

See Downloads

Downloads

Snapshots:

Tinderbox Patches:

Bonsai Patches:

Build Commenting Patches:

CSS Files:

Installer:

1. Other files are required to generate the installer from source. They are listed at the top of the nsi file.

Related Links

Misc.

Personal tools
special sections