I really don't get our build system.  I don't think anyone does.  What's 
up with buildit namespaces?  What's up with commands vs. tasks?  What 
are the represent and check methods on commands?  Who knows...?  (I 
suppose Chris M. does.)

And I can't help but feel that the problem we are trying to solve really 
isn't that hard.  It's important, but just not *hard*.  Shell scripts 
would almost be more clear.

So I'm considering, maybe this weekend, tossing together something that 
I feel more comfortable with.  Most of the tools to build this are 
available, so I think it can be just a some glue, and it overlaps with 
paster create and some other stuff I've done so I feel like I have an 
intuition for what it should do.  Debuggability and transparency are the 
most important goals; hopefully everything else can flow from there over 
time and by demand.

Obviously some builds have to be developed alongside the build system, 
as a kind of basic test of does-this-do-anything-that-matters.  I'm not 
sure what builds to do; obviously opencore is one of the more tricky 
ones, though stuff like doing database setup in TaskTracker has its own 
complexities.

What follows are some rough notes about goals/design of the system:



hese commands are informed by configuration.

These are the basic commands:

* Copy a tree of files
* Run a generic shell command
* Create a virtualenv
* Install packages there
* Do an svn checkout

The last three are kind of subsets of a generic shell command.  Shell 
commands will be run *carefully*, just like ``paster`` already does: 
non-zero exit codes will not be ignored, output will not just spew 
across the screen.

There are no dependencies; all tasks are run in order.  It is up to the 
task to save work if it can do so (though it's far more important that 
it get everything correct than attempt to save work).  An example of 
saving work might be using ``svn up`` when a checkout exists, instead of 
wiping the checkout and doing ``svn co`` (that's actually a safety 
concern as well, as there may be updates in the svn checkout).

Obviously there's just the option to write Python code too.


Commands have these properties:

* They can be run/executed
* They can (sometimes) detect differences between the current build and 
what they would do; this might not be accurate (e.g., you can't really 
tell what an arbitrary shell command *would have* done), but can be used 
for auditing a build, or a change in a build


Configuration works like:

* The configuration file can indicate that it should bring in values 
from other locations, similar to how zc.buildout works
* The command setup can do the same thing, overlapping files (typically 
based on command-line options, or global config file locations)
* The build itself can modify the configuration in-place (e.g., to setup 
default calculated values)
* The resulting configuration can be displayed and inspected


Logging works like:

* Very explicit logs are written to disk
* Very abbreviated log messages are written to the console (except with 
-v, of course)
* Logging messages will be grouped, so in case of a failure all messages 
from the group leading up to the failure can be displayed
* I may look into some sys.trace and sys._getframe hacks to show the 
flow of the code in the logs too; I'm not sure yet


The command-line scripts will generally take these options:

* A config file that provides a master set of overrides.  This is where 
the global config goes too
* Explicit setting overrides
* A --simulate option
* A --display-config option (giving the user an opportunity to confirm 
the config and abort)
* Any extra options that the build adds; all options get resolved in the 
config fixup stage

Checking out from svn branches will be handled with options, like 
``--opencore-branch=geolocation``; this will write to the 
``opencore_branch`` setting, and the default value for this will be 
``trunk``.  And there will be some routine available in the templating, 
so you can do 
``https://svn.openplans.org/svn/opencore/{{opencore_branch|svnlocation}}`` 
in a substitution.  I'm thinking of using Tempita for all string 
substitution (except on-disk filenames, which will be like paster 
create).  I.e., even using it for little inline strings.  Mostly for the 
availability of expressions like this example.