This is a draft - please edit. I'm thinking of posting it to the operations blog at some point, probably next week.

Late last night in the office, an idea was floated based on these three points:

1. Openplans needs refactoring at some point (a more workable codebase for future developer-sanity, easier implementation of new features, and so on).
2. It's going to be hard to get everyone to agree on how to do it, both in terms of design and implementation and... pretty much anything else you could think of.
3. It's going to be harder to refactor the longer we wait and the more the codebase grows.

The idea floated was Operation: Rewrite Sprint.

Basically, post-woonerf launch (plus some time for recovery for the caffiene to seep out of our collective bloodstreams), all interested people split into teams of their own choosing. Small teams, maybe 2-4 people. They have 2 weeks to - in any way they want - rewrite openplans.

Note that this is a rewrite-in-spirit, not necessarily a rewrite-in-functionality, meaning that we may actually go back and do stuff like...

http://www.openplans.org/projects/opencore/objectives-and-site-definition
http://www.openplans.org/projects/opencore/info-architecture

Judging will be done by those not participating, based on some combination of:

• Scaleability
• Speed
  
• Usability - how quickly and easily can a non-coder user do a series of simple tasks? (Something like the smoke test.)
  
• Tests and testing framework written for the software
• Hackability - some new required features will be announced at the 1-week mark, and teams will have to scramble to incorporate them into their designs (alternatively, each team may have to teach a coder from outside their team how to contribute something to their code base a day or two before the deadline)
• Other criteria I've forgotten here
  

Note that implementation is not in this scoring schema - if you wrote it in pylons or zope or django or... heck, if you wrote it in php, even - that doesn't matter. Maintainability and extensibility does, which is why the last two scoring criteria exist. Also note that good looks aren't in this scoring scheme.

Yes, the idea is nuts. But so are we.

Thoughts? Criticisms? Edits?

jeff's response

I'll say up front that I think the basic idea is a good one.  But I
also think that it needs to be done with consummate care in order to
have a productive result in the end.

The competition should be fun.  That is to say the emphasis should be
on having fun writing good code that supports the laid down objectives
(from Jackie or whoever), and not a vicious competition or a
competition aimed at proving a point or with the idea that if you win
the game that your software will be used without review or
consideration of other ideas.

The competition should be a learning experience.  The goal should be
to turn out good code, but it should be kept in mind that the code may
be discarded or used in a different form than the author intended.  I
think critical to the competition should be the idea that, at the end
of the day when whichever team wins, we take some serious time as a
group and review all of the different implementations.  We talk about
strengths and weaknesses of each implementation and decide (preferably
as a group, but by the "judges" if that is not possible) what can be
taken away from each team's implementation to go into openstack 2.0.
Team A might have a really nice event server, but Team B might have a
really nice WSGI middleware layer for wiki, commenting, and UI.  It
doesn't have to be winner takes all, and in my view it shouldn't be.

The rules of the game should be set by the "judges" (Jackie + whoever)
and be very clear on what the rewrite entails and the parameters of
the competition.  Is this a functional rewrite?  Is this an in-spirit
rewrite? To me, the interesting problem is how the various components
communicate together.  If the software is a monolith, this isn't an
issue.  I would personally vote against monolithic software in this
and almost all cases, but having a monolith does have the advantage
that it avoids API issues.  If the software isn't a monolith -- if our
blog solution and our wiki solution and our mailing list solution
should each work on their own -- then there needs to be an "API" (in
the loosest possible sense) whereby components communicate.  Where
does UI and javascript live?  Where does auth live?  Can one include a
blog roll on a wiki page?  What is a project?  Where does that concept
live?  Whether the various components should be standalone apps or not
should be specified as part of the competition parameters.

Since this is such a critical part and issue with our software
currently, I think its very important to be up front on what our
communication mechanism is.  It could be specified before the
competition -- use cabochon (presumedly with a code freeze such that
people aren't stepping on each other's feet) or use an XML format such
as atom or use any event notifier with the following interface, etc --
or it could be part of the competition.

Judges can and probably should specify tests as a baseline that
guarantees a nice API.  These could be flunc tests, for instance.
They might have to be refined as the competition goes on, but its at
least a starting point.

Build process: is this part of the competition?  I mean, it is
implicitly.  The result should not be a demonstration that one's
software works on one's computer.  It should be grabbing whatever
build we set up, rebuilding, and deploying on some arbitrary machine.
I think if we don't have that, we'll get really hacked together
software that is working in one place locally but is difficult to
build elsewhere.  Maybe we agree upon a port to hit, have the team
provide a build script, and the judges fire it off on an arbitrary
computer.  Internally, people will probably mainly use fassembler
(though again, we should really code-freeze anything people are using;
 branching is allowed and encouraged;  committing to the fassembler
trunk should not be), but if someone wanted to do everything with
setuptools or (blasphemy!) make, then they (IMHO) should be free to do
so.

There is one other critical issues as far as a rewrite goes, which is
only casually related to the idea of a competition.  Namely, do we
rewrite incrementally or do we swap out our entire core overnight?
I'm much more comfortable with incremental rewrites, though they tend
to take longer to complete.  Part of our problem is that we have never
done much reworking of our software except as a last resort, as since
NUI we have been non-stop on pressure to release various deployments.
While I realize this is no excuse, it is an excuse we have used.  We
could do substantial improvements slowly, for instance: factor the
wiki out of zope into a WSGI app, rework our collective need for feeds
of latest activity, generally consolidate and reduce our API into
something manageable, make listen a 1st class product and standalone
app, put auth in front...all things we have talked about, but no
man-hours have been devoted to them.  Rewrites are fun, but they're a
big deal.

Again, to reiterate, if we do this competition, it should be fun for
all, should not unfairly reward the winner or punish losers, should be
a collective learning experience, and should result in forward
progress on getting opencore in better shape in the end.  Once all is
said in done, it is all our code -- not just because it is opensource,
but because we are TOPP and TOPP doesn't work well when a small number
of people exclusively own code.  She has been frantically busy, but I
would try to run this idea past Jackie when you get a chance.  I'm
sure she would have strong opinions on the subject, though I can't
guess what they might be.  Also, you are free and encouraged to add
any of this info to referenced wiki page (or I can do it, if you
really really want).