I'm depressed.  My avowed position is to work on the unit + flunc tests until they work.  Maybe that will be forever, or maybe just a few weeks.  In any case, I feel very used and alone.  I'm not going to write more about this now, as it will just make me more depressed, but there is a blog post I wanted to write and don't *think* I have already done that I'll write instead, which will hopefully be more inspiring to me:

How Writing Software and Writing Fiction are the Same (and not)

[Note that this contains some spoilers to my fiction.  I mean, not really unless you're supersensitive about such things, but if you are...you've been warned]

I'm an aspiring author, if you didn't know.  I'd probably plug some of my stories here, but my website is down due to lack of internet at home, so you'll just have to take my word for it.  Increasingly at TOPP, especially when most of my time was taken up writing code, I've found my process for writing both code and fiction to be very similar.  You could say that this is because I'm small-minded and am trying to use one tool for both, but I would argue that this is not the case, that in fact they are very similar problems on different levels or perhaps specific incarnations of an abstract problem.

 Writing fiction basically takes place with an idea:  what do I want to say?  With software, its what do I want to do?  From this usually vague beginning ("there is a very dark girl named Hryn..." | "I want a plain-text format that's more human-readable than HTML that I want to make web pages out of"), I'll start writing little snippets of fiction/code.  Sometimes I'll start with a framework (chapter outlines | classes and interfaces) or sometimes I'll write something very specific that is important ("When all is dark forever, perhaps then I will have peace in the shadows"| a function that marks up its arguments), both most often my mind tends to go up and down from the general to the specific doing both of the above, my fingers never able to keep up with the proliferation of ideas.  Usually at this stage I'm very excited:  I'm making something new.  Long ago I gave up on making anything truly unique (either in the software world or the writer's world), but I'm still very excited when I start putting an idea on paper.

 Sometimes after this stage I shelve it or trash it.  Either the idea is already out there somewhere, or I decide I'm not as excited about it as I should be, or I just don't have the time.  Its not a bad thing to do:  not every story or piece of software needs to be written.

 But sometimes there is something that the next day (or week or month) still seems a good idea to go ahead with.  Then comes probably my favorite part of software development or writing:  exploring the long dark forest.  In software, this is the problem solving stage.  You have some vague classes and ways they should work together.  What functions do you need?  How are different elements going to talk to one another?  Are you really writing one piece of software or is this actually several pieces that should be kept discrete?  You get to write the fun functions, classes, and other really "cute" pieces of code.  Of course, I've been around the block enough to know that today's "clever" is tomorrow's daily wtf.  But that doesn't really decrease the fun factor.   Its all new ideas, or new to me at least, or new ways of looking at old ideas.

 The skeleton-growing stage of writing differs a little in form:  who are these characters?  (They feel like real people disparate from me instead of fictional beasts I just made up).  What are they doing?  How are they going to interact?  In my earliest years writing, I was much more concerned with "What happens?" (the plot), but generally I have found that if you put some interesting characters together they'll naturally interact and make a plot.  I suppose that makes it sound more ad hoc than it is...I generally have a good idea what is going to happen ("Aora + Hryn are going to confront each other and sparks will fly"), and a "character" to me includes much about their background, their world, and other details that aren't strictly part of their personality.  Kain is an officer in the Imperial Corps, for instance.  Which brings up "What are the Imperial Corps?" and other questions leading to more questions that get to be explored in this stage.  In both this stage of writing and software development, the real process (its own reward) is playing with ideas.  You need a computer or piece of (virtual) paper to keep track of things (at least I do), but the ideas are the interesting part.

 At some point in time, this youthful stage transitions to the "fleshing out" stage.  At some point the program needs to do something.  At some point  the story needs to be readable by someone else.  In both cases, this usually involve making commitments and decisions that I can tell you I do not enjoy doing, but I am getting better at it.  In both cases, when I was both younger and more idealistic, I  would wait for the "perfect" idea to come across before making any decision that would pin me down.  I guess I was talented/lucky enough to get away with this in the software world some of the time.  As far as writing goes, all this strategy gave me was an excuse not to write.  Hence my Chapter 1 only being readable from start to finish now as opposed to ten years ago.  For code it was largely the same excuse, I just largely got away with it.

But at some point, in both disciplines, I have begun to see the value of "getting something done" as opposed to finding the perfect idea.  I still probably err on the side of the latter (as do so many programmers and writers), but its something I'm working through.  At some point in my Chapter 1, I just wanted to finish it.  Its not sloppy, but some bridges are rough (both in software and in writing, bridges are my weak points ... I have pieces A and B that both work wonderfully on their own....but they don't match up.  I guess this is called an "interface" pattern in software, but its the same thing.  Easy to do sloppy, often hard to do well in my experience).  Likewise, in NUI, we rolled out much code that was completely unpolished, hacked together, some of it beautiful of its own right, but at best its cohesion was poor.  But it needed to get out.  Overall, I have been trying to make my strategy more towards getting things done and out there than making things perfect.  Because for that, there is editting. Or refactoring.  But I'm not going to use the r-word except here (and maybe one other place).

 So you have a program that does something and you need to add a new feature.  You look where you want to plug it in, open the file, and OMG IS COVURD IN WURMS!!1.  Sorry to go all lolcats there for a minute.  But the point is that where you hoped to hook in the new feature is far from suitable to adding it.  You might be able to hack it.  Or it might make more sense to rework the underlying code.  I'm not  going to talk about when to refactor.  Its largely a matter of  experience governed by immediate needs and judgements about the future, I've already talked about it, its completely off topic, and  its a boring discussion.  Suffice it to say, if you have a sloppy piece of code that's going to be touched again and again, at some  point it should be rewritten to be not-sloppy.  This is a continuous process, going up and down in scope, but new features small and large will need to be added for the life-cycle of the software, and its innards need to be reworked.  After all, you were writing the damn  thing to get it out there, there was no time to make it perfect and it didn't have to be.  So you have something out there but it needs more  work to move forward.  This is a reality of not releasing perfect software.

Likewise, I've given up on writing the perfect novel, especially not in one  straight shot.  My strategy for my novel  (similar to that for software) is get it done  and then review its present form.  It might need to be reworked completely.  But the time to futz around with "is this feasible?" was wayyyy back in its infancy.  The only thing more annoying with people that try to write without editting (except maybe Kerouac) are people that try to design software that they don't expect the design to change over time.

While it seems like its a straight-forward point when a novel is done, to me this seems less the case (he said, not having finished one yet). There is the moment when the last chapter is ready for review.  But long before this has happened, one is deep within the editing.  Probably glaring errors are noticed that need reworking of the entire piece (including the just "finished") chapter.  It is more that gradually less work is done creating and scripting new ideas more time is spend reworking and refining what is already there.

 With software, it is much the same.  We have NUI.  What is done in opencore is, I think, basically done.  From now on, things will very gradually look much more like Ian's diagram and much less like Zope2/Plone.  This is the case where the next "edit" is already known in form.  But there is still substantial work in opencore (the zope one) that needs to be done.  Of course all of this is a self-reflecting phenomenon:  towards the end of NUI, we were only fixing and not introducing anything new.  We're moving towards this in a bigger scope now, with opencore.  Now the wild frontier is how our software works together.  Someday, that will be solid.  Then??? 

Its the same with writing.  Now, I'm working on the second chapter, getting that reviewable.  Then there will be a long time putting down timelines (admittedly, my novel is much more geeky than some).  Subsequent with that, the rest of the novel will be being writing.  Right now I'm only doing modifications to make things more readable, but at some point there will be "functional" modifications, if you will ("how could they get from point A to point B in X years on a sublight ship?  the acceleration would kill them!") and making the bridges less sloppy.  I'm also working on all the sequels at the same time (there is a definite scope of this work, and I'm still in the elated period where I'm both happy that I've pinned down the scope to something doable and ecstatic that within that scope I've chosen I can express a whole friggin lot).

I hope I've convinced you why writing software and fiction are basically methodologically the same in my mind.  There are some fundamental differences that make these processes look different that I'll note in brief here:

• writing a novel is just for me.  I mean, I hope people read it and I hope that people I love and respect think its good, but ultimately, it needs to satisfy me and I'm the only one that can judge that.  Maybe I'll look back in 20 years after its done and think, "Wow, that's naive."  I probably will, but such is life, and I hope that I can still appreciate it in the same way that I appreciate some of the elegant and naive algorithms I wrote in fortran for my Master's thesis.  I don't think they're great, but given the tools and knowledge that I had at the time, I'm quite proud of them and I learned much from them.  In contrast, writing software for openplans is to get something done.  I care about our users and hope that I can do my best to give them software and a site that will help them do whatever they need to do.  There is of course the personal exploration of ideas that makes programming enjoyable instead of "just a job", but that's just the icing.  The cake is the product.
• Also, novels are about aesthetic ideals.   I have a certain sense that I want to convey.  While I can't read things from someone else's eyes (the best I can do is get them to tell me what they think of things, hopefully being really nasty and critical in the process), what I am trying to do with my writing is convey a certain abstract idea with words.  Basically, to convey something not expressible in words with subtext.  Not only is it probably impossible to tell if I've done this or not, its also probably impossible to weigh the idea(s) i'm trying to convey on any sort of absolute scale. In software, there is generally a specific goal.  While it might not be measurable, it is (maybe?) more easily ascertainable whether the goal is met.  As a very concrete "for instance", if you want a program to find the first N prime numbers the fastest, this is generally easy to measure and debug.  If the goal is "to provide tools to help NGOs better form projects and participate in online communities"...that's harder to measure, but we can define pieces of that and measure.  In both cases, not meeting the goal does not mean the work has failed, as the novel may have some beauty despite failing to convey (e.g.) "immortality", and the software might have some good patterns despite failing as an application server.
  
• Software is a group effort.  Sure, some wonderful bits have written by individuals, but what makes them great is that they're used by other people.  [This is my opinion and I'm sure many a mathematician or programmer would disagree....but from my perspective as a member of TOPP and the open source community].  I work with people everyday to develop software that works well together.  Writing a novel is purely a luxury that I can dabble at and only has to serve me.  So I can make it as complicated and intricate and subtle as I like.  These are all great things for novels and horrible things for software.
• There are different criterion for when a novel and a piece of software is finished.  Usually, with a work of fiction (though increasingly less fundamentally true) there is a goal, a canonical form of the work that the author can say "This is 'The Imperium Cycle'".   Maybe this shouldn't be the case, but I know for me that this is the case.  For  some software,  things are this straight-forward.  Its not "done" unless there is a political reason for it to be done, but its brought into maintenance mode when no big new features will be added.  Other pieces like the linux kernel or the OpenPlans software will always have expanding scope.   Things eventually become more and more stable, better and better methodologies come up.  I'm sure in both cases these pieces of software won't be around in 3000A.D., but they're being developed as if they will be around for a long long time.  Its an open-ended process, which is quite different from a novel in that sense.
  

If anyone has any opinions about such things I'd love to hear them.  I can't believe I'm the only writer/programmer that sees really no difference in methodology of code-writing and fiction-writing.