OpenCore Software

Robert and I have been working on the opencore feeds. Currently, these are only used by the project summary page, but looking at the site, we have many things that could be feeds: Latest Projects, Latest Members, Recently Updated Projects, just to name a few. This blog post is sponsored by the word ‘agnostic’, where agnostic means “optimistically not incestuous code” and “incestuous” means “code that unfairly presumes coupling or functionality that destroys perceived modularity and forces binding to a framework even when the desired functionality should not depend on the framework”. Okay, those are just words. The idea is that incestuous code is badly coupled code. Its a word I use alot to describe code I don’t like, because something can (perhaps wrongly) be described as modular but be completely incestuous. Agnostic is the antidote (and the antonym) to incestutous.

Sorry for that gratuitous introduction. The point is that I think what we did with feeds is not a bad model:

[So I tried to upload this image through wordpress’s xinha.  Of course, horrible things happened, so now I’m just linking to it.  GUIs are my bane; did i mention I’m now editting this in html mode, as I almost always do, even though basically i want text ]

This is a horrible diagram with symbolism that only makes sense to me. So let me describe. You have a bunch of content (projects, mailing lists, etc) that provide feed data. They may provide feed data in more than one way, but that’s another story. Instead of having the wild west, I make an interface called IFeedData that forces the feed in the standard format. In this way, I have decoupled what the feed comes from from what a feed is. IFeedData (and its items) have some things that are mandatory to define — the essence of what a feed is — and what is optional.

Moving further right on the chart, there are a bunch of templates. Any of these templates can be used to render any feed. Some of them will work better with particular type of feed than others — for instance, if the feed doesn’t have icons, the portrait_feed_snippet.pt is pretty silly; but each can render the basic functionality of “what a feed is”, and may support optional items, conditionally displayed if they exist. To extend functionality, add a new template.

So we have feed providers, which give us feeds, and templates, which display feeds. How are they used?

So we have this viewlet manager, which has a bunch of viewlets. The viewlets have some sort of context which gets adapted to give a feed. The viewlets *also* have something that says “use *this* template”. In other words, the viewlets decide how the feed is displayed.

I’m not writing this to say “ooooh! look at this cool pattern i helped make!” I’m writing this as an illustration of how something that isn’t (amazingly) incestuous can work. Notice you have decoupling on every level. You have content providers which have rules to transform them to feeds. You have feeds, which have various templates that can be used to display them. You have viewlets, which contain the display logic, tell how to adapt their context to the feed, and choose the template. On top of that, its a simple system: abstract enough to handle anything I can imagine atm, but concrete enough to actually use.
There are problems going forward that I haven’t conquered yet, but I am at least comfortable with the pattern.

Robert pointed out that our “microapp” architecture has the potential to become extremely hard to work with and isn’t really giving us any benefits in the process, since we’re not actually decoupling any of the components. I’d like to think more about that, but in the meantime, I’ve been thinking about what I think would be a really compelling use for proper microapps on our site that wouldn’t be increasing our administrative burden in the way Robert talks about.

(Incidentally, just to be clear, when I talk about proper microapps, I mean simple UI-less applications with a very stable API over HTTP that provide a single service intended to be plugged in to other contexts.)

So, we’re trying to use (quasi-)microapps to build our site on the backend, but I think microapps would be most valuable as services we provide *for users* instead. It seems more compatible with the microapp tendency towards runtime configuration over HTTP; while we end up bending the code and our builds to configure these applications to set up our site at “build and compile” time, users could just configure the applications for themselves on an already-running site to gain additional and personalized functionality.

Offered as user facing tools, these microapps would provide extremely flexible functionality beyond what we offer universally — ways for individual users to layer their own custom configurations on top of the site. Transcluder is a simple example; by just turning on a sitewide transcluder, we would allow users to build very flexible componentized webpages. I think an HTTP request hub like Cabochon could be another really powerful service for users: while we handle our own events internally with code and configuration, users could configure a set of individually saved hookups between HTTP events and resources. So one user could set up a rule: “When I publish to the project’s blog, mark my `write blog post` task as completed” while someone else sets up a rule that “When anyone edits this wiki page, create an account page message for me.” Throw in a cron-like microapp to schedule HTTP requests and now users can have a new “Write weekly blog post” task created every week and more.

Admittedly I’m doing a tremendous amount of hand-waving here (authorization? authentication? request interception? recursive requests? — implementation details!) and I haven’t come up with very compelling use cases, but I don’t think it’s impossible and I think the real potential of the idea is that *we don’t have to come up with the use cases* — people could come up with really interesting combinations of events far beyond anything we’d want or need to build into our software.

Another big advantage here is that since these microapps would have very well-defined APIs that work over HTTP, we could actually deploy them with minimal user interfaces and gradually build up UI over time. “Power users” would immediately get to experiment with the tools immediately, and we could even use their feedback to determine common usage patterns to inform the friendlier interfaces. So we can turn on Transcluder without any UI, and people could immediately start using it by editing a bit of HTML; then, as a first pass at UI, we could build a switch into the Xinha linking interface to toggle the “rel=’include’” attribute; and eventually we could design a much more sophisticated UI (perhaps leaving the “design-less” interface available underneath in the same way that WYSIWYG editors allow you to drop down to HTML) but without that design ever being a bottleneck to deployment or usage by at least some people. Likewise an event hub could start out by letting people input URLs and HTTP verbs, and eventually build a descriptive interface which lets people connect “things to do” from a list.

I think this would be a really powerful way to add, essentially, arbitrary flexibility to a site without having to hard-code it, and without having to come up with individual use cases, instead offering the tools to users and letting them come up with their own uses. And since we really could treat it as a layer on top of our software, which we don’t actually have to know anything about the use of, it wouldn’t be adding to our administrative burden, since we wouldn’t have to worry about the connections between microapps. I don’t think it replaces the need for some sort of internal connection between features and data — we can inevitably come up with much more tightly integrated services on the backend, and in any event I’m sure there are commonly useful hookups which we ought to just provide directly. But it would be a great way to provide functionality beyond what we decide to offer or what occurs to us to implement.

Here are some preliminary results from a look at the openplans.org data. I should say that the values are approximate, but I think they should be pretty close to the real values.

Here is what it all means:

mem active - the number of active members that logged in to the site during the previous 30 days

mem avg life - the average number of days a member is active (estimated from all members who are now dormant)

proj active - the number of active projects who have had their wiki edited during the previous 30 days

proj avg life - the average number of days a project is active (estimated from all projects who are now dormant)

ml active - the number of mailing lists which received a post to the archives during the previous 30 days

ml avg life - the average number of days a mailing list is active (estimated from all those that are now dormant)

One thing to note is the sharp drop in active members in November. This is perhaps, in part, due to a migration of members from openplans.org to nycstreets.org. Also, of all the data points, the December value is the one I trust the least due to the way things are approximated. In January I plan on doing a bit of work to verify the accuracy of these estimates.

The mailing list activity is higher than I expected. I checked and made sure that this isn’t due to spam (spam doesn’t actually make it into the archive.)

So this is just the start of what we can do to analyze our usage information to help us make good decisions about the evolution of our software. From here we can go on to analyze Task Tracker and WordPress usage.

Once again, I’d been thinking of making a blog post about what I’m working on (short version: I’m taking a break from REST configuration (I’ll leave the puns as an exercise for the reader) to work on what basically amounts to a server-side optimization for the Vespucci project) but I find myself more interested in this discussion on the OpenCore dev list. I thought about making this post an email to that list, but the comments I’m planning to make are basically tangential to the thread there.

What I take away from that thread is that we the OpenCore developers are planning on making people on openplans.org more like projects, with their own featurelets (mailing lists and task trackers and such) to go along with the wiki that’s currently provided. This seems kind of weird to me; as my understanding is that openplans.org is about projects and making it easy for groups with similar goals to collaborate and share skillsets and experience. It’s not clear to me how letting a person track personal tasks or run a personal blog is helping to accomplish that. Instead, I would think a generalization of my earlier thoughts on blogging would be more appropriate: have tabs on the user account page for each of the available featurelets, but show them as a filter on the entire site rather than unique content. So, the task tracker tab would show only tasks assigned to the user, the mailing lists tab would show threads the user participated in, etc. That would let you see things from a people-oriented point of view without creating this kind of island of content where the user has their own personal stuff on a site that’s supposedly intended to help them share with others.

Of course, I have no idea whether this really makes sense. I often find that ways of doing things that make sense to me are kind of lost on non-developers (like when I bitch to Windows users about how hard it is to change file extensions on their platform and they stare blankly and wonder what possible reason you could have to want to do that). But, I’m not a developer on OpenCore so hopefully my perspective on things isn’t too tainted by elbow grease.

Another thought that occurred to me while thinking about this is that if people are projects, and people can be members of projects, then OpenCore obviously supports having projects as members of projects. This excited me more than a little (being a computer geek, I of course love hierarchies!) I think if that sort of nesting of projects is allowed then you can structure things in a way that makes a lot of sense. For example, you can have an Organization be an entity recognized by openplans.org in the same way that People and Projects currently are. An Organization could probably be nearly identical to a Person, but with some string changes (an organization has a logo, not a photo, and services rather than skills, etc.) Of course you can’t log in as an organization either. Organizations and Projects could have as members People, Projects, or other Organizations (think of local chapters or subdivisions for Organization->Organization, and planning committees for Project->Organization) ), and People wouldn’t be allowed to have any members, of course. The neat thing that I see here is that the membership wouldn’t have to be exclusive; in the same way that People can be members of multiple Projects, Projects could be sponsored by multiple Organizations. Organizations could have multiple parent Organizations as well (like Geoserver could be both a subdivision of TOPP and a member of OpenGIS).

Anyway, like I said I’m not really aware enough of OpenPlans to know whether any of what I said is really applicable. To follow in the recent trend of posts on what success is for openplans, I’d say it’s probably not measured by how well the software models the relationship between people, projects, and organizations But I do think that modeling them well makes it easier to present them in a navigable way (that is, making it easier to find projects that are related in the types of ways our software knows about). If our goal for openplans.org is to bring projects together, then it’s probably a step in the right direction.

Sobering thought for the day… there are two kinds of people: the disabled, and the temporarily abled.

With that in mind: “Degrading gracefully” for non-javascript browsers isn’t just a theoretical nicety. For some people it’s the only way they can use a website at all.

I’m a javascript novice, so I hardly know a lot about the best practices here, but there’s some interesting commentary out there if you google a bit. For example, “Test your pages, you wanker. With assistive technologies. And real disabled people. Or you’re a bigger nob than you look.”

Neither am I an expert on accessibility, disability rights, etc. (but my mother-in-law is).

In a funny way, we’re lucky that our current functional testing technology doesn’t support javascript, because it ensures that things mostly work.  But for a concrete example - the one that prompted this post - can flunc “see” a textarea if you style it as hidden by default? I suspect it doesn’t care about css-driven visibility; all it does is find the element in the document.

As we move out of the Dark Age of Kupu and into the Enlightened Age of Xinha we are faced with an interesting set of issues and opportunities: not only do we have to determine the degree of user control we allow but also—and to some degree as consequence to that—we have to question certain fundamental assumptions about the (in)appropriateness of the WYSIWYG model. For other information regarding our WYSIWYG decisions see: What WYSYWIG is Up To and the WYSYIWYG Wishlist.

User Control

In recent conversations with Jackie I argued for limiting (though I prefer to say managing) user options to increase user happiness. My justification for this is best described by the adjacent line graphs.

­Our editor should make it easy for our users to create documents for the web, but it should not provide so many options as to be stressful or confusing. As Kathy Sierra says, “the amount of pain and effort should match the user’s perceived payoff”. Our users want control and ownership of their pages, and that is something with which we provide them, but at what point do we provide so many options that the user no longer feels in control of the page?

We need to tailor the editor to fulfill the needs of our users: basic word processing, including basic inline type styling (bold, italic, underline, strikethrough, color); list creation (ordered and unordered, definition lists(?), and nesting); table creation and management (table stying, row and column manipulation); image inclusion; hyperlinking and page creation (wicked, internal, and external should be consolidated). But, we don’t need to provide much more… in fact we probably shouldn’t provide things like: font families, absolute font sizes, perhaps even text alginment (left, right, center)…

The WYSIWYG Model

…because the WYSIWYG model, as we have been thinking about it, may not be the ideal way of fulfilling these (perceived) needs. As Luke and I discussed it: the WYSIWYG model is predicated on emulating the experience of a typical word processor, specifically in regards to designing for presentation, and thus fails engage with the medium that is the web. As stated above, our editor should make it easy for our users to create documents for the web. The concept we have been working with emphasizes presentation (how things look) when it should equally emphasize structure (how things work) and semantics (what things mean).

To provide our users with a better experience we need to better understand web standards (like A List Apart) and incorporate all three modes of thinking/editing into our editor in a way that is easy for users to understand. WYMEditor, an example Luke pointed out, works on a WYSIWYM model and emphasizes the meaning of content over its presentation.

The WYSIWYM Model: Ideas About Implementation

It’s possible and desirable to implement core elements of the WYSIWYM model in our editor in several ways:

+ Clean Markup

Example: If something like text-alignment is crucial (which it’s not) then non-exclusive block-level CSS classes are the place where it should be defined, not the “align” or even “style” attributes.

Producing acceptable (X)HTML is perhaps the technically most difficult yet also more important thing for our editor to do. There is little point in focusing on the presentational niceties if the markup won’t render correctly anyway. The markup produced should strive to be structurally and semantically proper, conform to standards, and not be contaminated by visual information like font styles and weights, borders, colors, etc.

We can accomplish by limiting certain tags (like the evil “font” tag), running operations on the output markup to ensure a base level of acceptability, and outsourcing visual information to pre-defined CSS classes that are applied at the block level wherever possible. Ideally the editor could even be aware of the block-level context and provide only those styles which are appropriate to that block type (for example table styles for tables, list styles for lists, etc). By constraining users’ choices intelligently we make it easier for the user to get things done. Furthermore, if we don’t provide clean markup then any future support for theming or alternate styles will result in extreme pain for both us and our users.

+ Semantic/Structural User-end Views

The editor should make the structure of the page transparent to our users, either by default or through a toggle (like the ¶ button in Microsoft Word). As you can see from their demo, WYMeditor does a good job of this (they clearly leave out presentational elements as they are committed to a “pure” WYSIWYM model). This level of information benefits our users because they learn to think of the web as distinct from print media and enables them to build pages that will function across themes and Deliverations without breaking.

We can accomplish a similar effect using CSS specific to the editor context that adds borders, backgrounds, margin, padding, and (vernacular) labels to the block-level elements as we determine to be appropriate.

Wiki Pages vs Collaborative Documents

As an aside, we should have the editor be somewhat context dependent. Quoting/paraphrasing an earlier comment Cholmes made on the the WYSYIWYG Wishlist: when people are in the collaborative document editing space it’d be nice if they could have access to the full range of fonts, sizes, etc. because that’s the area we want people treating it like a word processor, not the wiki pages. We’d have to discuss how we would make collaborative documents migratable to wiki docs from this perspective as well as a variety of other issues specific to the differences between these two document types.