• Managing Downtime

  last modified July 8 by nickyg

This page intends to gather our collective knowledge and form a strategy for managing downtime on our various websites. Please add specific notes to any section as you see fit.


When to schedule downtime

There are many factors when deciding when to schedule downtime. The ones that seem most important are:

1) low-traffic times. We should look at traffic patterns and choose a time when the least # of users will be affected.

2) "eyes on the street" -- while evening maintenance makes sense in terms of traffic, we want to be sure that we will be around to monitor the effects afterwards -- that means making sure that people will be around to watch the sites for a sufficient amount of time after returning from maintenance.

"Down for Maintenance" page

We always need to have a "Down for Maintenance" page available when we go down for scheduled maintenance. We need to ensure that this page will get shown in response to every possible request (www.openplans.org and www.openplans.org/projects/myproject/blog -- during today's downtime it was only showing up at www.openplans.org). Ideally, this page should be informative and have a sense of humor, like the Bloglines Plumber.

Implementation details:

  • [please fill in]


Advance Warning

Any downtime that exceeds a certain amount of time (what amount is debatable -- perhaps 5 minutes, maybe less) should have some sort of advance warning for our users. Options include: an email to all users (giving an approximate estimate of when downtime will occur), an email to opt-in users, a notification bar at the top of the site, an account page message (not likely to be very effective) or a news item. In the #topp channel, Arne pointed out that we want to be careful not to *only* email our users when something bad is going to happen, which is a fair point (my preference would be to send out more frequent positive emails to counter this). The more extensive the planned downtime, the farther in advance we should give the notice.

 

Email during downtime

For opencore sites, we need to ensure that emails that get sent (particularly to listen lists) during downtime are handled gracefully. In the past, these have been queued up and sent after the site returns (today, they were bounced).