Last night Roland ran the database maintenance on newer hosted service sites powered by Drupal 5. We are looking at doing another pass soon, because today we noticed there are a few other things about the newer databases that we can tune up.

While some sites are singing along happily today, we did notice that sites that use the Image module heavily, especially those that use the Random Image block, are suffering performance issues. Today we upgraded the version of the image module to its latest official release, which seems to have fixed an issue with thumbnailos, but I'm not convinced that it improved performance much. Specifically, sites that use the Image module in Drupal 5 are experiencing Internal Server 500 errors, but for reasons different, I suspect, than database performance issues that plagued us in the recent past. We are continuing to investigate this specific issue.

After running the database maintenance, details of which are below, on over 350 sites over the weekend, we noticed an increase in speed for our flagship site bryght.com (its database was reduced to 1/10 its original size!) as well as some of the sites we targeted for special attention. We're still waiting for word from those we've contacted specifically, so until we hear from them, we're not ready to say what we've done has successfully rooted out the 500 error problems people have been seeing on the hosted service. (We're waiting to hear from them as they are the people who watch over the site the most.) This evening we'll run the database maintenance on our newer sites, but we don't expect to see as much improvement as we did with the older sites, partly because the newer sites handle their individual databases better, and because at the outset we had the "Database Logging" module disabled by default. The equivalent module on older sites was filling up databases, making them run slowly for many operations.

A note on what we did to older databases, which we'll also do for newer sites tonight: we cleaned out the following database tables, which don't contain any content but rather user login sessions and old system logs: accesslog, watchdog, and sessions. Clearing out the last one means everybody on the site will be logged out. We hope that the inconvenience of having to login again is outweighed by a faster-running site overall. About 180 sites will undergo maintenance tonight, but like I say, we don't expect the performance gain to be the same as with older sites.

At any time during the day, please let us know if you encounter Internal Server 500 errors and what you were doing when it happened so we can take a deeper look. We will update this post with a comment when we're done with tonight's maintenance, and follow up with a post tomorrow morning.

We have the database maintenance scripts written, and this morning we tested them out on one of our older sites (that we built for ourselves). We did say that we would do the maintenance overnight, but we were so happy with how fast and smoothly the test went that we decided to run it on 100 active sites on this, a Friday afternoon, and evaluate from there. We make backups on every site before proceeding with the maintenance, so no content will be lost.

If you see something unusual or unexpected on our site, please continue to report it and we'll look at it right away.

Yesterday morning everybody on the hosted service experienced their site going down, and since then sites have experienced numerous 500 errors. (Some sites have experienced this off and on for the past few weeks as well.) We are currently finishing with the checks of all databases on the hosted service, and then we will run some proactive maintenance—i.e. deleting the data from sessions, accesslog and watchdog tables that have not been automatically cleared like they should have. Don't let that stop you from reporting errors you see that you think are out of place. If you see "500 error", please send in a report with the URL of your site and we can double-check that you're on the list for the planned maintenance.