We don't know what we are doing

In Dries Buytaert's blog, he recently reminded the Drupal community:

So let's capture that thought for future reference. Sweeping changes are required to make major advances in technology, and often times there is a lot of pain before the pay-off

I agree with Dries' premise and want to expand upon it in my own words.

Dries refers to major changes in the Drupal API which break existing modules. Often, Drupal admins/developers get angry about this breakage, and advocate that we avoid major API changes at all costs (we already avoid them where possible). They don't want to edit to their modules every 6 months or so. They want progress and backward compatibility, and don't believe that these are sometimes mutually exclusive.

I think these angry folks are missing a key idea:

We don't know what we are doing

I get the feeling that these folks think that there is a master roadmap, and all that the development team does is fill in the functions as we have time. But thats not how open source works, and especially not the Drupal project. We are making this up as we go along. We do not know what will Drupal will look like a month from now, any more than a jazz musician knows how tonight's set will sound. If we knew what we were doing, we would just design the ultimate flexible, powerful, calafragilistic API and never need to break it.

This lack of foresight leads to some seemingly silly outcomes. For example, menu callbacks in Drupal are basically responsible for the main content of any given page. They are rather central. Initially, these function s were supposed to return their content. Then we changed the API and they were supposed to print their own content. Then, we changed it again and they were supposed to return content again. Some folks wondered aloud - please make up your mind. These folks fail to realize that the Drupal project has no mind, and if it did it would be terribly conflicted.

I do think of the Drupal development team as a jazz orchestra. Players come and go, and have side gigs, and compose their own melodies (i.e. modules). So please understand us when we change directions and sound off key.

Musicians wanted.