Migrating IMAP mailboxes using Linux and Mailsync

IMPORTANT NOTE: This is a “running blog” today, updated as I complete several steps. It will continue to change until I’ve nailed this process down — I just wanted it recorded, and my blog seems a good spot to put up technical documentation.

This is my personal how-to on how I am migrating users from Groupwise to Cyrus Mail. I know this may sound backwards to some people! If it sounds odd to you, guess you’re not a free/open-source software aficianado.

Background

We were given a directive by our CIO that was quite simple. “Replace Groupwise. We’re done running Novell stuff on our network. If you choose an alternative where we can own the source code, you may implement it without question. If you choose a proprietary alternative, you must provide written cost justification.”

Exhibiting the enthusiasm that all systems administrators show for options that require cost-justification, I, of course, chose the free software option 🙂 Spencer Tuttle did the initial Cyrus implementation and LDAP integration (a process well-documented elsewhere on the web), and eventually it fell to me to figure out how to migrate the Groupwise mailboxes to Cyrus. This, unfortunately, is much tougher than it sounds. Yeah, there’s an easy way, but given the choice between doing it the easy way and relying on users to get it right, and doing it the hard way but being in control of the process and knowing it’s done right, I’d rather be in control.

Products evaluated

  1. Mozilla Thunderbird. This is our chosen mail client for post-Groupwise life, so we figured we could just have all the users log into both their Groupwise and Cyrus mail accounts at the same time, and copy the folders over that they want to keep.

    Upsides:

    • It just works™. You can log into both mailstores and drag folders.
    • Users gain complete control over what gets moved.

IMPORTANT NOTE: This is a “running blog” today, updated as I complete several steps. It will continue to change until I’ve nailed this process down — I just wanted it recorded, and my blog seems a good spot to put up technical documentation.

This is my personal how-to on how I am migrating users from Groupwise to Cyrus Mail. I know this may sound backwards to some people! If it sounds odd to you, guess you’re not a free/open-source software aficianado.

Background

We were given a directive by our CIO that was quite simple. “Replace Groupwise. We’re done running Novell stuff on our network. If you choose an alternative where we can own the source code, you may implement it without question. If you choose a proprietary alternative, you must provide written cost justification.”

Exhibiting the enthusiasm that all systems administrators show for options that require cost-justification, I, of course, chose the free software option 🙂 Spencer Tuttle did the initial Cyrus implementation and LDAP integration (a process well-documented elsewhere on the web), and eventually it fell to me to figure out how to migrate the Groupwise mailboxes to Cyrus. This, unfortunately, is much tougher than it sounds. Yeah, there’s an easy way, but given the choice between doing it the easy way and relying on users to get it right, and doing it the hard way but being in control of the process and knowing it’s done right, I’d rather be in control.

Products evaluated

  1. Mozilla Thunderbird. This is our chosen mail client for post-Groupwise life, so we figured we could just have all the users log into both their Groupwise and Cyrus mail accounts at the same time, and copy the folders over that they want to keep.

    Upsides:

    • It just works™. You can log into both mailstores and drag folders.
    • Users gain complete control over what gets moved.

    Downsides:

    • It requires training on the part of every user.
    • If the user screws it up (deleting the folder, for instance), I’m left holding the bag to restore a Groupwise mailbox from backup. That’s really something that’s painful and time-consuming! Particularly if they continue to receive messages between the incident and the mailbox restore.

    All things considered, I’d rather avoid this option.

  2. imapcp, the IMAP Copy Tool.

    Upsides:

    • Written in Python. I know Python a little, and it’s pretty easy to read and debug.
    • A single script, rather than a hard-to-compile collection of utilities.
    • Straightforward usage. Typing “python mbcp.py” gives you a brief help, and the README provides sufficient instructions for synchronizing your mailbox.
    • Copies folders as well as messages, so you can just point to, say, INBOX, and get all the subfolders.

    The downsides:

    • The single biggest downside to this, and I need to stress this for all the Googlers who are coming here searching for mbcp.py: THIS SCRIPT DOES NOT WORK!. I’m trying to hack it now, but at the moment it creates mailbox names with the first letter truncated, fails with many obscure errors for no apparent reason, and appears to build some kind of table in memory, leading to crashes due to out of memory errors on very large mailboxes.
    • It doesn’t do differential comparisons between mailboxes, just a blind copy. Which means if you have the messages already in your box, you’ll get ’em again if you run the program again.
    • It doesn’t seem to handle the “alternative namespace” we use in Cyrus Mail very well. It crashes, actually, trying to create more folders.
    • It doesn’t read Groupwise mailboxes; it appears the Groupwise mailstore occasionally stores messages with an odd message ID (or something, haven’t fully debugged it). The program dumps out right after reading the Groupwise INBOX.

    The problems with this script prevented me from using it, although I might revisit it if I can’t create a superior working option.

  3. The IMAP Migration Tool

    Upsides:

    • With just a little hacking, this script seems to mostly work.
    • Follows a disconnected mode of operation: define your parameters, then set up cron (or whatever) to actually run the shell script and migrate the mailbox.

    Downsides:

    • Required hacking to make it work 🙂 But I think I’m resolved that this will be the case no matter what.
    • Created a unique set of interdependent shell scripts and PHP for management of each mailbox. It seems to be geared towards a “one user at a time” approach rather than all 160+ users in our corporation.
    • It seemed sloppily coded and documented. For instance, the fact that it deletes and expunges all the messages in the old mailbox without checking that the message was actually migrated is a big downside, and completely undocumented. I could hack around it, but would rather not if I can find a tool that doesn’t have this unfriendly behavior.
    • No two-way diff mechanism to synchronize boxes rather than a blind copy.

    Overall, it seems a really nifty piece of coding, and is pretty cool in that it invokes PHP from a shell script for processing (leveraging the strengths of both languages), but combining hard-coded nuking of old messages with blind copying and no error checking makes for a package I’m uncomfortable using.

  4. Mailsync. This little utility is designed to keep various mailboxes synchronized in varying formats, including IMAP and POP.

    Upsides:

    • Very mature, been around many years.
    • Has non-live test modes to enumerate mailboxes.
    • Can handle arbitrary namespaces.
    • Reasonably well-documented and well-commented code; at least, it’s better than the alternatives.
    • Is the only utility I’ve found that can be run multiple times without copying the same messages over and over. It’s designed to synchronize stuff like CVS does — and that’s pretty cool.

    Downsides:

    • Relies on a library called “c-client”, which is turning out to be a real bummer to compile and make work, particularly if you want SSL support. I eventually gave up trying to compile this library. Even though I’d built it, and pointed mailsync’s “./configure –with-c-client=/some/path/” appropriately, it refused to recognize it as a working c-client. Instead, I just installed the “imap” and “imap-devel” libraries from my Redhat 9 CD, and they worked fine.
    • Little support to speak of. The maintainer of the project really, really doesn’t want you emailing him for free support (and I empathize!). The sole mailing list runs about 2 or 3 posts a month, and is housed with the Project Home on Sourceforge.
    • Lots of unnecessary complexity for our task — we just want to sync up some IMAP mailboxes, not handle tons of other mail formats.
    • Confusing configuration file syntax. I’m sure I’ll get a handle on it with usage, and like Fetchmail eventually come to regard it as “normal”, but right now it’s fairly cryptic.

Important tips:

  • It appears that cyrus and the imap package (yeah, c-client is just the usual “imap” package on redhat) conflict. So you can’t run Mailsync on your Cyrus mail server.
  • If you are running Redhat9 and have the imap and imap-devel packages installed, getting mailsync compiled relies on the standard five-step:
    1. tar xzvf mailsync_5.1.0.orig.tar.gz
    2. cd mailsync-5.1.0
    3. ./configure
    4. make
    5. (And, if you wish)

    6. make install
    7. Me, I chose not to install it, and instead ran it out of the src/ directory.

  • You need to be triple-careful with your .mailsync file in your home directory. I recommend in the strongest possible way that you test this on a non-important mailbox first! Otherwise, you are nearly certain to regret it.
  • If you’re using Cyrus with an alternate namespace, your delimeter on mailboxes is a “/” instead of a “.”. This is very important! Some scripts assume, if the IMAP server says it’s Cyrus, that it uses the Cyrus default namespace separator of “.” which just isn’t so in our case.

EDIT 5 September 2003: Unfortunately, after getting mailsync working fine, I discovered it only can import/export folders that are already set up with the exact same folder names on both sides. For instance, you can have a /sync folder in IMAP where you store messages you want to sync, and that will work. In this case, though, we need it to propagate folder names as well. I’m sure I can do manual c-client commands to create these folders and then mailsync them (and I may yet do that), but at this point just debugging the problems with imapcp seems easier.

Beauty

I had a wonderful Labor Day weekend, and enjoyed my time with my family and children. We celebrated Zach’s sixth birthday on Sunday with grandparents, cousins, aunts, an uncles. I had a chance to meet up with a friend from a common mailing list in person, and he was very interesting and fun to talk to — can’t wait to do it again, with more people. We went to the park tonight and played while picnicking with a neighbor family. I’m just about to retire for the night and snuggle with my beloved.

Of course, due to all this activity, I had zero blog entries 🙂

“Beauty is the bait which with delight allures man to enlarge his kind.” –Socrates

“The pursuit of truth and beauty is a sphere of activity in which we are permitted to remain children all our lives.” –Albert Einstein

“Youth is happy because it has the capacity to see Beauty. Anyone who keeps the ability to see Beauty never grows old. –Frank Kafka”

My life is beautiful.

I had a wonderful Labor Day weekend, and enjoyed my time with my family and children. We celebrated Zach’s sixth birthday on Sunday with grandparents, cousins, aunts, an uncles. I had a chance to meet up with a friend from a common mailing list in person, and he was very interesting and fun to talk to — can’t wait to do it again, with more people. We went to the park tonight and played while picnicking with a neighbor family. I’m just about to retire for the night and snuggle with my beloved.

Of course, due to all this activity, I had zero blog entries 🙂

“Beauty is the bait which with delight allures man to enlarge his kind.” –Socrates

“The pursuit of truth and beauty is a sphere of activity in which we are permitted to remain children all our lives.” –Albert Einstein

“Youth is happy because it has the capacity to see Beauty. Anyone who keeps the ability to see Beauty never grows old. –Frank Kafka”

My life is beautiful.

EjectEjectEject: On Individual Responsibility

I just finished reading an excellent, lengthy essay by Bill Whittle, on the topic of individual responsibility. If I get permission from him, I will reprint it here. If you plan on reading it, you need to allocate probably about an hour (for a reasonably speedy reader) to get through the essay and all the comments. Quotes follow, not necessarily in order in the essay (emphasis mine):

I just finished reading an excellent, lengthy essay by Bill Whittle, on the topic of individual responsibility. If I get permission from him, I will reprint it here. If you plan on reading it, you need to allocate probably about an hour (for a reasonably speedy reader) to get through the essay and all the comments. Quotes follow, not necessarily in order in the essay (emphasis mine):

“They, like me, call themselves conservatives, but we are indeed a new breed: pro-choice, pro-gay, vigorous defenders of equality of race, religion, gender and sexual orientation. We’re big on freedom and big on responsibility. The left hates us. We are harder to attack than the racist, homophobic, misogynists that they formerly could comfortably lambaste as right-wingers … Today’s politics are more like a Rubik’s cube, where someone you may stand shoulder-to-shoulder with on one subject, can become, with a simple twist of the issues, a bitter opponent in some other fight.”

“Individual responsibility frees us from our past, from the fate of our birth, from the millennia of class and caste and of failed ideas that have kept so many in bondage for so long. If we indeed do have the ability to control our own selves, then we can free our own minds from the river of history and experience.”

“There were some major problems with Frontier Justice: it was brutal, it was often error prone, and once made those errors could not be corrected by cutting down the offender, apologizing, and sending him on his way. But Frontier Justice did have one immeasurably attractive virtue. It understood, in a way we are rapidly forgetting, the difference between perpetrator and victim. It realized that the former started into motion a chain of events, and that all of the consequences could therefore be laid at the feet of the individual person committing the crime … Give your responsibility to the group, and you give your freedom to the group. Freedom without responsibility becomes, very rapidly, a farce. When laws become farcical, the result is anarchy. Anarchy is unacceptable, so measures are taken to reduce freedom and increase controls on the population. That is precisely what is happening at full gallop.

(Pardon my censoring of this excerpt; go read the original essay if you want the full effect. And yes, I catch the irony of this statement. -MPB)
“To those who want to limit speech they see as hateful, I can only utter these simple words of protest: Go straight to —-ing hell you miserable authoritarian —–uckers! Forgive me, I know that offended some of you. But remember this: words are words. They are encapsulated ideas, and the only harm they can do us is the harm we ourselves allow them to do us … The defense against hate speech is not to put our hands over our ears, our eyes, and someone else’s mouth. The way to fight this human virus is to do what we have been doing: hold those who use such language up to ridicule and scorn, to use our own words as a people blessed with freedom of speech, and to let such archaic and diseased notions and epithets die a quick death in the marketplace of better ideas.”

“I promised I would tell you who is responsible for the mess we find ourselves in.

Proceed into your bathroom and take a long, hard look in the mirror.

I also promised to tell you who can get us out of this fix. Well, keep looking. While you’re looking, make a decision.”

Essays like these remind me that freedom of speech is alive and well in the U.S.A., and to fail to exercise it is to participate in its destruction.

Hotmail Out Of Memory

Just had a funny entry scroll past my window. We use sender address verification where I work to confirm that a sender, in fact, exists at the domain from which the mail is purported to be sent. Some spam just came our way claiming to be from hotmail, so of course Postfix (our mail transfer agent) ran out to Hotmail to make sure the user existed. What do we get on the reply? (a couple names changed to protect our network)

Aug 28 14:18:32 bubba postfix/smtpd[6705]: 1AA1FB8024: reject: RCPT from bay4-f28.bay4.hotmail.com[65.54.171.28]: 450 <maukc@msn.com>: Sender address rejected: unverified address: host mx2.hotmail.com[65.54.252.230] said: 452 Out of memory (in reply to MAIL FROM command); from=<maukc@msn.com> to=<pcoker@bankloan.com> proto=ESMTP helo=<hotmail.com>

Hotmail ran out of memory? They have to be getting slammed by something, they run some pretty beefy mail servers.

Just had a funny entry scroll past my window. We use sender address verification where I work to confirm that a sender, in fact, exists at the domain from which the mail is purported to be sent. Some spam just came our way claiming to be from hotmail, so of course Postfix (our mail transfer agent) ran out to Hotmail to make sure the user existed. What do we get on the reply? (a couple names changed to protect our network)

Aug 28 14:18:32 bubba postfix/smtpd[6705]: 1AA1FB8024: reject: RCPT from bay4-f28.bay4.hotmail.com[65.54.171.28]: 450 <maukc@msn.com>: Sender address rejected: unverified address: host mx2.hotmail.com[65.54.252.230] said: 452 Out of memory (in reply to MAIL FROM command); from=<maukc@msn.com> to=<pcoker@bankloan.com> proto=ESMTP helo=<hotmail.com>

Hotmail ran out of memory? They have to be getting slammed by something, they run some pretty beefy mail servers.

Autism or Demonic Possession?

Minister charged with abuse in boy’s death

A church minister was charged Tuesday with physical abuse of a child in the
death of an 8-year-old autistic boy who died as church leaders tried to
heal him at a storefront church in Milwaukee.

This so-called “minister” sat on a kid for two hours, suffocating him to death, in trying to drive out the “demons” that caused his autism. The mother of the boy helped hold him down as he was murdered in the name of religion. And people wonder why I insist that rationality, not emotion or the supernatural, govern my life.

Minister charged with abuse in boy’s death

A church minister was charged Tuesday with physical abuse of a child in the death of an 8-year-old autistic boy who died as church leaders tried to heal him at a storefront church in Milwaukee.

This so-called “minister” sat on a kid for two hours, suffocating him to death, in trying to drive out the “demons” that caused his autism. The mother of the boy helped hold him down as he was murdered in the name of religion. And people wonder why I insist that rationality, not emotion or the supernatural, govern my life.

Far worse tragedies have occurred in the name of a god, but none so recent as of this writing.

My daughter is turning 8 years old in a few months. This boy was 8. It makes me wish some form of divine Justice would intervene to show this minister what a horrible death he gave this boy, to have the life crushed out of you by an overweight sweaty man screaming at the top of his lungs.

Jim Stingl has something to say about this case. Here are some compelling excerpts:

Parents of children with autism will tell you how challenging and frustrating it can be to deal with the disorder and the behavioral problems it causes. Terrance’s mom, Patricia Cooper, apparently thought she found a shortcut.

She helped hold her 8-year-old son motionless while church elder Ray Hemphill, who admits to having no formal theological training, lay on the child to drive out the evil spirit that wasn’t there in the first place. For two sweaty hours, Hemphill ordered demons to leave Terrance. When he was finished, the poor boy had suffocated and was soaked with his own urine from the ordeal…

You can almost imagine God looking down on the twisted scene and saying, keep me out of this. If you go to this church and you have heart disease or you need a liver transplant, you might want to keep it to yourself…

Hemphill’s brother, Faith Temple Church Pastor David Hemphill, said the church would not change the way it operates. He sounds insulted that anyone would question him or his church, even when a boy is dead.

The minister should not be faced with felony child abuse. He should be prosecuted for murder, homicide, or manslaughter. Belief in the supernatural is no excuse for killing another person.

Pitch correction

Posted to this article on Slashdot today, and I figured I’d like to archive it somewhere useful…

Tuning, pitch, and scale are closely correlated. The two most common “tunings” in the western world are even-tempered and Pythagorean. The most common “scales” are Ionian and Aeolian (major and minor), with Dorian and Phrygian sometimes chiming in on popular music, but rarely others. Other cultures offer non-pentatonic scales with sometimes only five notes. I’m not confusing pitch and scale. I’m explaining that often pitch correction is necessary, particularly in some unusual recording situations, due to the conflict between modern even-tempered 12-tone tuning of certain instruments and the natural instinct of a singer or inexact-pitch instrument (such most strings, which depend on finger position for pitch, and some woodwinds where one can slightly adjust pitch via jaw tension) to gravitate towards a sweeter, non-logarithmic tuning.

Posted to this article on Slashdot today, and I figured I’d like to archive it somewhere useful…

Tuning, pitch, and scale are closely correlated. The two most common “tunings” in the western world are even-tempered and Pythagorean. The most common “scales” are Ionian and Aeolian (major and minor), with Dorian and Phrygian sometimes chiming in on popular music, but rarely others. Other cultures offer non-pentatonic scales with sometimes only five notes. I’m not confusing pitch and scale. I’m explaining that often pitch correction is necessary, particularly in some unusual recording situations, due to the conflict between modern even-tempered 12-tone tuning of certain instruments and the natural instinct of a singer or inexact-pitch instrument (such most strings, which depend on finger position for pitch, and some woodwinds where one can slightly adjust pitch via jaw tension) to gravitate towards a sweeter, non-logarithmic tuning.

It appears you’ve never done harmonic analysis of choral music, or tried to match an accompaniment to an in-tune choral arrangement when said piece was first performed a cappella. Any competent digital piano will allow you to change tunings (note: NOT change pitch, A=440 all the way here) to match the harpsichord needs of pre-Baroque pieces or gain the sweet sound of a perfect Pythagorean chord.

If a piano is tuned to the Pythagorean scale in, say, the key of B flat, trying to play a piece in C major on the same piano without retuning will sound horrible. This is perfectly well-understood in the music community. If you wish to play an even-tempered instrument in multiple keys, you accept a slight dissonance across all ranges of the keyboard in exchange for the flexibility of playing in any key without unbearable dissonance. It is perfectly possible, and often done even today with harpsichords, to tune a keyboard instrument to a non-even-tempered scale in order to provide “perfect” consonance in playing pre-Baroque period pieces.

Now on to the rest of your nearly-coherent rant:

Good singers have perfect pitch

Baloney. You can be a good singer with good relative pitch. “Perfect Pitch”, as inexpertly named for this article, is a totally different thing from singing in tune, or having good relative pitch. Given that I mentioned “imperfect pitch”, above, I stand by what I said: all singers have imperfect pitch. They will not always nail the note perfectly, particularly at the end of an exhausting recording session. There will be times that pitch correction is welcomed as a practical measure in many vocalist’s lives. There are, of course, purists who will raise holy hell if someone were to pitch-correct them.

Since when does a key change sound awful?

If your instrument is even-tempered, key changes within a piece do not sound awful, although there is a slight dissonance to this tuning. If you are using a natural temperament or other alternative, sweeter tuning, it will sound awful in other keys, particularly if those keys don’t have a fundamental on the major fourth or fifth with few accidentals versus the primary scale. Since you are obviously a complete novice to the understanding of tuning systems, allow me to recommend checking out this brief talk on “Math and Music”. These days, we’ve taken the even-tempered scale a bit further by using logarithmic tuning devices rather than simply dividing octaves by 12, but even those tuning devices are not quite “perfect” when tuning a piano. You need to stretch the octaves on the upper regions of the piano in order to avoid perceived dissonance on the part of the listener, and that is a skill that takes a long time to master.

It is not and has never been called the Cher Effect. Its called over compression.

OK. I believe you. No, no, actually, I don’t. It’s very often referred to as “The Cher Effect” when you have fast response times on pitch correction (or vocoder) that force a slur into an unnatural abrupt pitch shift that sounds electronic. Welcome to reality, dude, it’s what an awful lot of people in the pro recording circles I frequent call it, and the moment anybody mentions “the Cher Effect”, nearly everyone knows exactly what they’re talking about.

And compression has absolutely zero to do with pitch adjustment. I think I can safely assume you’ve never been caught late at night in the hypnotic glare of the lights on your audio equipment, compressor readouts gently bouncing to the soft knee you set to manage volumes on the last step of your effects chain as you dump to your mastering deck. Compressors are wonderful, useful (and today, somewhat over-used) pieces of equipment — but they don’t effect pitch, just volume.

Nobody in their right mind really thinks bullet time happened (Matrix fans can flame me later), but correcting a lack of ability and passing it off as ‘quality’ is just plain dishonest.

Bullet time is an example of art in action through technical excellence. It’s over-used today, but nevertheless it takes skill, preparation, and knowledge to get it working right. It’s but one tool in the arsenal of the special effects master.

I was a music theory & composition major in college. I admit that I lean more toward the engineering & composing side of things, as my performance skills are merely above average. The job of the sound engineer is to make the piece sound perfect, listenable, and balanced, and pitch correction is just another tool in the vast array of options we have available to us. Pitch correction is neither dishonest nor is it correcting a “lack of ability”. It’s just part of making a song perfect.

Would you tell the director of a movie he can’t use blue screens because that’s “dishonest” and the performers should be able to do in real life what is portrayed on-screen? Of course not. I admit singing is a different field, but the principle applies. Performers are selling entertainment and illusion; if people are entertained, the artists have done their job well.

Their “natural” sound is talent and ability. Your sound may be fun or interesting, but the reason others thrive is because they don’t need use technology to sound good.

When I was referring to the natural sound, what I meant was the gritty sound, the not-quite-perfect, late-at-night-and-six-beers-down singing that we all get down to. There are people that really enjoy in-your-face music that has nothing extra on it. While it’s cool to listen to, and I enjoy a lot of that music (heck, I’m a Garageband member with many reviews under my belt, I love hearing raw talent), I also enjoy technical excellence and applaud the engineers who elevate the level of already outstanding performances to perfection. Some folks don’t appreciate that. And that simply boils down to taste. So yeah, some artists thrive in simple performance of their tunes, free of adornment. Others get their satisfaction from the engineering and compositional skill of the product. I’m in the latter camp, happily — and that’s something up to personal taste.

Real artists neither want nor need pitch corection.

More baloney. Composers are artists, and often want the performers of their works pitch-corrected. Vocalists sometimes find certain passages impossible to sing, and after fifteen takes are just sick of trying to get it perfect and welcome the pitch correction. Guitarists miss a bend halfway through a solo that they can’t easily repeat in the studio. Hired backup singers turn out to have missed a note, but are already out of the studio. A singer with no natural vibrato may want some added to a particular passage, or one with too much vibrato may want it reduced. The saxophone player you hired for your session may have been slightly sharp the whole time. Whatever the reason, there are plenty of uses for pitch correction, and it is used on many, many professional products these days.

The human brain may be good at sensing when something’s not right, but the competent engineer first gets a take that’s close enough to work with, and the subtle manipulations of that take simply enhance the work, without “faking” anything. You’re simply hearing the combined efforts of the performer(s) and the engineer(s) on a CD or, these days, a live performance.

I stand by my statement that singers and instrumentalists with non-even-tempered instruments naturally seek out a slightly different scale/tuning than the even-tempered one to which our ears have become accustomed. This can often lead to tonal clashes that are easily remedied by very slight pitch adjustments in post-processing of the work. It’s not my “arse talking” — it’s fact. A singer is often slightly flat on the third of many chords because that’s the natural tonal balance, where the even-tempered piano or guitar accompanying them is slightly sharp of the sweeter, instinctive tuning of the singer.

I admit that, you’re right, rap artists are almost certainly not pitch-corrected. Neither are recordings where everything is choral, since there are no known pitch-correction algorithms that can handle multiple-voice correction other than in the roughest manner like changing the key of a piece one semitone. Orchestral pieces, likewise. But mainstream pop, rock, and heavy metal or alternative with strong melodic lines? If there’s been a mainstream million plus-selling melodic release without pitch correction of any sort on any instruments or vocalists on the album, I’ll be very surprised.

I somehow can’t shake the feeling, though, that I’ve just responded to a very subtle troll, due to the apparent familiarity of topic, combined with numerous factual innacuracies of Anonymous’s post…

Best rant ever!

Eric S. Raymond (ESR) just posted the best rant I’ve ever read.

Eric is one of the luminaries of the Open-Source/Free Software community. His seminal writings, such as “The Cathedral And The Bazaar”, “Homesteading the Noosphere“, and “The Magic Cauldron” have shaped the values of a generation of software programmers steeped in the open development tradition. I’ve never seen anything make him angry before in his writing. This is an amazing first.

If you’re interested in more background on this case, I’d recommend Googling for details on “SCO versus IBM“, “Linux versus SCO“, and hitting the archives on Slashdot.org searching for “SCO”. There’s a lot of history for one short year.

Eric S. Raymond (ESR) just posted the best rant I’ve ever read.

Eric is one of the luminaries of the Open-Source/Free Software community. His seminal writings, such as “The Cathedral And The Bazaar”, “Homesteading the Noosphere“, and “The Magic Cauldron” have shaped the values of a generation of software programmers steeped in the open development tradition. I’ve never seen anything make him angry before in his writing. This is an amazing first.

If you’re interested in more background on this case, I’d recommend Googling for details on “SCO versus IBM“, “Linux versus SCO“, and hitting the archives on Slashdot.org searching for “SCO”. There’s a lot of history for one short year.

WE ARE MOVING

I am spending today moving servers to a newer, safer location. I’m tired of the power outages in this colo; they are very frustrating and frequent. I’m locking both the Outlanders Outfit and barnson.org databases here once I get the move underway, so if you are unable to post comments you’ll know why. I should be done by the end of today, Aug 25 2003.

It was a close thing that my server wasn’t hosed forever. Guess I need to start doing nightly mysqldump’s and exports from Cyrus mail.

For your daily dose of freakishness, try this link. Yes, it’s work-safe. Just pictures from the 1991 Anything Goes production at Quince Orchard High School. I was “Sir Evelyn Oakleigh”, first picture on the upper left. Man, I look different.

For the impatient, here’s the image without clicking (boy, the things I do for you!). The strangest thing is, Mary’s kept these pictures around for 12 years, and I think they’ve been up on the web nearly that long. Dig the rouge!

Matt Barnson and Jen Wolfe as Evelyn Oakleigh and Reno Sweeney in Anything Goes

I am spending today moving servers to a newer, safer location. I’m tired of the power outages in this colo; they are very frustrating and frequent. I’m locking both the Outlanders Outfit and barnson.org databases here once I get the move underway, so if you are unable to post comments you’ll know why. I should be done by the end of today, Aug 25 2003.

It was a close thing that my server wasn’t hosed forever. Guess I need to start doing nightly mysqldump’s and exports from Cyrus mail.

For your daily dose of freakishness, try this link. Yes, it’s work-safe. Just pictures from the 1991 Anything Goes production at Quince Orchard High School. I was “Sir Evelyn Oakleigh”, first picture on the upper left. Man, I look different.

For the impatient, here’s the image without clicking (boy, the things I do for you!). The strangest thing is, Mary’s kept these pictures around for 12 years, and I think they’ve been up on the web nearly that long. Dig the rouge!

Matt Barnson and Jen Wolfe as Evelyn Oakleigh and Reno Sweeney in Anything Goes

One Man: Prerelease!

OK, folks, early reviews are in, and although the song is not finished, they are highly favorable. So I figure I’ll throw this over the wall. Click “Read More” to get the full blog and the download link for this tune. Best work I’ve ever done. But still not Garageband-ready.

“One Man” is a song with many lives. At the age of 17, I was conflicted, coming out of the other side of my mom’s divorce, and had just spent a week

OK, folks, early reviews are in, and although the song is not finished, they are highly favorable. So I figure I’ll throw this over the wall. Click “Read More” to get the full blog and the download link for this tune. Best work I’ve ever done. But still not Garageband-ready.

“One Man” is a song with many lives. At the age of 17, I was conflicted, coming out of the other side of my mom’s divorce, and had just spent a week having my head filled with Latter-Day Saint philosophy at a summer youth camp called EFY, or “Especially For Youth”. I knew that I needed to choose something; rootless existence wasn’t for me. I figured the whole Jesus thing was just a man who’s reputation had grown with time, and pictured myself in his role, wondering what he’d think about about all the hullabaloo raised in his name.

Around this time, I received an out-of-the-blue phone call from a girl named Jenny. In 1973 when we’d both been born, Jennifer was an incredibly popular name. Anyway, I’d known her all through my elementary years, up through the middle of sixth grade. Some of our neighbors were murdered (not close, about half a mile away), and that was the final straw; my parents decided to move to a safer neighborhood. Jenny said she’d always wondered where I’d gone to, and then one day looked up my last name. We were the only “Barnson” in the phone book for the D.C. area, so she took a chance and called.

Jenny was sincerely pacifistic and outspoken; this was a perspective with which I was unfamiliar and intrigued. The desire to write One Man came out of my newfound hope that there was some reality to this whole religion thing, and to impress this girl with how sensitive I was, writing about a “man of emotion” and that violence need no longer be a fundamental underpinning of man’s existence. So, as in most things, the good story always starts with the girl.

This is the third recording of One Man ever released. The first one was on the Wayward Sun tape, “The Right of Way”, released ca. 1990. The second edition, recorded around 1993, was done while I was serving a mission for the LDS church, on the album “No Further ?” (No Further Questions). I was, and am, dissatisfied with both of those recordings, mostly for technical reasons.

This new version is *almost* where I want it to be, and barring several mistakes and orchestration that’s not entirely there yet, I am satisfied with how it’s come out. It represents several hundred hours of effort as I re-acquainted myself with recording techniques, Cakewalk Sonar, and made two false starts which consumed their own massive share of time. (hint: it’s a really good idea to back up the music folders on your hard drive.)

Acknowledgements: Justin Timpane sings much of the lead vocal on this tune.
Special Thanks: Sam Graber, Ben Schuman, Kevin Graham, and Ed Copeland, who made some modifications to the tune that I’ve incorporated into my vision.

Without further ado:

download and play One Man.

If you have troubles with the above link, right-click it and “save target as”, then play it from your hard drive. Some ancient sound cards and software have trouble playing back 48KHz (DAT standard) files, rather than 44.1KHz (CD standard). I’ll try to get a 44.1 version up here as soon as I’ve figured out how to do it in my new MP3 encoder 🙂

As always, this tune and all other material on this website is Copyright © 2003 Matthew P. Barnson. All Rights Reserved. You are free to copy anything you wish from this site, as long as you provide provide a prominent link, endnote, or footnote back to this web site.

My Sunday Sermon

As is often the case on Sunday mornings, our toddler and my wife were able to get some sleep around the same time Sunday morning, so I caught up on the conversations in some mailing lists. A conversation is raging through one list regarding a judge who secretly installed a two-ton stone Ten Commandments monument in the Alabama State Supreme Court building. Some list members suggested that if the Ten Commandments were an appropriate monument on the property of a government-owned building, perhaps the Wiccan “do what ye will, but harm no one” belief, or the Mormon “Articles of Faith” should also be given government ground. Good old ELC, the raving catholic, chose to speak up:

As is often the case on Sunday mornings, our toddler and my wife were able to get some sleep around the same time Sunday morning, so I caught up on the conversations in some mailing lists. A conversation is raging through one list regarding a judge who secretly installed a two-ton stone Ten Commandments monument in the Alabama State Supreme Court building. Some list members suggested that if the Ten Commandments were an appropriate monument on the property of a government-owned building, perhaps the Wiccan “do what ye will, but harm no one” belief, or the Mormon “Articles of Faith” should also be given government ground. Good old ELC, the raving catholic, chose to speak up:

What you are advocating is separation of RELIGION and state. That is NOT what the First Amendment speaks to. Since neither the Articles of Faith, nor the Koran, nor the sayings of Confucius, etc., could remotely be considered guiding principles and values of our founding fathers, your hypothetical is not at all analogous nor instructive to this present situation. Common sense draws the line. Judeo-Christian values are a bedrock of our Western civilization. Wiccan “values” and Mormon “values” are not … most Americans want their government to acknowledge Judeo-Christian values which give us our identity, direction, and grounding. If the secular lobby triumphs it will be a lot darker than it was when the lights were turned out for a day or so. Now, that is a scary thought.

I see no difference, other than semantic, between “religion”, “church”, and “faith” (as in one’s faith, not the act of having faith). List reader Llona had this to say:

Judeo-Christian values were perfectly comfortable with slavery, public hangings, workhouses for the poor, beating, threatening, and killing those who tried to organize unions, withholding the vote first from non-property holders, then from women and blacks, wiping out millions of Indians, etc.

Do you think any Iraqi constitution should favor Islam and post portions of the Koran around in public places because commmon sense says that is the bedrock of their civilization (which is far, far older than ours)?

Then Eric, another alert list reader, chimed in with some excellent quotes:

I believe that many of the founding fathers were also Mason and anti-religious. Should we not let anyone into government buildings unless they know the secret handshakes?

The religious preferences of the founding fathers have no legal bearing on our societal institutions. If they had wanted to include the ten commandments in government buildings they could have, and would have damn well written it into the constitution. Instead their references to religion are vague: In God we Trust (not Jesus).

The founding fathers were fans of Christianisty? Food for thought:

“Experience witnesseth that ecclesiastical establishments, instead of maintaining the purity and efficacy of religion, have had a contrary operation. During almost fifteen centuries has the legal establishment of Christianity been on trial. What has been its fruits? More or less, in all places, pride and indolence in the clergy; ignorance and servility in the laity; in both, superstition, bigotry and persecution.”

– James Madison, “A Memorial and Remonstrance”, 1785

“It is not to be understood that I am with him (Jesus Christ) in all his doctrines. I am a Materialist; he takes the side of Spiritualism; he preaches the efficacy of repentence toward forgiveness of sin; I require a counterpoise of good works to redeem it. Among the sayings and discourses imputed to him by his biographers, I find many passages of fine imagination, correct morality, and of the most lovely benevolence; and others, again, of so much ignorance, so much absurdity, so much untruth, charlatanism and imposture, as to pronounce it impossible that such contradictions should have proceeded from the same being. I separate, therefore, the gold from the dross; restore him to the former, and leave the latter to the stupidity of some, the roguery of others of his disciples. Of this band of dupes and imposters, Paul was the great Coryphaeus, and the first corruptor of the doctrines of Jesus.”

– Thomas Jefferson to W. Short, 1820

“I think vital religion has always suffered when orthodoxy is more regarded than virtue. The scriptures assure me that at the last day we shall not be examined on what we thought but what we did.”

– Benjamin Franklin letter to his father, 1738

Then Tami chimed in with some helpful URIs:

For obvious and completely selfish reasons, I regard freedom of religion to include freedom from religion as well. The First Amendment is deceptively simple:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.

Now, we’re in a situation where the first and second phrases of the Bill of Rights seem to be in competition with one another. Does allowing a government official to put up a monument to a particular religion constitute making a law respecting an establishment of religion? And does forcing the official to remove the monument “prohibit the free exercise thereof”?

For my part, I think that the placement of a two-ton monument to any religion on government property amounts to an endorsement of that religion. Judeo-Christian values have nothing to do with this argument; our leaders are specifically instructed by the First Amendment that government must be a secular institution and not dally in religious affairs. If one person is allowed to “exercise” his religion by placing large stone monuments with religious creeds on government property, then others should be allowed to do likewise with no respect towards the religion involved. Such a situation, though, would be ridiculous — although it’s been done before, and if I understand correctly, Ogden Utah’s city hall property is now littered with the screeds of at least three religions.

The U.S. is republic, with some strange ways of electing certain leaders, rather than a true democracy for important reasons, including this one: we must prevent a tyranny of the majority. The electoral college, and “winner takes all” requirements for much of the electorate, force presidential candidates to cater to the needs of minorities and balance them with majority demands in order to win the race. A simple natinoal majority vote would allow the prospective president to simply identify the two or three most popular viewpoints and cater to them, ignoring minority voices. The requirement of a two-thirds Congressional majority for consitutional amendments, plus similar ratification by the states, sets up a situation which is favorable to minority voters having a voice against the majority which would otherwise rob them of their rights.

As a self-selected “minority” now, with a naturalistic rather than supernatural worldview, I find the promotion of any particular “religion” (including state-condoned strong atheism, or the dogmatic assertion of the nonexistence of god) an anathema. There are many other rational, centrist individuals, regardless of religious beliefs, who also agree that we must balance acknowledgement of religion with an even-handed, “blind” approach when dealing with anything but abuses of the law by those religions. By displaying a monument to Christianity’s Commandments in a government building, the government is announcing support for the religions that created them, and making law by tradition disrespecting those who do not share the same view.

Yank it. Hooray to the House of Representatives and Senate for refusing to donate federal funds to the removal of this eyesore. Make the judge or the state that allowed this to happen pay for their own self-righteous mistakes.