Posted to this article on Slashdot today, and I figured I’d like to archive it somewhere useful…

Tuning, pitch, and scale are closely correlated. The two most common “tunings” in the western world are even-tempered and Pythagorean. The most common “scales” are Ionian and Aeolian (major and minor), with Dorian and Phrygian sometimes chiming in on popular music, but rarely others. Other cultures offer non-pentatonic scales with sometimes only five notes. I’m not confusing pitch and scale. I’m explaining that often pitch correction is necessary, particularly in some unusual recording situations, due to the conflict between modern even-tempered 12-tone tuning of certain instruments and the natural instinct of a singer or inexact-pitch instrument (such most strings, which depend on finger position for pitch, and some woodwinds where one can slightly adjust pitch via jaw tension) to gravitate towards a sweeter, non-logarithmic tuning.

It appears you’ve never done harmonic analysis of choral music, or tried to match an accompaniment to an in-tune choral arrangement when said piece was first performed a cappella. Any competent digital piano will allow you to change tunings (note: NOT change pitch, A=440 all the way here) to match the harpsichord needs of pre-Baroque pieces or gain the sweet sound of a perfect Pythagorean chord.

If a piano is tuned to the Pythagorean scale in, say, the key of B flat, trying to play a piece in C major on the same piano without retuning will sound horrible. This is perfectly well-understood in the music community. If you wish to play an even-tempered instrument in multiple keys, you accept a slight dissonance across all ranges of the keyboard in exchange for the flexibility of playing in any key without unbearable dissonance. It is perfectly possible, and often done even today with harpsichords, to tune a keyboard instrument to a non-even-tempered scale in order to provide “perfect” consonance in playing pre-Baroque period pieces.

Now on to the rest of your nearly-coherent rant:

Good singers have perfect pitch

Baloney. You can be a good singer with good relative pitch. “Perfect Pitch”, as inexpertly named for this article, is a totally different thing from singing in tune, or having good relative pitch. Given that I mentioned “imperfect pitch”, above, I stand by what I said: all singers have imperfect pitch. They will not always nail the note perfectly, particularly at the end of an exhausting recording session. There will be times that pitch correction is welcomed as a practical measure in many vocalist’s lives. There are, of course, purists who will raise holy hell if someone were to pitch-correct them.

Since when does a key change sound awful?

If your instrument is even-tempered, key changes within a piece do not sound awful, although there is a slight dissonance to this tuning. If you are using a natural temperament or other alternative, sweeter tuning, it will sound awful in other keys, particularly if those keys don’t have a fundamental on the major fourth or fifth with few accidentals versus the primary scale. Since you are obviously a complete novice to the understanding of tuning systems, allow me to recommend checking out this brief talk on “Math and Music”. These days, we’ve taken the even-tempered scale a bit further by using logarithmic tuning devices rather than simply dividing octaves by 12, but even those tuning devices are not quite “perfect” when tuning a piano. You need to stretch the octaves on the upper regions of the piano in order to avoid perceived dissonance on the part of the listener, and that is a skill that takes a long time to master.

It is not and has never been called the Cher Effect. Its called over compression.

OK. I believe you. No, no, actually, I don’t. It’s very often referred to as “The Cher Effect” when you have fast response times on pitch correction (or vocoder) that force a slur into an unnatural abrupt pitch shift that sounds electronic. Welcome to reality, dude, it’s what an awful lot of people in the pro recording circles I frequent call it, and the moment anybody mentions “the Cher Effect”, nearly everyone knows exactly what they’re talking about.

And compression has absolutely zero to do with pitch adjustment. I think I can safely assume you’ve never been caught late at night in the hypnotic glare of the lights on your audio equipment, compressor readouts gently bouncing to the soft knee you set to manage volumes on the last step of your effects chain as you dump to your mastering deck. Compressors are wonderful, useful (and today, somewhat over-used) pieces of equipment — but they don’t effect pitch, just volume.

Nobody in their right mind really thinks bullet time happened (Matrix fans can flame me later), but correcting a lack of ability and passing it off as ‘quality’ is just plain dishonest.

Bullet time is an example of art in action through technical excellence. It’s over-used today, but nevertheless it takes skill, preparation, and knowledge to get it working right. It’s but one tool in the arsenal of the special effects master.

I was a music theory & composition major in college. I admit that I lean more toward the engineering & composing side of things, as my performance skills are merely above average. The job of the sound engineer is to make the piece sound perfect, listenable, and balanced, and pitch correction is just another tool in the vast array of options we have available to us. Pitch correction is neither dishonest nor is it correcting a “lack of ability”. It’s just part of making a song perfect.

Would you tell the director of a movie he can’t use blue screens because that’s “dishonest” and the performers should be able to do in real life what is portrayed on-screen? Of course not. I admit singing is a different field, but the principle applies. Performers are selling entertainment and illusion; if people are entertained, the artists have done their job well.

Their “natural” sound is talent and ability. Your sound may be fun or interesting, but the reason others thrive is because they don’t need use technology to sound good.

When I was referring to the natural sound, what I meant was the gritty sound, the not-quite-perfect, late-at-night-and-six-beers-down singing that we all get down to. There are people that really enjoy in-your-face music that has nothing extra on it. While it’s cool to listen to, and I enjoy a lot of that music (heck, I’m a Garageband member with many reviews under my belt, I love hearing raw talent), I also enjoy technical excellence and applaud the engineers who elevate the level of already outstanding performances to perfection. Some folks don’t appreciate that. And that simply boils down to taste. So yeah, some artists thrive in simple performance of their tunes, free of adornment. Others get their satisfaction from the engineering and compositional skill of the product. I’m in the latter camp, happily — and that’s something up to personal taste.

Real artists neither want nor need pitch corection.

More baloney. Composers are artists, and often want the performers of their works pitch-corrected. Vocalists sometimes find certain passages impossible to sing, and after fifteen takes are just sick of trying to get it perfect and welcome the pitch correction. Guitarists miss a bend halfway through a solo that they can’t easily repeat in the studio. Hired backup singers turn out to have missed a note, but are already out of the studio. A singer with no natural vibrato may want some added to a particular passage, or one with too much vibrato may want it reduced. The saxophone player you hired for your session may have been slightly sharp the whole time. Whatever the reason, there are plenty of uses for pitch correction, and it is used on many, many professional products these days.

The human brain may be good at sensing when something’s not right, but the competent engineer first gets a take that’s close enough to work with, and the subtle manipulations of that take simply enhance the work, without “faking” anything. You’re simply hearing the combined efforts of the performer(s) and the engineer(s) on a CD or, these days, a live performance.

I stand by my statement that singers and instrumentalists with non-even-tempered instruments naturally seek out a slightly different scale/tuning than the even-tempered one to which our ears have become accustomed. This can often lead to tonal clashes that are easily remedied by very slight pitch adjustments in post-processing of the work. It’s not my “arse talking” — it’s fact. A singer is often slightly flat on the third of many chords because that’s the natural tonal balance, where the even-tempered piano or guitar accompanying them is slightly sharp of the sweeter, instinctive tuning of the singer.

I admit that, you’re right, rap artists are almost certainly not pitch-corrected. Neither are recordings where everything is choral, since there are no known pitch-correction algorithms that can handle multiple-voice correction other than in the roughest manner like changing the key of a piece one semitone. Orchestral pieces, likewise. But mainstream pop, rock, and heavy metal or alternative with strong melodic lines? If there’s been a mainstream million plus-selling melodic release without pitch correction of any sort on any instruments or vocalists on the album, I’ll be very surprised.

I somehow can’t shake the feeling, though, that I’ve just responded to a very subtle troll, due to the apparent familiarity of topic, combined with numerous factual innacuracies of Anonymous’s post…

Eric S. Raymond (ESR) just posted the best rant I’ve ever read.

Eric is one of the luminaries of the Open-Source/Free Software community. His seminal writings, such as “The Cathedral And The Bazaar”, “Homesteading the Noosphere“, and “The Magic Cauldron” have shaped the values of a generation of software programmers steeped in the open development tradition. I’ve never seen anything make him angry before in his writing. This is an amazing first.

If you’re interested in more background on this case, I’d recommend Googling for details on “SCO versus IBM“, “Linux versus SCO“, and hitting the archives on Slashdot.org searching for “SCO”. There’s a lot of history for one short year.

I am spending today moving servers to a newer, safer location. I’m tired of the power outages in this colo; they are very frustrating and frequent. I’m locking both the Outlanders Outfit and barnson.org databases here once I get the move underway, so if you are unable to post comments you’ll know why. I should be done by the end of today, Aug 25 2003.

It was a close thing that my server wasn’t hosed forever. Guess I need to start doing nightly mysqldump’s and exports from Cyrus mail.

For your daily dose of freakishness, try this link. Yes, it’s work-safe. Just pictures from the 1991 Anything Goes production at Quince Orchard High School. I was “Sir Evelyn Oakleigh”, first picture on the upper left. Man, I look different.

For the impatient, here’s the image without clicking (boy, the things I do for you!). The strangest thing is, Mary’s kept these pictures around for 12 years, and I think they’ve been up on the web nearly that long. Dig the rouge!

OK, folks, early reviews are in, and although the song is not finished, they are highly favorable. So I figure I’ll throw this over the wall. Click “Read More” to get the full blog and the download link for this tune. Best work I’ve ever done. But still not Garageband-ready.

“One Man” is a song with many lives. At the age of 17, I was conflicted, coming out of the other side of my mom’s divorce, and had just spent a week having my head filled with Latter-Day Saint philosophy at a summer youth camp called EFY, or “Especially For Youth”. I knew that I needed to choose something; rootless existence wasn’t for me. I figured the whole Jesus thing was just a man who’s reputation had grown with time, and pictured myself in his role, wondering what he’d think about about all the hullabaloo raised in his name.

Around this time, I received an out-of-the-blue phone call from a girl named Jenny. In 1973 when we’d both been born, Jennifer was an incredibly popular name. Anyway, I’d known her all through my elementary years, up through the middle of sixth grade. Some of our neighbors were murdered (not close, about half a mile away), and that was the final straw; my parents decided to move to a safer neighborhood. Jenny said she’d always wondered where I’d gone to, and then one day looked up my last name. We were the only “Barnson” in the phone book for the D.C. area, so she took a chance and called.

Jenny was sincerely pacifistic and outspoken; this was a perspective with which I was unfamiliar and intrigued. The desire to write One Man came out of my newfound hope that there was some reality to this whole religion thing, and to impress this girl with how sensitive I was, writing about a “man of emotion” and that violence need no longer be a fundamental underpinning of man’s existence. So, as in most things, the good story always starts with the girl.

This is the third recording of One Man ever released. The first one was on the Wayward Sun tape, “The Right of Way”, released ca. 1990. The second edition, recorded around 1993, was done while I was serving a mission for the LDS church, on the album “No Further ?” (No Further Questions). I was, and am, dissatisfied with both of those recordings, mostly for technical reasons.

This new version is *almost* where I want it to be, and barring several mistakes and orchestration that’s not entirely there yet, I am satisfied with how it’s come out. It represents several hundred hours of effort as I re-acquainted myself with recording techniques, Cakewalk Sonar, and made two false starts which consumed their own massive share of time. (hint: it’s a really good idea to back up the music folders on your hard drive.)

Acknowledgements: Justin Timpane sings much of the lead vocal on this tune.
Special Thanks: Sam Graber, Ben Schuman, Kevin Graham, and Ed Copeland, who made some modifications to the tune that I’ve incorporated into my vision.

Without further ado:

download and play One Man.

If you have troubles with the above link, right-click it and “save target as”, then play it from your hard drive. Some ancient sound cards and software have trouble playing back 48KHz (DAT standard) files, rather than 44.1KHz (CD standard). I’ll try to get a 44.1 version up here as soon as I’ve figured out how to do it in my new MP3 encoder 🙂

As always, this tune and all other material on this website is Copyright © 2003 Matthew P. Barnson. All Rights Reserved. You are free to copy anything you wish from this site, as long as you provide provide a prominent link, endnote, or footnote back to this web site.

As is often the case on Sunday mornings, our toddler and my wife were able to get some sleep around the same time Sunday morning, so I caught up on the conversations in some mailing lists. A conversation is raging through one list regarding a judge who secretly installed a two-ton stone Ten Commandments monument in the Alabama State Supreme Court building. Some list members suggested that if the Ten Commandments were an appropriate monument on the property of a government-owned building, perhaps the Wiccan “do what ye will, but harm no one” belief, or the Mormon “Articles of Faith” should also be given government ground. Good old ELC, the raving catholic, chose to speak up:

What you are advocating is separation of RELIGION and state. That is NOT what the First Amendment speaks to. Since neither the Articles of Faith, nor the Koran, nor the sayings of Confucius, etc., could remotely be considered guiding principles and values of our founding fathers, your hypothetical is not at all analogous nor instructive to this present situation. Common sense draws the line. Judeo-Christian values are a bedrock of our Western civilization. Wiccan “values” and Mormon “values” are not … most Americans want their government to acknowledge Judeo-Christian values which give us our identity, direction, and grounding. If the secular lobby triumphs it will be a lot darker than it was when the lights were turned out for a day or so. Now, that is a scary thought.

I see no difference, other than semantic, between “religion”, “church”, and “faith” (as in one’s faith, not the act of having faith). List reader Llona had this to say:

Judeo-Christian values were perfectly comfortable with slavery, public hangings, workhouses for the poor, beating, threatening, and killing those who tried to organize unions, withholding the vote first from non-property holders, then from women and blacks, wiping out millions of Indians, etc.

Do you think any Iraqi constitution should favor Islam and post portions of the Koran around in public places because commmon sense says that is the bedrock of their civilization (which is far, far older than ours)?

Then Eric, another alert list reader, chimed in with some excellent quotes:

I believe that many of the founding fathers were also Mason and anti-religious. Should we not let anyone into government buildings unless they know the secret handshakes?

The religious preferences of the founding fathers have no legal bearing on our societal institutions. If they had wanted to include the ten commandments in government buildings they could have, and would have damn well written it into the constitution. Instead their references to religion are vague: In God we Trust (not Jesus).

The founding fathers were fans of Christianisty? Food for thought:

“Experience witnesseth that ecclesiastical establishments, instead of maintaining the purity and efficacy of religion, have had a contrary operation. During almost fifteen centuries has the legal establishment of Christianity been on trial. What has been its fruits? More or less, in all places, pride and indolence in the clergy; ignorance and servility in the laity; in both, superstition, bigotry and persecution.”

– James Madison, “A Memorial and Remonstrance”, 1785

“It is not to be understood that I am with him (Jesus Christ) in all his doctrines. I am a Materialist; he takes the side of Spiritualism; he preaches the efficacy of repentence toward forgiveness of sin; I require a counterpoise of good works to redeem it. Among the sayings and discourses imputed to him by his biographers, I find many passages of fine imagination, correct morality, and of the most lovely benevolence; and others, again, of so much ignorance, so much absurdity, so much untruth, charlatanism and imposture, as to pronounce it impossible that such contradictions should have proceeded from the same being. I separate, therefore, the gold from the dross; restore him to the former, and leave the latter to the stupidity of some, the roguery of others of his disciples. Of this band of dupes and imposters, Paul was the great Coryphaeus, and the first corruptor of the doctrines of Jesus.”

– Thomas Jefferson to W. Short, 1820

“I think vital religion has always suffered when orthodoxy is more regarded than virtue. The scriptures assure me that at the last day we shall not be examined on what we thought but what we did.”

– Benjamin Franklin letter to his father, 1738

Then Tami chimed in with some helpful URIs:

• http://www.ffrf.org/nontracts/xian.html
• http://www.infidels.org/library/modern/farrell_till/myth.html
• http://www.secularhumanism.org/library/shb/buckner_18_1.html
• http://www.dimensional.com/~randl/founders.htm
• http://ffrf.org/tm.php?tm=http://ffrf.org/issues/secular.html

For obvious and completely selfish reasons, I regard freedom of religion to include freedom from religion as well. The First Amendment is deceptively simple:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.

Now, we’re in a situation where the first and second phrases of the Bill of Rights seem to be in competition with one another. Does allowing a government official to put up a monument to a particular religion constitute making a law respecting an establishment of religion? And does forcing the official to remove the monument “prohibit the free exercise thereof”?

For my part, I think that the placement of a two-ton monument to any religion on government property amounts to an endorsement of that religion. Judeo-Christian values have nothing to do with this argument; our leaders are specifically instructed by the First Amendment that government must be a secular institution and not dally in religious affairs. If one person is allowed to “exercise” his religion by placing large stone monuments with religious creeds on government property, then others should be allowed to do likewise with no respect towards the religion involved. Such a situation, though, would be ridiculous — although it’s been done before, and if I understand correctly, Ogden Utah’s city hall property is now littered with the screeds of at least three religions.

The U.S. is republic, with some strange ways of electing certain leaders, rather than a true democracy for important reasons, including this one: we must prevent a tyranny of the majority. The electoral college, and “winner takes all” requirements for much of the electorate, force presidential candidates to cater to the needs of minorities and balance them with majority demands in order to win the race. A simple natinoal majority vote would allow the prospective president to simply identify the two or three most popular viewpoints and cater to them, ignoring minority voices. The requirement of a two-thirds Congressional majority for consitutional amendments, plus similar ratification by the states, sets up a situation which is favorable to minority voters having a voice against the majority which would otherwise rob them of their rights.

As a self-selected “minority” now, with a naturalistic rather than supernatural worldview, I find the promotion of any particular “religion” (including state-condoned strong atheism, or the dogmatic assertion of the nonexistence of god) an anathema. There are many other rational, centrist individuals, regardless of religious beliefs, who also agree that we must balance acknowledgement of religion with an even-handed, “blind” approach when dealing with anything but abuses of the law by those religions. By displaying a monument to Christianity’s Commandments in a government building, the government is announcing support for the religions that created them, and making law by tradition disrespecting those who do not share the same view.

Yank it. Hooray to the House of Representatives and Senate for refusing to donate federal funds to the removal of this eyesore. Make the judge or the state that allowed this to happen pay for their own self-righteous mistakes.

I was reviewing some of my old Slashdot posts, and I came across some interesting arguments from June of this year that, it’s interesting to note, my own opinions sway slightly from.

Part of this is my being honest: I got a C&D (Cease And Desist) from Universal a month or so after I wrote these comments. I’d tried out a program called “eMule” (apparently a clone of another program called “eDonkey”), got myself a username, and went out hunting for what sort of copyright-infringing stuff I could download. Along with the usual assortment of cracked software, mp3’s, and pornographic crap that’s floating around that network, there is also a large assortment of movies.

Including one called “The Hulk”.

I’ve seen it in the theater, and I’ve seen the version that was trading on eMule. They BOTH stink. I’m not joking, it’s just a really bad movie. I loved the TV show, and I had a few of the comics as a teenager. It just didn’t work for me.

But anyway, I decided to download it. I left up eMule overnight, came by the next morning to see that it was all transferred, turned off eMule, burned the two video CD’s on my CD burner, and watched the first CD. Other than the fact I was watching a really crappy camcorder version of the movie, it looked pretty much the same as “The Hulk” is seen in theaters. There were some editing and post-production things that changed, but by and large it was the same movie. I thought nothing of it, really, blew away the files from my hard drive, stuck the two copies of The Hulk on my CDR spool, and went about my business. eMule sat, forlorn and forgotten, on my hard drive.

About three weeks later, I received a notice from my DSL provider that Universal Studios had sent them a cease & desist order, and that my identity could be requested and would have to be provided, without even a subpoena from a court, due to the Digital Milennium Copyright Act.

BUSTED.

I felt sick. Literally. I quickly responded to the message with a cryptic “The offending material has been long since removed” (and that was backed up by Universal, they had only found it on the network one night) “and the offender has been lectured”.

Never mind that it was me lecturing myself on getting caught!

Anyway, my opinion on copyright has been swayed a bit since I wrote the original article, mostly due to that C&D experience. I still value copyright, and favor its limitation, but also seriously think the system needs massive evaluation that incorporates an understanding of the existence of easy duplication of any copyrighted work.

At the time the Constitution was written, duplicating a copyrighted work was an enormous investment of time and money. Today, to do the same can often be as simple as a cut & paste. The Digital Millenium Copyright Act was an attempt to address these concerns, but it was done so with little concern for the legitimate file-trading of individuals, and no foresight as to enormous peer-to-peer networks and the role of fair use.

I don’t know that I’m qualified to answer the questions, either. But I feel an allegory coming on:

When I was a kid, I remember sitting next to my radio and listening to songs for hours on end. Sometimes I would read a book, but quite often I could just be found chilling out to the tunes. From time to time, I’d want to make a “collection” for a friend, so I’d put a trusty cassette into the deck, and patiently wait for the announcer to tell me that certain songs were about to be played. The advent of dual-cassette tape decks made this even better and easier. My friends and I used to trade these, often interspersed with our own comments and banter, as a show of affection or joking around.

This is perfectly normal, natural behavior. If we didn’t have recordings, we’d probably have just sung songs that we heard to one another.

How do you handle this same altruistic behavior, the desire to share, in a situation where millions of people are doing the sharing, and you have no personal relationship with the one doing the sharing?

I’m not sure.

But I am sure of one thing:

Telling them they are criminals for sharing, and sending threatening letters to inspire fear in hopes they’ll do what the copyright holder wants, is *not* the solution. It’s a stopgap, inhumane method of criminalizing social human behavior. To outlaw the sharing of ideas and art without a cover charge, when the successful history of our race built on intuitive, imitative behavior, is to outlaw much of what makes us an human.

What a shame.

Afterword: I think the ultimate solution is probably going to be a ‘cover charge’ of some sort for participation in activities which lead to significant copyright infringement. Much as buying blank music CD’s carries a tariff paid directly to the Recording Industry Association Of America, if you participate in a legal peer-to-peer network you’re going to be required to pay a tariff so that those people who’s works are infringed may be compensated. How do you enforce that on a global scale, though? That’ll probably take someone smarter than me to figure out.

I just read an excellent piece by Bruce Perens (a personal hero of mine that I’ve heard speak on several occasions) which was based on SCO’s recent slide show on “offending” code in the Linux kernel which they claim is their stolen IP.

Rather than link to the story, I’ll add my own mirror and an “Amen, brother”. I would be terrifically offended if some company claimed ownership on code that I had written, and the Linux kernel development community is right in calling for SCO to show more of the source they claim is infringing.

To continue this dog and pony show of not allowing anybody to see the source that is in dispute (or even to name the line numbers of the disputed code in the publicly-available Linux kernel) is just dishonest. They are intentionally playing a game of fear, uncertainty, and doubt with the future of Linux in order to support their greatest financier, Microsoft, and try to wring sales out of an unwilling public through intimidation.

Yes, I realize the Microsoft part is simple unsubstantiated allegation (although the multi-million dollar contract for the “source” really seems like an MS payment for services to be rendered, to me), and the intimidation claim is speculation (based on observation and correlation from the desparate actions of the failing company, however). I stand by it: they are desparate for cash, paid a handsome sum by the richest corporation on the planet to discredit GNU/Linux, and using that blood money are attempting to blackmail Linux kernel users everywhere into paying them exhorbitant sums of money.

Bruce’s document follows, my rant is over:

---

Analysis of SCO’s Las Vegas Slide Show

Bruce Perens, Perens LLC <bruce@perens.com>
With help from Linus Torvalds and the Open Source community.

You may re-publish this material. You may excerpt it, reformat it and translate it as necessary for your presentation. You may not edit it to deliberately misrepresent my opinion.

An SCO presentation shown in Las Vegas on August 18th alleged infringement by the Linux developers. The presentation, in Microsoft PowerPoint format is here, and an conversion of the presentation that can be viewed using a web browser is here .

SCO released the presentation to Bob McMillan, a reporter for IDG News Service, without any non-disclosure terms. Bob asked me to comment upon it. here’s his story.

I will start with SCO’s demonstrations regarding "copied" software. It is likely that SCO would present the very best examples that they have of "copied" code in their slide show. But I was easily able to determine that of the two examples, one isn’t SCO’s property at all, and the other is used in Linux under a valid license. If this is the best SCO has to offer, they will lose.

Slide 15 shows purports to show “Obfuscated Copying” from Unix System V into Linux. SCO further obfuscated the code on this slide by switching it to a Greek font, but that was easily undone. It’s entertaining that the SCO folks had no clue that the font-change could be so easily reversed. I’m glad they don’t work on my computer security 🙂

The code shown in this slide implements the Berkeley Packet Filter, internet firewall software often abbreviated as “BPF”. SCO doesn’t own BPF. It was created at the Lawrence Berkeley Laboratory with funding from the U.S. Government, and is itself derived from an older version called “enet”, developed by Stanford and Carnegie-Mellon Universities. BPF was first deployed on the 4.3 BSD system produced by the University of California at Berkeley. SCO later copied the software into Unix System V.

The BPF source code is here on the Lab’s web site. A paper on its design, published in 1993, is here

BPF is under the BSD license. That license allowed SCO to legally copy the code into Unix System V in 1996, but since SCO doesn’t own the code, they have no right to prevent others from using it.

So, in this case the SCO “pattern-recognition” team correctly deduced that the Linux and SCO implementations of BPF were similar. But I was able to determine the origin of BPF after a few minutes of web searches on google.com . Why couldn’t a “pattern-recognition team” do the same? It’s difficult to believe they simply didn’t bother to check. It’s also likely that SCO dropped attribution of the Lab’s copyright from the System V copy of the BPF source code, or the team would have known.

The Linux version of BPF is not an obfuscation of the BPF code. It is a clean-room re-implementation of BPF by Jay Schulist of the Linux developers, sharing none of the original source code, but carefully following the documentation of the Lab’s product. The System V and Linux BPF versions shown in slide 15 implement the same virtual machine instruction set, which is used to filter (allow, reject, change, or reroute) internet packets. And the documentation for that VM even specifies field names. Thus Schulist’s and the Lab’s implementations appear similar. Had Schulist chosen to directly use the Lab’s code, it still would have been legal. But the version in Linux is entirely original to the Linux developers. There is no legal theory that would give SCO any claim upon it.

Slides 10 through 14 show memory allocation functions from Unix System V, and their correspondence to very similar material in Linux. Some of this material was deliberately obfuscated by SCO, by the use of a Greek font. I’ve switched that text back to a normal font.

These slides have several C syntax errors and would never compile. So, they don’t quite represent any source code in Linux. But we’ve found the code they refer to. It is included in code copyrighed by AT&T and twice released as Open Source under the BSD license: once by Unix Systems Labs (a division of AT&T), and again by Caldera, the company that now calls itself SCO. The Linux developers have a legal right to make use of the code under that license. No violation of SCO’s copyright or trade secrets is taking place.

The oldest version of this code we’ve found so far is in Donald Knuth’s The Art of Computer Programming, published in 1968. Knuth was probably working from earlier research papers. He didn’t write in C, so details differ but the algorithm is the same. The implementation shown in the slides was written by Dennis M. Ritchie or Ken Thompson at AT&T, in 1973. You can see the 1973 version of the function in this file, originally called dmr/malloc.c. The code is from Unix version 3, the oldest known version of Unix that still exists in machine-readable form. The complete source for that system can be found here on the net. In 2002, Caldera released this code as Open Source, under this license. Caldera is, of course, the company that now calls itself SCO. The license very clearly permits the Linux developers to use the code in question. Historical information on why Caldera released the Unix source code to the public is here, and contains some information relevant to the SCO court cases.

Another version of the code is copyrighted by the University of California as part of the BSD Unix system that they produced for the U.S. Army and released as Open Source. That code is also under the BSD license, and appears here in this file released in 1984. It’s interesting to consider how this code came to belong to the University.

In the early 1990s, AT&T’s Unix Systems Labs (USL) sued BSDI, a company vending the BSD system, and the University of California, over this and other code in the BSD system. The claims that SCO is making are very similar to the AT&T claims. AT&T lost. It was found that AT&T had copied heavily from the university without attribution, and thus AT&T settled the case. In the settlement, the University agreed to add an AT&T copyright notice to some files and to continue to distribute the entire system under the BSD license. AT&T agreed to pay the University’s court costs. Some details of the lawsuit are here.

AT&T was actually found to have lost its copyright to the code in question during the lawsuit, because the code wasn’t published. This would not be the case today, as there have been changes in copyright law. But the judge’s decision back then was:

Consequently, I find that Plaintiff has failed to demonstrate a likelihood that it can successfully defend its copyright in [Unix version] 32V. Plaintiff’s claims of copyright violations are not a basis for injunctive relief.

The result is that between the judge’s finding and 1996, when there were additional changes to the Bern copyright convention that would have made the AT&T code copyrightable, the code was essentially in the public domain. Code derived from Unix before and during that time would be legal.

The AT&T code that was subject of this lawsuit survives into SCO’s current system. SCO’s “pattern analysis team” found this code and correctly concluded that it was similar to code in Linux. But they didn’t take the additional step of checking whether or not the code had been released for others to copy legally.

The code in question has already been removed from the most recent development versions of the Linux kernel, for technical reasons. It duplicated a function provided elsewhere, and thus never should have been included. The code was intended for one SGI system that was never sold, and another that is extremely rare, and was not used in the mainstream Linux kernel.

In slide 20, SCO alleges that it owns essentially all of the code in Linux that has been touched at all by IBM, SGI, and other Unix licensees. These contributions constitute over 1.1 Million lines of code, 1549 files, totalling 2/3 of the new code developed between the releases of Linux 2.2 and 2.4. But how could SCO possibly own all of this code that is copyrighted by other companies and individuals? SCO’s legal theory, explained in slide 6, is that the AT&T Unix license compelled all of these companies to assign to AT&T, and later SCO, all derived works that they created incorporating the Unix source code. Here is the key clause on slide 6:

Such right to use includes the right to modify such SOFTWARE PRODUCT and to prepare derivative works based on such SOFTWARE PRODUCT, provided the resulting materials are treated hereunder as part of the original SOFTWARE PRODUCT.

Under SCO’s theory, if any code created by a Unix licensee ever touches Unix, SCO owns that code from then on, and can deny its creator the right to make use of it for any other purpose.

SCO’s legal theory fails, because they ignore the fact that if a work doesn’t contain some portion of SCO’s copyrighted code, it is not a derived work. This is especially glaring on slide 20, in which SCO claims ownership of JFS, IBM’s Journaling File System. The version of JFS used in Linux was originally developed for the OS/2 operating system, and was later ported to both Unix System V and Linux. SCO’s claims fail in a similar manner for the other products they mention: RCU or Read Copy Update, software that keeps processors in a multi-processor system from interfering with each other, was developed by Sequent, a company later purchased by IBM. Sequent developed RCU under Dynix, a Unix-derived operating system. They later removed RCU from Dynix – separating it from any code owned by SCO – and added it to Linux. Similarly, SGI’s XFS, the eXtent FileSystem, was separated from IRIX, a Unix-derived operating system, and ported to Linux.

SCO’s contention is that copyrighted software can never be separated, that any code created by a Unix licensee that ever touches SCO Unix or is even loosely based on Unix is entirely SCO’s from that moment on, and can never be used for another purpose by its creator without authorization from SCO. SCO’s contention goes against any reasonable understanding of the boundaries of intellectual property. It’s unlikely that it would survive a court room.

SCO’s responses to this document are We own Unix and would know what it looks like, and It’s his word against ours. I’m not, however, asking you to rely on my word. I’ve presented you with links to the evidence, most of which is available at web sites not under my control. Please examine it and make your own conclusion.

Bruce Perens

Links

• Older version of this article, from when I only had two slides.
• Bruce Perens’ web site.

So I’ve had the lovely task of dealing with the recent sobig.f outbreak on the Internet where I work. The same dunderheads that let themselves get infected by the last big virus failed to run Windows Update so that they could be prevented from getting this one.

It just goes to show that border security, basically, isn’t. People set up ways of blocking the bad stuff from getting to them, but they don’t bother to fix the underlying reason the “bad stuff” can cause problems in the first place. The moment anything makes it through the border, it can cause all the havoc it wants to. People aren’t taking responsibility for keeping their nodes secure on this big, wide Internet world, and the lack of their adequate policing is causing problems for the rest of us.

I ran into a related experience with my daughter this morning. I’m working from home today, since I have been keeping an eye on the pain to our mail server from virus transmissions. Anyway, she was getting some laundry out of the washer and the washer lid fell on her head. The obvious, rational conclusion is that she bumped the washer, causing the lid to dislodge from its open position and land on her head. I responded to the screaming wail of pain, brought over an ice pack, and asked for the explanation of what happened.

Now, we could propose alternative explanations, I suppose. But Sara insisted, “I didn’t bump the washer. It wasn’t my fault. The washer lid hit me in the head, and I don’t know how it happened!” I carefully explained cause and effect to her, that our goal is not to place blame but to figure out what happened, and how it happened, so that, in this case, we can prevent it from happening in the future.

She sullenly accepted my explanation and stalked off back to the washroom to move her laundry, this time without the accompanying crashing noises and loud crying.

But it made me think of the whole virus, and the tendency we human beings have to avoid responsibility for bad things. In one way, the recent outbreak of the RPC worms attacking Windows workstations was a good thing: the annoyance factor of rebooting your machine every 3 minutes or less forced people to update their PCs and take responsibility for helping police the Internet. I know it will be short-lived, but it’s progress. I’m just glad it wasn’t a very destructive worm, or recovery would have been far more painful than it was.

So, anyway, sobig.f is floating around today and I’m updating postfix rules. Here’s the meat of it.

Add these two entries to /etc/postfix/main.cf (or, if you’re using FreeBSD, /usr/local/etc/):

 body_checks = regexp:/etc/postfix/body_checks header_checks = pcre:/etc/postfix/header_checks.pcre 

Then you need to create the files “/etc/postfix/body_checks” and “/etc/postfix/header_checks.pcre”. I distinguish “.pcre” files this way, because that stands for “Perl Compatible Regular Expressions”, which are slightly different than normal “regex” regular expressions. If you don’t have PCRE support compiled into Postfix, the header_checks.pcre file won’t help you at all, and will actually cause Postfix to not start up at all, or in some cases just spit an error message out to your syslog.

Anyway, this is body_checks:

 # sobig rejection # The following statement should all be on one line, # with a space before "reject" # It's two lines due to formatting constraints. /^TVqQAAMAAAAEAAAA\/\/8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA$/ REJECT keep your viruses

# Klez rejection # The following statement should all be on one line, # with a space before "reject" # It's two lines due to formatting constraints. /^<iframe src=3Dcid:\S+ height=3D0 width=3D0>/ REJECT No IFRAMEs please /^<FONT>/ REJECT No viruses wanted here

##############################

This is header_checks.pcre (this can be multi-line as formatted):

 /^\s*Content-(Disposition|Type).*name\s*=\s*"?(.*\.( ade|adp|bas|bat|chm|cmd|com|cpl|crt|dll|exe|hlp|hta| inf|ins|isp|js|jse|lnk|mdb|mde|mdt|mdw|msc|msi|msp|mst|nws| ops|pcd|pif|prf|reg|scf|scr\??|sct|shb|shs|shm|swf| vb[esx]?|vxd|wsc|wsf|wsh))(\?=)?"?\s*(;|$)/x REJECT Attachment name "$2" may not end with ".$3" 

#################################

So, the nice thing here, is that these two rules will check for any attachments of known nasty types, and simply refuse to allow them to be delivered, period. The unfortunate weakness here is that if someone uses base64 encoding on the mail, we aren’t really checking it. You really need a virus-scanning package on the back-end for that. On the plus side, few virusses use base64 — it’s mostly reserved for spammers and people who aren’t using a native English mailers.

Hope that helps! It took a little bit of digging on mailing lists to get this far, but so far it seems to be helping us a lot. On the back-end, I have a script that just uses iptables (on Linux) to stop abusive mailers. Once I’ve sanitized out the stuff that definitely won’t apply to someone else’s environment, I’ll post that here, too.

P.S. Yeah, I know “view as PDF” is broken for this doc right now. I’m not certain how to properly handle <pre> tags when formatting for PDF, so you’ll have to be content with what’s here for now 🙂 Besides, I expect you would rather just cut & paste what’s above, rather than print it out and type it, wouldn’t you? Thought you would.

The headline:

U.N. HQ in Baghdad Car Bombed – FLASH: Large car bomb attack at U.N.’s Baghdad HQ. [Winds Of Change]

What, they don’t have JERSEY BARRIERS in Baghdad? I mean, the White House wasn’t even a war zone 15 years ago when I remember them wheeling in the big concrete barricades to stop any potential vehicular bomb from getting anywhere near the building. I note with some twisted humor that the Salt Lake City Wallace F. Bennett Federal Building seems engineered to prevent drive-ups, with large decorative, but obviously stout and concrete, pylons. I was just thinking the other day, “Wow, someone driving a car bomb would have a heck of a time getting near that building. Maybe a motorcycle bomb or something could make it, but how much damage could something that small do?”

Important buildings should have a perimeter of Jersey Barriers preventing casual drivers-by from unloading a car bomb into them. Even if they’re temporary. I wonder which U.N. genius dropped the ball deciding not to Jersey-Barrier this puppy?

I’m in the midst of a discussion with a close friend, via email, of some of the fundamental questions regarding the Bible, Christian thought, and the evolution of religion. In my searches, I came across this simply amazing, honest article, written by Richard Packham and entitled “How I Became An Atheist”.

And, of course, the title itself will put off some of my readers. Try another of Packham’s essays, The Man Who Bought A House, to really understand where he’s coming from. If you find yourself strongly disagreeing, perhaps you, too, have bought the house? His web site is an excellent collection of essays he’s written and links he’s collected over a lifetime of skepticism.

I kept saying to myself, “Oh, man, this is me. Did this guy read my mind?” Only obvious dissimilarities (like the fact he’s at least 35 years older than me, and graduated from college with a law degree) kept me grounded in the reality that this wasn’t my history I was reading. I could see myself writing an essay similar to this.

As a matter of fact, it was research into how I’d write a similar essay that led me to his site. Now I’m not so sure I want to write one of my own, since I’ve found one that so closely mirrors my own perceptions. Time will tell.