Reliability, Availability, & Fault-Tolerance

I wrote up a blurb on Slashdot today about my perspective on creating highly reliable, available, or fault-tolerant systems, and how you really need to choose which of the three you are going for in designing your compute environment.

I’ve also adjusted my opinion since I posted this. There are a couple more factors, which include initial expense and maintenance cost, that need to be factored in. They are normally the domain of the bean counter, but it’s important that the admin/systems architect be aware of what tradeoffs he is willing to make in order to bring those costs down, and where the sacrifice of reliability, availability, or fault-tolerance needs to be made. And he also needs to appraise the bean counter of the importance of those factors that are being lost, as well.

I wrote up a blurb on Slashdot today about my perspective on creating highly reliable, available, or fault-tolerant systems, and how you really need to choose which of the three you are going for in designing your compute environment.

I’ve also adjusted my opinion since I posted this. There are a couple more factors, which include initial expense and maintenance cost, that need to be factored in. They are normally the domain of the bean counter, but it’s important that the admin/systems architect be aware of what tradeoffs he is willing to make in order to bring those costs down, and where the sacrifice of reliability, availability, or fault-tolerance needs to be made. And he also needs to appraise the bean counter of the importance of those factors that are being lost, as well.

Proper machine administration is a balance of: * reliability * availability * fault-tolerance

Reliability generally refers to the components of a piece of hardware not breaking very often. In our Compaq DL360 and DL380 rackmount units, reliability is a critical factor in their power supplies. The power supplies are simply crap. The fans die, the supplies blow up. It’s terrible.

Availability, on the other hand, even if you have components or a system that is not highly reliable, it can still be highly available. For instance, 1U chassis systems with dual power supplies, although the power supply reliability tends to be low, nevertheless tend to be available despite the low reliability.

Fault-Tolerance is how well the system handles unusual circumstances. A system may be highly available and reliable, but if it does not handle system faults well, you may have a problem. Fault-Tolerance really refers to how well the system handles unusual conditions.

Now, I realize the distinctions seem rather vague. That is intentionally so! But separating the question into three parts helps grant admins a better look at what the weak points are of a system.

I generally prefer 1U or 2U units to have a single, reliable power supply, rather than dual power supplies with lower reliability. Because of the higher reliability of their PS, they tend to also be available more. However, you have very poor fault tolerance in the power area with just a single power supply, so you’ll normally need redundant systems in order to have a fault-tolerant environment. If that is the case, availability may suffer, for when the first machine goes down in that rare situation that the highly reliable power supply dies, you have some cutover time to the secondary system.

Really, you have to evaluate what’s most important for you. In the bank where I work, reliability is critical; fault-tolerance, somewhat less so. Availability is not so much of an issue. We close at 5:30, and really don’t do any business after that, so from 5:30 PM through 5:30 AM, our availability can be nonexistent for certain types of maintenance, and we’re just fine.

Like I said, the designations there are pretty arbitrary. But if you can come to an approach covering at least three angles on your machines and evaluate by those criteria, your overall uptime goals can be met in a way that suits your organization.

Me, I’m just sick of cheap hardware. I enjoy 1U rackmount systems, but I’d much rather have two higly reliable 2U boxes than four inherently less-reliable 1U units. Of course, vendors come into play there… Sun makes some killer 1U’s that never seem to die, while Compaq and Dell’s 1U (in my subjective experience) have horrible failure rates, particularly in power supplies. Of course, with a sample size of only a few dozen of each type, it’s possible I’m drawing too much of a generalization from too limited a set 🙂

Pink Fluffy Bunnies Unite!

I read a posting on one of my mailing lists today that just really got my goat. Unfortunately, my response was, I think, a bit too much advocacy of a certain point of view to be tolerated on that mailing list, so I’ve posted it here where I control what I think is OK and isn’t 🙂 The thread of conversation was this (paraphrased):

I read a posting on one of my mailing lists today that just really got my goat. Unfortunately, my response was, I think, a bit too much advocacy of a certain point of view to be tolerated on that mailing list, so I’ve posted it here where I control what I think is OK and isn’t 🙂 The thread of conversation was this (paraphrased):

“Don’t blame Jesus for [a certain faith]”

To which another poster, apparently a fellow nonbeliever, responded:

“Your words have more wisdom than you know! Blaming Jesus would be like blaming the Pink Easter Bunny for Easter, including egg rolling and too much candy!”

Lumping Jesus in the same category as the Easter Bunny touched off a nerve in the raving Catholic of the list, someone known only pseudonymously as “ELC”. Below is my complete response to his short rant.

—-

Note that I’m trying hard to avoid preaching here 🙂 Unfortunately, reasoned argument is the key evangelical method for atheist advocacy. The handle of this blade also cuts. Realize I’m not advocating atheism, just correcting a misconception.

ELC said:
> Oh, the blind faith [and stupidity] of atheism!

First, I’ll address the “blind faith” of atheism, relevant definitions from Dictionary.com:

a-the-ism, n. [syn: godlessness] [ant: theism] 2: a lack of belief in the existence of God or gods.

faith, n. 2. Belief that does not rest on logical proof or material evidence
4. The theological virtue defined as secure belief in God and a trusting acceptance of God’s will.

I’m guessing, like many theists, you define atheism by this alternative entry, also at dictionary.com:

atheism, n. 1b. The doctrine that there is no God or gods.

If atheism were simply dogmatic assertion that there is no God, I’d agree with you that it would require blind faith. However, for most, atheism is just the default position if you have *no* faith. If there is any “faith” to be found among many atheists, it is that there must be logical proof or material evidence for everything.

As far as the “stupidity” of atheism: have you ever met an unintelligent atheist? I admit, this is a limited subset of all atheists, but the atheists I know arrived at that position after a great deal of study. There are also enormous numbers of “closet atheists” in various religions (including LDS) who often refer to themselves by a different term: intellectuals.

Now, if you’re asserting that atheism itself is stupid, I must assume you mean this definition of “stupid”:

stupid, adj 5. Pointless, worthless: a stupid job

To the contrary, the refusal of faith itself promotes critical thinking in all aspects of life. But I suspect, rather than the clinical definition, you meant “stupid” as a simple perjorative, as in “something I don’t like”.

Gallery back up, maintenance

Well, I caved, and decided to stick to the old PHP Gallery software rather than going to the new Drupal module. It just represents too many hours of work for me to want to change over. I’ll be doing some maintenance to bring it up-to-date, and there’s definitely quite a bit of a style change between the main barnson.org site and the Gallery at the moment, but I’ll be hacking them both to bring them into line soon.

It’s available via the “images & photos” link on the right. Now we just have to find our digital camera so that we can upload more…

Well, I caved, and decided to stick to the old PHP Gallery software rather than going to the new Drupal module. It just represents too many hours of work for me to want to change over. I’ll be doing some maintenance to bring it up-to-date, and there’s definitely quite a bit of a style change between the main barnson.org site and the Gallery at the moment, but I’ll be hacking them both to bring them into line soon.

It’s available via the “images & photos” link on the right. Now we just have to find our digital camera so that we can upload more…

Downtime and Lessons Learned

Well, barnson.org, and its sister site, outlanders-outfit.org, were down over the weekend. It is a brief object lesson on interdependencies in the Internet world; click “read more” for details.

Well, barnson.org, and its sister site, outlanders-outfit.org, were down over the weekend. It is a brief object lesson on interdependencies in the Internet world; click “read more” for details.

You see, I had a script on Outlanders Outfit page that monitored our outfit’s Teamspeak server. The script has worked pretty well for months, with occasional problems that generally worked themselves out in short order. Here’s the code. It ran within a “block” entry on the sidebar of my Drupal front page for Outlanders. Drupal drives barnson.org as well. Pat yourself on the back if you can figure out the problem:

//Display formatting to match our block layout...
echo ("<div class=\"box\">
<p class=\"boxtitle\">Who's On Teamspeak</p>
<div class=\"item-list\">");

// opens a connection to the teamspeak server

function getSocket($host, $port, $errno, $errstr, $timeout) {

global $errno, $errstr;

$socket = fsockopen($host, $port, $errno, $errstr, $timeout);

if(!$socket or fread($socket, 4) != "[TS]") {

echo("Server error: Teamspeak server not running!");

return false;

}// end if

return $socket;

}// end function getSocket(...)



// sends a query to the teamspeak server

function sendQuery($socket, $query) {

fputs($socket, $query."\n");

}// end function sendQuery(...)



// returns the result of the last query

function getResults($socket) {

return fgets($socket);

}// end function getResults(...)



// closes the connection to the teamspeak server

function closeSocket($socket) {

fputs($socket, "quit");

fclose($socket);

}// end function closeSocket(...)





// ---=== main program ===---



// establish connection to teamspeak server

$socket = getSocket("teamspeak.outlanders-outfit.org", 51234, $errno, $errstr, 1);

if($socket == false) exit;



// select the one and only running server on port 8767

sendQuery($socket, "sel 8767");

if(getResults($socket) != "OK\r\n") {

echo("Internal server error! Cause unknown.");

exit;

}// end if



// retrieve player list

sendQuery($socket,"pl");



echo("<table><thead><th align=\"CENTER\">ID</th><th>&nbsp;</th><th align=\"CENTER\">Name</th></thead>\n");

$counter = 0;

do {

$playerinfo = fscanf ($socket, "%s %d %d %d %d %d %d %d %d %d %d %d %d %s %s");

list($number, $d, $d, $d, $d, $d, $d, $d, $d, $d, $d, $d, $d, $s, $name) = $playerinfo;

if($number != "OK") echo("<tr><td align=\"CENTER\">$number</td><td>&nbsp;</td><td align=\"CENTER\">$name</td></tr>\n");

$counter++;

} while($number != "OK");

if($counter == 1) echo("<tr><td colspan=\"3\" align=\"CENTER\">No players on server</td></tr>\n");

echo("</table>\n");

// close connection to teamspeak server

closeSocket($socket);

echo ("</div><p><a href=\"node/view/55\">Click here to use Teamspeak yourself!</a></div>");

If you look closely, nowhere do I handle timeouts. That means that php is waiting on the timeout value of the operating system to indicate that it was unable to connect to a remote host.

This is bad on a web site with even a moderate number of hits. Outlanders is small, but several sections (notably, some of my Humorous Pictures get quite a few reads and are occasionally linked from high-traffic sites. Kind of funny, that, since I pretty much just grabbed someone else’s images and put my own captions on them.

To compound the problem, as a temporary workaround I disabled the .htaccess file controls for the site, and replaced the index page with a plain HTML page that said “Ouldanders is down for the moment, it should be back Monday morning”. I had thrown that up there when I hadn’t figured out the problem yet, and figured that would help it hold until I returned from vacation late Sunday night.

Well, apparently the Teamspeak server operator decided my site was querying his TS server too much (and I must agree, it hit the TS server pretty much for every page read on my site due to the way I have this configured), so set the Teamspeak server to drop packets from my virtual host.  Not a problem for him, but I eventually ended up with my Apache server maxxed out on processes as it waited for the return from Teamspeak.  It doesn’t take many people clicking “refresh” waiting on a 3-minute operating system timeout on a tcp connect() to max out your Apache processes.

  1. My FreeBSD server began spiralling apache processes out of control, because it was having to wait on a timeout for the Teamspeak monitor for each process that never came.
  2. This continued for an entire day.
  3. The automated monitoring scripts caught on that my virtual server was sucking up too much RAM.
  4. Rather than just killing the offending process, it shut down my entire VDS (jailed FreeBSD server)
  5. I was offline until the 65535.net admin intervened.

So, the lesson learned is this: when I have a dependency on another system, I need to be a bit more careful in how I handle communication with that other system.

I cranked down my StartServers for this Apache to 2, the MinSpareServers to 2, MaxSpareServers to 3, and Maxclients to 20.  MaxClients had been set to 150, and each Apache process takes up 7.1Mbytes.  Since I’d maxxed out Apache with all these processes just spinning their wheels, that sucks up an entire gigabyte of RAM.  I don’t think these little VDS servers even have that much RAM!

With MaxClients set to 20, it means that I won’t survive a Slashdotting very well, but I also won’t suck up all the RAM on this poor box.  It makes me wonder how well putting a Squid proxy-cache in front of Apache as a reverse HTTP accelerator might work, since there seems to be plenty of CPU horsepower — just not very much memory.

Anyway, that’s the nutshell of why barnson.org and outlanders-outfit.org were unavailable for the weekend.  I learned a little bit as I sorted out what was dying.  The one thing I need to figure out, though, is where Drupal stores the Blocks configuration and Modules configuration settings in MySQL.  If I hadn’t had a browser window left up since Friday that had been logged into the Outlanders administration menu (and thus had the right cookie to admin without login) I might never have been able to use the web interface to login and make changes.  That was pretty hairy.  So I’ll be plunging into the code, figuring out first how to emergency-admin Drupal, and then writing some error checking/shorter timeout values and caching into my Teamspeak monitoring script and then negotiate with the TS2 server admin to allow my server to query his again.

Whew, what fun!

Home “sick”?

You know, if I weren’t a logical person, I would think it’s time to stop chewing gum.

I’ve had three teeth now that have had various problems.

  1. I broke this one in half crunching down on a popcorn kernel. The half that broke off was irreperably shattered when I masticated the next bite.
  2. Another tooth simply fell apart while chewing gum.
  3. And the third tooth, several weeks ago, felt like the middle just dropped out of it while chewing gum. Suddenly, just like tooth two, I had some extra ‘crunchies’ in my gum out of nowhere, and gum kept sticking to that tooth due to the large hole that just opened up in it.

You know, if I weren’t a logical person, I would think it’s time to stop chewing gum.

I’ve had three teeth now that have had various problems.

  1. I broke this one in half crunching down on a popcorn kernel. The half that broke off was irreperably shattered when I masticated the next bite.
  2. Another tooth simply fell apart while chewing gum.
  3. And the third tooth, several weeks ago, felt like the middle just dropped out of it while chewing gum. Suddenly, just like tooth two, I had some extra ‘crunchies’ in my gum out of nowhere, and gum kept sticking to that tooth due to the large hole that just opened up in it.

Downright weird! In the last two cases, I’d had a clean bill of health from the dentist on prior examination, but the decay was either disguised by an existing filling, or in the case of tooth 3, it was at an angle that was difficult to discern in standard X-rays.

Anyway, I got it repaired yesterday. I’m not normally nervous at the dentist, but for some reason yesterday I was nearly terrified. It was a very uncomfortable feeling, but after a lot of pain for several hours yesterday, the tooth today doesn’t hurt anymore and just feels a little sensitive to cold. And it’s nice that I can go back to my gum-chewing with no pain and no gum sticking into a hole.

But I still worry that I’ll end up like many of my relatives, with massive dental problems by the time I’m forty. And I brush twice a day and floss once a day! Then again, I didn’t get into that habit until I was twenty-five and began worrying more about my health. I guess twenty-five years of poor dental hygeine are just catching up with me…

Religion, Sunstone, and Good Friends

This has been an interesting week on the religion side. By and large, I’ve simply preferred to avoid discussing religion due to my somewhat… *ahem*… non-standard beliefs in this predominantly LDS town. However, this week I’ve had a couple chances to yak with people about it, and while it’s been slightly tense and stressful from time to time, overall it’s been very positive.

There have been four specific incidents; click “read more” below if you’re interested.

This has been an interesting week on the religion side. By and large, I’ve simply preferred to avoid discussing religion due to my somewhat… *ahem*… non-standard beliefs in this predominantly LDS town. However, this week I’ve had a couple chances to yak with people about it, and while it’s been slightly tense and stressful from time to time, overall it’s been very positive.

There have been four specific incidents; click “read more” below if you’re interested.

The Sunstone Episode

Anyway, out of the blue an acquaintance who lives in my neighborhood, Dan Wotherspoon, leaves me a note in a thick, large envelope filled with three “Sunstone” magazines. I won’t reprint the whole thing here, but the gist was that he’d noticed I haven’t been to church in many months, and his wife had heard that I’d read some things and had some doubts about the church.

Not quite on the mark, there. I consider myself largely agnostic (although lately I’ve begun referring to myself as a “Bright“), but I’d read precisely zero anti-mormon literature before the day I decided I could not find sufficient evidence for the existence of God to continue purporting to believe. Unfortunately, I was still Sunday School teacher for several months after that and it’s pretty tough to try to teach one thing and think another. Oh, but that’s beside the point.

Anyway, I popped by his house to ask about his motivations. I’d heard about this Sunstone thing before, but only in the context of some General Authorities getting really upset about people considering themselves liberal, intellectual, or homosexual Mormons and putting together a commitee to watch them and eventually kick them out of the church for their non-faith-promoting activities.

Sunstone wasn’t at all what I thought.

Dan and I talked for about 40 minutes, and by the end I was convinced this wasn’t some sort of strange fellowshipping effort on his part. From where he sat, the news that I was an “atheist” (yeah, I used that word, because I really detest the fact it has such a negative connotation yet really means “not a believer”, not “anti-religion”) was a surprise — I guess he thought I followed the normal track of acquiring anti-mormon literature, questioning, etc. Guess my effort to remain somewhat incognito regarding my beliefs panned out there. He reassured me that I wasn’t “the talk of the ward”, and that he’d belabored writing me for about two hours. He’s the editor and a member of some commitee for Sunstone, and invited me to attend their symposium if I’d like to.

I’m kind of thinking I’d like to. Although Sunstone seems less like Mormon intellectuals, and more like Mormon fence-sitters, nevertheless I think there might be some kindred spirits there, and I certainly agree with (most) LDS moral ideals. There are some areas where I think conservative LDS philosophy is very over-the-top, but at the same time until I find another organization which can provide friendship for my children and family with similar goals in clean living and morals, I’ll probably be sticking around.

Still doesn’t mean I’ll pray over meals or offer “blessings” of any sort, but I’m comfortable bowing my head in respect for the beliefs of the person offering them.

The Justin Conversation

I yakked with my buddy Justin on the phone for several hours the other night, and he admitted to me he was a bit shaken up by my disclosure of the fact that I don’t consider myself religious anymore, and have, in fact, acknowledged that I don’t know if there are even Gods or other supernatural beings. I hope he won’t mind me talkinga about this; if he does, I’m sure he’ll let me know. Anyway, he’d had a conversation with his wife talking about the state of his soul and his own questions. I’m no religious advocate, but I hope talking to helped him sort out where he is, and what he wants to become. It was a pretty deep chat, and sometimes just having a friend to talk to is important.

He was there for me when I was weirded out about the whole MRI thing, chronicled earlier in my blog. Friendships go both ways. Glad to have a friend.

The Sam Call

I emailed Sammy G, one of my best friends from high school, wondering how to get in touch with Ben. This flowered into about 30 messages over the course of one day, with me CC’ing Ben and Sam on every message, just catching up on life and people we knew. He heard that I was kind of thinking about religion, and gave me a call to see how I was doing. Although it’s been a year since I let my wife know about my change of religious beliefs, I still struggle every day with de-programming myself and trying to see things as they really are, rather than how I’d prefer to filter them thruogh my own preconceptions. It’s incredibly difficult and draining, but rewarding as I feel better able to cope with reality.

Anyway, Sam’s a bass player in several bands. His advice? “Believe in four strings and the truth”. Never heard better advice. Maybe you have to be a musician to get it.

The Daughter’s Questions

My daughter, Sara, also ended up inquiring about religion this week, asking me why I don’t pray over meals anymore (it’s taken her a year to notice?), science, and how everybody’s related through Adam and Eve. I gave her the brief overview, and typical of children she simply accepted it and said “that’s cool, Dad”. We even started making up games to play with the Scientific Method, figured out how to make a hypothesis, and did a few experiments and observations to come to know the world around us. I think Zach is still a bit young (at nearly 6 years old) to totally “get it”, but he tried hard 🙂 Baptism time for Sara is rapidly approaching, as she was born in November. Being that this is usually a family occasion, if my mother or my dad haven’t already read this blog and figured it out, I’ll have to make sure they know what’s up before they get surprised that someone else is baptizing my children. That is, if Sara decides she wants to be baptized. I want to make sure she knows that this is a choice, baptism is an optional thing that you should do only if you believe in Jesus and want to show him that you are willing to be born all over again to show your commitment to him. I’m pretty certain she’ll want it to please her Mom, and since I’ll be officially neutral on the issue after explaining the options, she’ll figure out who she won’t upset by her actions.

Oy, veh, and with the upcoming birth of our next child in February, life will become even more interesting.

At least now, I feel like I can approach every day honestly. I have a clear conscience that I’m not pretending at being anything that I’m not, that I’m willing to change when I encounter truth, and that I won’t allow peer pressure and groupthink to force my choices. I know what’s right, I know what’s wrong, and I know that I put everything I had into discovering the truth about deity, reality, and faith. Now I’m into discovering truth, helping others cope with change, and trying to improve the world. My contributions are largely through message boards and mailing lists, but it’s a wonderful feeling to know that I’ve been where somebody else is, and I can share my coping strategies with them. You don’t have to go it alone in your disbelief; there are millions of people who are also honest with themselves and acknowledge they don’t know.

Done yakking. My hope is just that some other person out there, one day, can read this and realize there’s somebody else who’s been in their shoes. My name’s Matthew Barnson, and I live in Tooele, Utah. You can easily look me up and give me a call if you’re in that situation, and you’ll find a sympathetic ear.

As long as you don’t call in the middle of the night. I’m grumpy that way 🙂

Together Again For The First Time

I have very few friends on the face of this planet. Justin Timpane is one of them. Three of my best friends in high school were Sammy G, Ben Schuman, and Kevin Graham.

So Justin found himself in the possession of a CD, created by Brett Clawson (an acquaintance of his and mine). Sam, Ben, Kevin, and I formed a band in high school called “Wayward Sun”. Brett had taken it upon himself to extract the audio from a tape we’d made, entitled “Together Again For The First Time”, and put it into CD format. That must have taken hours; I couldn’t believe someone would put in that much time.

Well, Justin took it a step further. He ripped all the songs, piped them through Cool Edit, applied some noise reduction and other techniques to restore the sound quality, and here’s the result! Unfortunately, our original master was crap, and that’s reflected in garbling on a couple of the tracks, but the songs still sound better than they ever did on tape. Click “read more” to read the full description and either download or stream the MP3 files.

I have very few friends on the face of this planet. Justin Timpane is one of them. Three of my best friends in high school were Sammy G, Ben Schuman, and Kevin Graham.

So Justin found himself in the possession of a CD, created by Brett Clawson (an acquaintance of his and mine). Sam, Ben, Kevin, and I formed a band in high school called “Wayward Sun”. Brett had taken it upon himself to extract the audio from a tape we’d made, entitled “Together Again For The First Time”, and put it into CD format. That must have taken hours; I couldn’t believe someone would put in that much time.

Well, Justin took it a step further. He ripped all the songs, piped them through Cool Edit, applied some noise reduction and other techniques to restore the sound quality, and here’s the result! Unfortunately, our original master was crap, and that’s reflected in garbling on a couple of the tracks, but the songs still sound better than they ever did on tape. Click “read more” to read the full description and either download or stream the MP3 files.

All songs are 320kbps MP3, so if the tune sounds bad, blame it on our mastering & recording abilities at the time, not on the MP3. This, unfortunately, means you have absolutely zero chance of streaming these files unless you have a 480kbps or higher connection. (ISDN? Forget it. DSL? Maybe. Cable? Probably, as long as it’s not a peak time.)

Wayward Sun was:

  • Jon Brusco: Bass. He left the band before we produced any tapes. If I recall correctly, he and Sam were like peanut butter and motor oil: they just don’t make a good sandwich together. Funny that they were my two best friends, but didn’t like one another much. Ben replaced him.
  • Ben Schuman: Keyboards, Vocals, Guitar
  • Kevin Graham: Drums, Keyboards, Vocals.
  • Sammy G: Keyboards, Bass Guitar, Vocals.
  • Matt Barnson: Keyboards, Guitar, Vocals.
  • Ed Copeland: Guitar. He replaced me when in 1992 I went on a mission for the LDS church, and was the guitarist featured on “No Further ?”, the band’s third album.

Yeah, it was kind of funny that we all played the piano. We used this in a couple performances by having all the band members switch positions to play a different instrument. I’ll never forget Ben Schuman showing me up on guitar playing a cover of “Silent Lucidity” by Queensryche. I’m still bitter 😉

So, enough with the formalities!

Stream the whole shebang from song 1 to song 10 by clicking this link.

  • Wayward Sun, our title track! I believe Ben’s Schuman’s exact words, after playing bass on the keyboard at our first rehearsal after Jon Brusco stopped playing bass for us, were “Guys, playing bass for this song is really boring.” I think it was at that moment that Sam began saving money towards buying his first bass guitar. Sam, wasn’t that bass Jon’s old one, actually?
  • A Moment Away From You. Featuring Ben Schuman on lead vocals, this piece forever cemented Kevin Graham’s hate of electronic drums, and Matt Barnson’s shortcomings as a lead guitarist.
  • Pushing Us Down. The requisite 16-year-old rage against the power of parents. The whole tune started one day when I was goofing off with the Ensoniq synthesizer in the front room of my folks home. Sam was chilling out on the couch in the family room, around the other side of the house. I had it turned up pretty loud. Anyway, I was playing with the sequencing function, just noodling. I noodled for about an hour, then took a break for a minute and heard Sam yell from the next room “Hey, don’t stop playing, I’m writing words for that!”. Thus the song was born. I think Sam was pissed at his Dad that day. Or was it is his sister? He spent a lot of time pissed in high school 🙂
  • Wrong Words. Required slow tune. This also was a fun tune to kick around as a quartet at parties. Kevin would sit down to the piano, and the rest of us would crowd around. We’d act like we were so into the music, really artsy-like, all the while eyeing the girls that would inevitably cluster aound us like seagulls on garbage. Mmm, I don’t think I like that allegory, but anyway, it was a great chick-grabbing tune 🙂
  • Madness. This was one of Wayward Sun’s signature tunes. We liked it so much that at one of our concerts poolside, we were asked to tone it down by the management. Kevin had a pretty killer double-drumset for that show and from that point on began lusting for double kicks.
  • Mad Mad World. This is Ben, all Ben, except some vocals! This one fits into the “great tune but impossible to play live” category. And made Kevin even more bitter about electronic drums 🙂 If I ever upload our second album, you’ll notice that we never had just a drum machine on our next two albums. I sang along with this one yesterday, and realized I could hit the low notes. When we originally recorded it, we had to speed up the tape so that I could. And then Ben overdubbed me anyway 😉
  • Sweet Lori. What was Lori’s last name, anyway? I can’t remember, but she was one of those “untouchable” girls. She was, to my 15-year-old eyes, incredibly beautiful, delicate-looking, sweet high soprano voice, and acerbic wit that made you think you were stung by a butterfly. Apparently, Sam thought the same and wrote this ode to her.
  • Into The Night. This is, quite possibly, one of the most boring tunes in existence. I wrote it one day following onto the success of Sam’s rebellious “Pushing Us Down”. I figured I could do “evil” just like Sammy G. I learned that NOBODY does Evil better than Sammy G. And here’s the permanent record of it. I click the “skip” button in XMMS (an MP3 player like WinAmp) when I get to this one.
  • The Sky Is Always There. This is, alternatively, titled “This Guy Is Always There” because, well, it’s impossible to tell which one is the right words. According to our first-ever concert for, oh, crap, what was my old girlfriend’s name? (Edit in February 2005: Her name was Grace. Don’t remember the last name, but she was the girlfriend that had pity on me for passing out during photos and cracking my head against the stage floor in high school). Her dad handed us $200 for performing there when we’d agreed to work for $30. That permanently inflated our fragile egos, I think. Anyway, I introduced the song as “written for a dead aunt”, and unfortunately pronounced “aunt” “ant”. Alas, the imagery of singing to an insect is more powerful than the original intent.
  • On My Way. It provided a nice bookend for the opening tune, but unfortunately our “fast song skills” at that point rendered “Wayward Sun” and “On My Way” as sounding really, really similar. This song is probably most memorable as living on forever as a much slower tune where Ben Schuman lampooned all of our band’s tunes. Think Weird Al’s “Polkas on 45”, but a little stranger.

The Genesis of Wayward Sun (repost from old blog)

This is a repost from my old blogging software, Movable Type. I figure, given my planned posting of some old Wayward Sun tunes, it would be good to give a little background here and there.

—————————–

February 28, 2003
The Genesis of Wayward Sun

Ever been interested in how the terrifically obscure East-coast band, Wayward Sun, got its start? Read below to find out more. I’m trying to make it moderately entertaining, but it may not be so to anybody but me 🙂

Wayward Sun, or Two Guys and a Dead Cat

I have a thing for cats. I like them alive, I like them dead… I just think they are cute, fluffy, and the name “cat”, rhymes with enough other words that it’s pretty convenient at some point to have a cat in most songs.

This is a repost from my old blogging software, Movable Type. I figure, given my planned posting of some old Wayward Sun tunes, it would be good to give a little background here and there.

—————————–

February 28, 2003 The Genesis of Wayward Sun

Ever been interested in how the terrifically obscure East-coast band, Wayward Sun, got its start? Read below to find out more. I’m trying to make it moderately entertaining, but it may not be so to anybody but me 🙂

Wayward Sun, or Two Guys and a Dead Cat

I have a thing for cats. I like them alive, I like them dead… I just think they are cute, fluffy, and the name “cat”, rhymes with enough other words that it’s pretty convenient at some point to have a cat in most songs.

One day in the summer of 1988, my buddy Jon Brusco and I were chilling in his garage. I’d just bought a really cool new electric guitar. It was a ?Mako? brand that I’d picked up at Victor Litz, the local guitar store, for about $75. I was freaking EXCITED, man. Unfortunately, I didn’t yet have an amp for my guitar, so Jon plugged his bass into his amp, and I plugged my guitar into the second input jack on the amp and we plugged away. The song of the day was an innovative little ditty I’d proudly entitled, “Fred the Cat”.

Fred Fred the Cat Fred was Dead On the welcome mat

Whoah, Fred. Fred the Cat. Fred was fat That was that

Fred. Fred. The. Cat. (end with flourish, attempt at soloing, rolling around on the garage floor on the back, etc.)

In case you’re interested, the chord progression was D minor, G major, C major, A minor. Make up a tune to match that chord progression, and you’re probably pretty close to what I sang in at least one of the rounds of singing this song. Jon and I were in stiches over it. Well, at least I was; he laughed politely a few times.

Anyway, so we’re chilling in the garage, and we figured out that we wanted to make a band. Of course, the first step in making any garage band is to figure out a really cool name. We tossed around some ideas we thought were cool, stuff like: Flying Butt Monkeys Artistic Nonsense Mud Scrapers Satin Knights Heinlein Lookalikes Football Umpires Wayward Sun Overdue

We eventually decided it was a toss-up between ?Satin Knights? (from a Moody Blues song, “Knights in White Satin”, if I recall correctly), and “Wayward Sun” (a play on “Carry on Wayward Son”, by Kansas). Like all good things that needed deciding, the best choice to make the decision for us was Jon’s Mom. In my opinion, she was the epitome of all things Good and Wise. I thought she was pretty, too, and for a 15-year-old, pretty was pretty important.

“Hey, Mom, I,” Jon began, then had a thoughtful look and restated, “I mean, ‘we'” he corrected, grinning at me, “have a question for you.” “What’s that?” she replied, shutting off the vacuum cleaner she was using on the upstairs hallway rug. “Umm, we’re trying to decide on the name for our band. It’s a choice between Satin Knights…” Jon began, “…and Wayward Sun,” I concluded. “Which one do you think is better?” asked Jon earnestly. Jon’s Mom had a thoughtful look for a moment, then spoke slowly. “Satin Knights sounds like somebody that wants to get you into their beds. Wayward Sun sounds like a disobedient child. Given the choice between a disobedient child and someone trying to seduce me, I’d take the disobedient child.”

Thus the band was born: Wayward Sun it was!

We practiced that summer, and I wrote a theme song for the band. One of these days, I’ll get an .ogg stream up on my weblog so that people can listen to it, but since the original sources appear to be permanently stuck in Ben Schuman’s grandmother’s basement, they will probably be pretty poor quality rips from one of the crappy tapes we made. Maybe one day I’ll fly back East to pick the tapes & original recording deck up to try and re-master the stuff.

Unfortunately, I’m out of time for the weblog today, so I’ll continue this story in another entry later.

For Jenny, For Josh

I wrote “For Jenny, For Josh” a few years ago upon the death of a good friend’s son. Josh had a very troubled time for his short life, and his death, although not unexpected, was still heartwrenching. I hope that my little tune brings solace to those who mourned, and honors his memory for his parents and others who’ve never met him.

I’ve often though about adding more accompaniment to this piece, and yet may, but by itself the piano seems to work pretty well. Still a work-in-progress, though, as evidenced by several errors in my performance. I’ll work on those before I upload this one to Garageband.

Download For Jenny, For Josh.

I wrote “For Jenny, For Josh” a few years ago upon the death of a good friend’s son. Josh had a very troubled time for his short life, and his death, although not unexpected, was still heartwrenching. I hope that my little tune brings solace to those who mourned, and honors his memory for his parents and others who’ve never met him.

I’ve often though about adding more accompaniment to this piece, and yet may, but by itself the piano seems to work pretty well. Still a work-in-progress, though, as evidenced by several errors in my performance. I’ll work on those before I upload this one to Garageband.

Download For Jenny, For Josh.

New version of My Lady!

OK, so I’m doing recording on a bunch of tunes. I know my one dedicated listener has heard “My Lady” before, but I decided to re-record it so I could get a better arrangement, mix, and recording quality. Hate to put up the same song twice, but the arrangement is different enough that I’m thinking you may take notice!

You can download the new track here. Let me know what you think! I was playing around with mastering options on this one, too, so I’m hoping the overall volume is louder and the song carries more energy.

OK, so I’m doing recording on a bunch of tunes. I know my one dedicated listener has heard “My Lady” before, but I decided to re-record it so I could get a better arrangement, mix, and recording quality. Hate to put up the same song twice, but the arrangement is different enough that I’m thinking you may take notice!

You can download the new track here. Let me know what you think! I was playing around with mastering options on this one, too, so I’m hoping the overall volume is louder and the song carries more energy.