Month: August 2003

So I’ve had the lovely task of dealing with the recent sobig.f outbreak on the Internet where I work. The same dunderheads that let themselves get infected by the last big virus failed to run Windows Update so that they could be prevented from getting this one.

It just goes to show that border security, basically, isn’t. People set up ways of blocking the bad stuff from getting to them, but they don’t bother to fix the underlying reason the “bad stuff” can cause problems in the first place. The moment anything makes it through the border, it can cause all the havoc it wants to. People aren’t taking responsibility for keeping their nodes secure on this big, wide Internet world, and the lack of their adequate policing is causing problems for the rest of us.

I ran into a related experience with my daughter this morning. I’m working from home today, since I have been keeping an eye on the pain to our mail server from virus transmissions. Anyway, she was getting some laundry out of the washer and the washer lid fell on her head. The obvious, rational conclusion is that she bumped the washer, causing the lid to dislodge from its open position and land on her head. I responded to the screaming wail of pain, brought over an ice pack, and asked for the explanation of what happened.

Now, we could propose alternative explanations, I suppose. But Sara insisted, “I didn’t bump the washer. It wasn’t my fault. The washer lid hit me in the head, and I don’t know how it happened!” I carefully explained cause and effect to her, that our goal is not to place blame but to figure out what happened, and how it happened, so that, in this case, we can prevent it from happening in the future.

She sullenly accepted my explanation and stalked off back to the washroom to move her laundry, this time without the accompanying crashing noises and loud crying.

But it made me think of the whole virus, and the tendency we human beings have to avoid responsibility for bad things. In one way, the recent outbreak of the RPC worms attacking Windows workstations was a good thing: the annoyance factor of rebooting your machine every 3 minutes or less forced people to update their PCs and take responsibility for helping police the Internet. I know it will be short-lived, but it’s progress. I’m just glad it wasn’t a very destructive worm, or recovery would have been far more painful than it was.

So, anyway, sobig.f is floating around today and I’m updating postfix rules. Here’s the meat of it.

Add these two entries to /etc/postfix/main.cf (or, if you’re using FreeBSD, /usr/local/etc/):

 body_checks = regexp:/etc/postfix/body_checks header_checks = pcre:/etc/postfix/header_checks.pcre 

Then you need to create the files “/etc/postfix/body_checks” and “/etc/postfix/header_checks.pcre”. I distinguish “.pcre” files this way, because that stands for “Perl Compatible Regular Expressions”, which are slightly different than normal “regex” regular expressions. If you don’t have PCRE support compiled into Postfix, the header_checks.pcre file won’t help you at all, and will actually cause Postfix to not start up at all, or in some cases just spit an error message out to your syslog.

Anyway, this is body_checks:

 # sobig rejection # The following statement should all be on one line, # with a space before "reject" # It's two lines due to formatting constraints. /^TVqQAAMAAAAEAAAA\/\/8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA$/ REJECT keep your viruses

# Klez rejection # The following statement should all be on one line, # with a space before "reject" # It's two lines due to formatting constraints. /^<iframe src=3Dcid:\S+ height=3D0 width=3D0>/ REJECT No IFRAMEs please /^<FONT>/ REJECT No viruses wanted here

##############################

This is header_checks.pcre (this can be multi-line as formatted):

 /^\s*Content-(Disposition|Type).*name\s*=\s*"?(.*\.( ade|adp|bas|bat|chm|cmd|com|cpl|crt|dll|exe|hlp|hta| inf|ins|isp|js|jse|lnk|mdb|mde|mdt|mdw|msc|msi|msp|mst|nws| ops|pcd|pif|prf|reg|scf|scr\??|sct|shb|shs|shm|swf| vb[esx]?|vxd|wsc|wsf|wsh))(\?=)?"?\s*(;|$)/x REJECT Attachment name "$2" may not end with ".$3" 

#################################

So, the nice thing here, is that these two rules will check for any attachments of known nasty types, and simply refuse to allow them to be delivered, period. The unfortunate weakness here is that if someone uses base64 encoding on the mail, we aren’t really checking it. You really need a virus-scanning package on the back-end for that. On the plus side, few virusses use base64 — it’s mostly reserved for spammers and people who aren’t using a native English mailers.

Hope that helps! It took a little bit of digging on mailing lists to get this far, but so far it seems to be helping us a lot. On the back-end, I have a script that just uses iptables (on Linux) to stop abusive mailers. Once I’ve sanitized out the stuff that definitely won’t apply to someone else’s environment, I’ll post that here, too.

P.S. Yeah, I know “view as PDF” is broken for this doc right now. I’m not certain how to properly handle <pre> tags when formatting for PDF, so you’ll have to be content with what’s here for now 🙂 Besides, I expect you would rather just cut & paste what’s above, rather than print it out and type it, wouldn’t you? Thought you would.

The headline:

U.N. HQ in Baghdad Car Bombed – FLASH: Large car bomb attack at U.N.’s Baghdad HQ. [Winds Of Change]

What, they don’t have JERSEY BARRIERS in Baghdad? I mean, the White House wasn’t even a war zone 15 years ago when I remember them wheeling in the big concrete barricades to stop any potential vehicular bomb from getting anywhere near the building. I note with some twisted humor that the Salt Lake City Wallace F. Bennett Federal Building seems engineered to prevent drive-ups, with large decorative, but obviously stout and concrete, pylons. I was just thinking the other day, “Wow, someone driving a car bomb would have a heck of a time getting near that building. Maybe a motorcycle bomb or something could make it, but how much damage could something that small do?”

Important buildings should have a perimeter of Jersey Barriers preventing casual drivers-by from unloading a car bomb into them. Even if they’re temporary. I wonder which U.N. genius dropped the ball deciding not to Jersey-Barrier this puppy?

I’m in the midst of a discussion with a close friend, via email, of some of the fundamental questions regarding the Bible, Christian thought, and the evolution of religion. In my searches, I came across this simply amazing, honest article, written by Richard Packham and entitled “How I Became An Atheist”.

And, of course, the title itself will put off some of my readers. Try another of Packham’s essays, The Man Who Bought A House, to really understand where he’s coming from. If you find yourself strongly disagreeing, perhaps you, too, have bought the house? His web site is an excellent collection of essays he’s written and links he’s collected over a lifetime of skepticism.

I kept saying to myself, “Oh, man, this is me. Did this guy read my mind?” Only obvious dissimilarities (like the fact he’s at least 35 years older than me, and graduated from college with a law degree) kept me grounded in the reality that this wasn’t my history I was reading. I could see myself writing an essay similar to this.

As a matter of fact, it was research into how I’d write a similar essay that led me to his site. Now I’m not so sure I want to write one of my own, since I’ve found one that so closely mirrors my own perceptions. Time will tell.

I’m just using this node to play trackback games with outlanders-outfit.org to make sure they are talking.

The auto-discovery node is here.

Ran across an interesting post over at http://windsofchange.net/archives/003927.html regarding the establishment of permanent “paramilitary” operations. My useless commentary below.

On the one hand, I think it’s a really good idea to involve the locals more when we are forced into involvement on foreign soil. Patriots freeing their own country seems much more of a productive idea than a bunch of ugly Americans watering the fields of another nation with the blood of its defenders. At the same time, I fear the permanent establishment of an international paramilitary police corp.

Will the U.S. miliatary have the wisdom to administer this kind of program on a permanent basis? While on the one hand I have high hopes, on the other hand the beaurocracy inherent to these organizations will really get in the way.

I guess I’m conflicted about the arrangement. On the one hand, I realize it’s probably necessary for us to have greater involvement in international affairs, and to reduce our handprint worldwide at the same time, but on the other hand I just wish we could mind our own business and be left alone.

Would be kinda’ nice, wouldn’t it? Particularly the part about having four hands…

This is yet another excerpt from a conversation I’ve been having on one of my mailing lists. If you’re not into religious recovery, you probably won’t be into this one. If you are into discussion about theology, philosophy, and personal choice, you might be interested.

We have a newcomer to the list, by the name of Leanne. She had this to say:

I have been reading lots of the posts since I joined this group awhile ago now. I feel like a freak. Are there any members of this group that actually miss the church? I wish it was so true I wish I could go back in time to the time before I knew the church wasnt true. There must be some people out there that are sad about having all your memories and dreams dashed??? I would appreciate hearing from anyone that shares my feelings. Thanks Leanne

---

Leanne,

I apologize in advance for the length of my post. I generally only write if something has touched a nerve…

It’s taken me a year to get where I am now (still quite “attached” to the church in family and environment, yet being completely open with all about my non-belief). A year ago, I was right where you are now. I was on the cusp; the decision lay before me to continue to say one thing with my mouth, and believe something else.

The night that I told my wife about my nonbelief, I cried into my pillow for over an hour. I normally don’t cry. As I wept, my wife comforted me, and I kept saying “it’s just so hard; I really want it to be true.” I look back now and think it’s a testament to my wife’s devotion that she chose not to try to dissuade me; if she had, at that vulnerable point, I may have chosen to live a lie the rest of my life. I’m certain that, had I done so, my life would have been short. Forcing myself to say one thing yet believe another had driven me to the brink of despair, where I needed to either put a bullet in my head and be done with it, or face my disbelief squarely and try to mend the gulf in my mind between what I wanted to believe, and what was real.

The more outspoken ones on this list tend to be those who have gone past that hard part, and are growing more firm (or are firm) in their newfound beliefs/non-beliefs — whatever those are. I’ve been on this list only a little over a month, and the reassurance I’ve gained from people has helped enormously, for me to take positive strides in my life, slowly end my fence-sitting, and improve my relationship with my wife and children. I feel less like a freak than I did before.

What you’re feeling is perfectly normal. Once you’ve gained the perspective that your hope for the church to be “true” is right there with your longing for the days when you believed your parents invulnerable, or when you thought yourself immortal, it gets easier.

Losing faith is a painful experience. With my current non-religious perspective, it seems like you face a choice:

1. Replace your faith with another. You may choose to seek comfort in another congregation of believers in something. In particular, Christian or non-denominational religious bodies would welcome you with open arms, and you might find that what you really miss is the fellowship of others who believe in God or Jesus (or something else). That’s up to you. There are many who go this road, and find it satisfying.
2. Rebel against this system, and choose some belief system that is widely despised in the U.S. (assuming you’re in the U.S.), such as Wicca, Satanism, or New-Age psychology.
3. Learn to live without faith, and figure out a label for yourself. Or live without a label. Agnosticism/Atheism/Bright-ism/non-dogmatic Taoism or Buddhism and other philosphies bring strength to some who research them. Many incorporate portions of those philosophies into their own lives and draw strength from them. Living without the comfort of another “Church” is very difficult, but can also be satisfying. For those who wish fellowship to still be some portion of their non-religious practices, the Unitarian Universalists or Secular Humanist congregations can still grant that feeling of fellowship without much in the way of dogma to interfere in the lives of their patrons.

My road was to choose to live with a naturalistic worldview. Who knows, one day I may embrace faith, if I have a sufficiently compelling subjective experience to cause me to wish to do so. I don’t think that’s going to happen, though, and I think I’d question my subjectivity if I did.

You’re not weird for desparately wanting the Church to be true. It would make life so much simpler. But for those who see past the lies, attempting to gain true fulfillment through faith in the Church just doesn’t work. There are some for whom it’s a fit, and some for whom it is not.

Welcome to the world of the rest of us misfits.

I, for one, am happy to hear someone else express the same feelings I’ve had. I feel less like a freak for feeling that way now .

I wrote up a blurb on Slashdot today about my perspective on creating highly reliable, available, or fault-tolerant systems, and how you really need to choose which of the three you are going for in designing your compute environment.

I’ve also adjusted my opinion since I posted this. There are a couple more factors, which include initial expense and maintenance cost, that need to be factored in. They are normally the domain of the bean counter, but it’s important that the admin/systems architect be aware of what tradeoffs he is willing to make in order to bring those costs down, and where the sacrifice of reliability, availability, or fault-tolerance needs to be made. And he also needs to appraise the bean counter of the importance of those factors that are being lost, as well.

Proper machine administration is a balance of: * reliability * availability * fault-tolerance

Reliability generally refers to the components of a piece of hardware not breaking very often. In our Compaq DL360 and DL380 rackmount units, reliability is a critical factor in their power supplies. The power supplies are simply crap. The fans die, the supplies blow up. It’s terrible.

Availability, on the other hand, even if you have components or a system that is not highly reliable, it can still be highly available. For instance, 1U chassis systems with dual power supplies, although the power supply reliability tends to be low, nevertheless tend to be available despite the low reliability.

Fault-Tolerance is how well the system handles unusual circumstances. A system may be highly available and reliable, but if it does not handle system faults well, you may have a problem. Fault-Tolerance really refers to how well the system handles unusual conditions.

Now, I realize the distinctions seem rather vague. That is intentionally so! But separating the question into three parts helps grant admins a better look at what the weak points are of a system.

I generally prefer 1U or 2U units to have a single, reliable power supply, rather than dual power supplies with lower reliability. Because of the higher reliability of their PS, they tend to also be available more. However, you have very poor fault tolerance in the power area with just a single power supply, so you’ll normally need redundant systems in order to have a fault-tolerant environment. If that is the case, availability may suffer, for when the first machine goes down in that rare situation that the highly reliable power supply dies, you have some cutover time to the secondary system.

Really, you have to evaluate what’s most important for you. In the bank where I work, reliability is critical; fault-tolerance, somewhat less so. Availability is not so much of an issue. We close at 5:30, and really don’t do any business after that, so from 5:30 PM through 5:30 AM, our availability can be nonexistent for certain types of maintenance, and we’re just fine.

Like I said, the designations there are pretty arbitrary. But if you can come to an approach covering at least three angles on your machines and evaluate by those criteria, your overall uptime goals can be met in a way that suits your organization.

Me, I’m just sick of cheap hardware. I enjoy 1U rackmount systems, but I’d much rather have two higly reliable 2U boxes than four inherently less-reliable 1U units. Of course, vendors come into play there… Sun makes some killer 1U’s that never seem to die, while Compaq and Dell’s 1U (in my subjective experience) have horrible failure rates, particularly in power supplies. Of course, with a sample size of only a few dozen of each type, it’s possible I’m drawing too much of a generalization from too limited a set 🙂

I read a posting on one of my mailing lists today that just really got my goat. Unfortunately, my response was, I think, a bit too much advocacy of a certain point of view to be tolerated on that mailing list, so I’ve posted it here where I control what I think is OK and isn’t 🙂 The thread of conversation was this (paraphrased):

“Don’t blame Jesus for [a certain faith]”

To which another poster, apparently a fellow nonbeliever, responded:

“Your words have more wisdom than you know! Blaming Jesus would be like blaming the Pink Easter Bunny for Easter, including egg rolling and too much candy!”

Lumping Jesus in the same category as the Easter Bunny touched off a nerve in the raving Catholic of the list, someone known only pseudonymously as “ELC”. Below is my complete response to his short rant.

—-

Note that I’m trying hard to avoid preaching here 🙂 Unfortunately, reasoned argument is the key evangelical method for atheist advocacy. The handle of this blade also cuts. Realize I’m not advocating atheism, just correcting a misconception.

ELC said:
> Oh, the blind faith [and stupidity] of atheism!

First, I’ll address the “blind faith” of atheism, relevant definitions from Dictionary.com:

a-the-ism, n. [syn: godlessness] [ant: theism] 2: a lack of belief in the existence of God or gods.

faith, n. 2. Belief that does not rest on logical proof or material evidence
4. The theological virtue defined as secure belief in God and a trusting acceptance of God’s will.

I’m guessing, like many theists, you define atheism by this alternative entry, also at dictionary.com:

atheism, n. 1b. The doctrine that there is no God or gods.

If atheism were simply dogmatic assertion that there is no God, I’d agree with you that it would require blind faith. However, for most, atheism is just the default position if you have *no* faith. If there is any “faith” to be found among many atheists, it is that there must be logical proof or material evidence for everything.

As far as the “stupidity” of atheism: have you ever met an unintelligent atheist? I admit, this is a limited subset of all atheists, but the atheists I know arrived at that position after a great deal of study. There are also enormous numbers of “closet atheists” in various religions (including LDS) who often refer to themselves by a different term: intellectuals.

Now, if you’re asserting that atheism itself is stupid, I must assume you mean this definition of “stupid”:

stupid, adj 5. Pointless, worthless: a stupid job

To the contrary, the refusal of faith itself promotes critical thinking in all aspects of life. But I suspect, rather than the clinical definition, you meant “stupid” as a simple perjorative, as in “something I don’t like”.

Well, I caved, and decided to stick to the old PHP Gallery software rather than going to the new Drupal module. It just represents too many hours of work for me to want to change over. I’ll be doing some maintenance to bring it up-to-date, and there’s definitely quite a bit of a style change between the main barnson.org site and the Gallery at the moment, but I’ll be hacking them both to bring them into line soon.

It’s available via the “images & photos” link on the right. Now we just have to find our digital camera so that we can upload more…

Well, barnson.org, and its sister site, outlanders-outfit.org, were down over the weekend. It is a brief object lesson on interdependencies in the Internet world; click “read more” for details.

You see, I had a script on Outlanders Outfit page that monitored our outfit’s Teamspeak server. The script has worked pretty well for months, with occasional problems that generally worked themselves out in short order. Here’s the code. It ran within a “block” entry on the sidebar of my Drupal front page for Outlanders. Drupal drives barnson.org as well. Pat yourself on the back if you can figure out the problem:

//Display formatting to match our block layout...

echo ("<div class=\"box\">

<p class=\"boxtitle\">Who's On Teamspeak</p>

<div class=\"item-list\">");



// opens a connection to the teamspeak server



function getSocket($host, $port, $errno, $errstr, $timeout) {



global $errno, $errstr;



$socket = fsockopen($host, $port, $errno, $errstr, $timeout);



if(!$socket or fread($socket, 4) != "[TS]") {



echo("Server error: Teamspeak server not running!");



return false;



}// end if



return $socket;



}// end function getSocket(...)







// sends a query to the teamspeak server



function sendQuery($socket, $query) {



fputs($socket, $query."\n");



}// end function sendQuery(...)







// returns the result of the last query



function getResults($socket) {



return fgets($socket);



}// end function getResults(...)







// closes the connection to the teamspeak server



function closeSocket($socket) {



fputs($socket, "quit");



fclose($socket);



}// end function closeSocket(...)











// ---=== main program ===---







// establish connection to teamspeak server



$socket = getSocket("teamspeak.outlanders-outfit.org", 51234, $errno, $errstr, 1);



if($socket == false) exit;







// select the one and only running server on port 8767



sendQuery($socket, "sel 8767");



if(getResults($socket) != "OK\r\n") {



echo("Internal server error! Cause unknown.");



exit;



}// end if







// retrieve player list



sendQuery($socket,"pl");







echo("<table><thead><th align=\"CENTER\">ID</th><th>&nbsp;</th><th align=\"CENTER\">Name</th></thead>\n");



$counter = 0;



do {



$playerinfo = fscanf ($socket, "%s %d %d %d %d %d %d %d %d %d %d %d %d %s %s");



list($number, $d, $d, $d, $d, $d, $d, $d, $d, $d, $d, $d, $d, $s, $name) = $playerinfo;



if($number != "OK") echo("<tr><td align=\"CENTER\">$number</td><td>&nbsp;</td><td align=\"CENTER\">$name</td></tr>\n");



$counter++;



} while($number != "OK");



if($counter == 1) echo("<tr><td colspan=\"3\" align=\"CENTER\">No players on server</td></tr>\n");



echo("</table>\n");



// close connection to teamspeak server



closeSocket($socket);



echo ("</div><p><a href=\"node/view/55\">Click here to use Teamspeak yourself!</a></div>");

If you look closely, nowhere do I handle timeouts. That means that php is waiting on the timeout value of the operating system to indicate that it was unable to connect to a remote host.

This is bad on a web site with even a moderate number of hits. Outlanders is small, but several sections (notably, some of my Humorous Pictures get quite a few reads and are occasionally linked from high-traffic sites. Kind of funny, that, since I pretty much just grabbed someone else’s images and put my own captions on them.

To compound the problem, as a temporary workaround I disabled the .htaccess file controls for the site, and replaced the index page with a plain HTML page that said “Ouldanders is down for the moment, it should be back Monday morning”. I had thrown that up there when I hadn’t figured out the problem yet, and figured that would help it hold until I returned from vacation late Sunday night.

Well, apparently the Teamspeak server operator decided my site was querying his TS server too much (and I must agree, it hit the TS server pretty much for every page read on my site due to the way I have this configured), so set the Teamspeak server to drop packets from my virtual host.  Not a problem for him, but I eventually ended up with my Apache server maxxed out on processes as it waited for the return from Teamspeak.  It doesn’t take many people clicking “refresh” waiting on a 3-minute operating system timeout on a tcp connect() to max out your Apache processes.

1. My FreeBSD server began spiralling apache processes out of control, because it was having to wait on a timeout for the Teamspeak monitor for each process that never came.
2. This continued for an entire day.
3. The automated monitoring scripts caught on that my virtual server was sucking up too much RAM.
4. Rather than just killing the offending process, it shut down my entire VDS (jailed FreeBSD server)
5. I was offline until the 65535.net admin intervened.

So, the lesson learned is this: when I have a dependency on another system, I need to be a bit more careful in how I handle communication with that other system.

I cranked down my StartServers for this Apache to 2, the MinSpareServers to 2, MaxSpareServers to 3, and Maxclients to 20.  MaxClients had been set to 150, and each Apache process takes up 7.1Mbytes.  Since I’d maxxed out Apache with all these processes just spinning their wheels, that sucks up an entire gigabyte of RAM.  I don’t think these little VDS servers even have that much RAM!

With MaxClients set to 20, it means that I won’t survive a Slashdotting very well, but I also won’t suck up all the RAM on this poor box.  It makes me wonder how well putting a Squid proxy-cache in front of Apache as a reverse HTTP accelerator might work, since there seems to be plenty of CPU horsepower — just not very much memory.

Anyway, that’s the nutshell of why barnson.org and outlanders-outfit.org were unavailable for the weekend.  I learned a little bit as I sorted out what was dying.  The one thing I need to figure out, though, is where Drupal stores the Blocks configuration and Modules configuration settings in MySQL.  If I hadn’t had a browser window left up since Friday that had been logged into the Outlanders administration menu (and thus had the right cookie to admin without login) I might never have been able to use the web interface to login and make changes.  That was pretty hairy.  So I’ll be plunging into the code, figuring out first how to emergency-admin Drupal, and then writing some error checking/shorter timeout values and caching into my Teamspeak monitoring script and then negotiate with the TS2 server admin to allow my server to query his again.

Whew, what fun!