I’ve been using Python for a few years, but mostly for extremely simple and short stuff. By and large, truly, I’ve mainly just referenced someone else’s libraries in Python to do something that I couldn’t do easily in Bash. Today, I decided to finally knuckle down and undertake a conversion script I normally would have done in Bash using grep, awk, and sed, and instead do the whole thing in Python.
Now, I realize this isn’t very Pythonic. It’s quite procedural, no classes or functions. There are several sections where I think I could dramatically shorten it by splitting some of the more repetitive areas out into classes. But for a first real attempt at using Python as a replacement for Bash shell scripting, it was fun!
Takes input from a file with a format like this:
filername-nfs:/vol/some_volume_name/some_qtree_name 20G private on some_hostname mounted at /some/path/to/nfs/mount
Now, the goal here is to automagically spit out a series of commands that I can copy/paste and run in various windows. I need it to calculate the size of all the qtrees in a volume and add 20% (divide by 0.8) and give me an integer to use in gigabytes. So imagine you just received a file containing, say, fifty different mounts in this format for perhaps 10 different hosts and two or three different Netapp Filers. At that point, the time savings become obvious! Using just the one sample above, the output from this file looks like this:
Volume size commands: rsh filername vol size /vol/some_volume_name +25G some_hostname /etc/fstab entries: filername-nfs:/vol/some_volume_name/some_qtree_name /some/path/to/nfs/mount nfs rw,bg,nointr,hard,timeo=600,wsize=32768,rsize=32768,nfsvers=3,tcp 0 0 some_hostname mkdir & mount commands: mkdir -p /some/path/to/nfs/mount mount /some/path/to/nfs/mount filername /etc/exports entries: /vol/some_volume_name/some_qtree_name -rw=some_hostname-nfs:root=some_hostname-nfs filername /etc/quotas entries: /vol/some_volume_name/some_qtree_name tree 20G
Here’s the script.
#!/usr/bin/env python
# Short little script to awk up things and do replacements in several files.
# Basically the same thing I do from bash, but in python.
import sys
# Debug this session?
debug = False
# Set the options we’ll spit out for mounts.
# Typically, you don’t need to alter this.
# But for Sun 7000-series filers, you need to add “,noacl” after “tcp”
nfsOptions = ” nfs \
rw,bg,nointr,hard,timeo=600,wsize=32768,rsize=32768,nfsvers=3,tcp 0 0″
# Name of the file we’re working from.
myFile= open( “qtrees.txt”, “rU” )
allQtrees=[]
for myRow in myFile:
try:
# Filter out empty lines
if not myRow.isspace():
# Filter out comments
if “#” not in myRow:
# Remove newlines
rsplitRow=myRow.rstrip(‘\n’)
# Split into space or tab-delimited fields like awk.
allQtrees.append(rsplitRow.split())
except ValueError:
pass
# Initialize a few variables…
mountContainer = {}
mountFiler = {}
mountVolume = {}
numRows = len(allQtrees)
for column in range(numRows):
# 0: nfs path, 1:size, 2:private/public, 3:, 4:host, 5:, 6:, 7:mountpoint
# print allQtrees[column]
myColumn = ( allQtrees[column] )
fullNfsPath = myColumn[0].split(“:”)
nfsFiler = fullNfsPath[0]
nfsPath = fullNfsPath[1]
nfsVolumeAndQtree = fullNfsPath[1].split(“/”)
#print nfsVolumeAndQtree
nfsVolume = nfsVolumeAndQtree[2]
nfsSize = myColumn[1]
privateOrPublic = myColumn[2]
nfsHost = myColumn[4]
mountPoint = myColumn[7]
# We’re creating several Lists of Lists to be put into Dictionaries here.
# Main purpose is to make it easier for me to visualize the data, I’m sure
# there’s a more efficient way than creating all these structures.
# Wrap this list by hostname
try:
listByHost = mountContainer[nfsHost]
except:
listByHost = []
listByHost.append([nfsFiler, \
nfsPath, \
nfsSize, \
privateOrPublic, \
nfsHost, \
mountPoint])
mountContainer[nfsHost] = listByHost
# Wrap this list by filer
try:
listByFiler = mountFiler[nfsFiler]
except:
listByFiler = []
listByFiler.append([nfsFiler, \
nfsPath, \
nfsSize, \
privateOrPublic, \
nfsHost, \
mountPoint])
mountFiler[nfsFiler] = listByFiler
# Wrap this list by volume
try:
listByVolume = mountVolume[nfsVolume]
except:
listByVolume = []
listByVolume.append([nfsFiler, \
nfsPath, \
nfsSize, \
privateOrPublic, \
nfsHost, \
mountPoint])
mountVolume[nfsVolume] = listByVolume
# Unwrap mountVolume for a list of volume sizes.
for volume in sorted(mountVolume):
volSize = 0
print
print “Volume size commands:”
for size in sorted(mountVolume[volume]):
qtreeSize = int(size[2].strip(‘G’))
curSize = str(qtreeSize)
#print “Qtree size of ” + size[1] + ” is ” + curSize
volSize = volSize + qtreeSize
#print volSize
volSize = int(volSize / 0.8)
volSizeStr = str(volSize)
print “rsh “ + size[0].strip(“-nfs”) + ” vol size /vol/” + volume + ” +” \
+ volSizeStr + “G”
# Unwrap mountContainer for a list of mounts by host.
for host in sorted(mountContainer):
mkdirCmds = []
mountCmds = []
print
print “%s /etc/fstab entries:” % (host)
for mount in sorted(mountContainer[host]):
print mount[0] + “:” + mount[1] + ” “ + mount[5] + nfsOptions
mkdirCmds.append(mount[5])
mountCmds.append(mount[5])
print “%s mkdir & mount commands:” %(host)
for path in mkdirCmds:
print “mkdir -p “ + path
for path in mountCmds:
print “mount “ + path
# Unwrap again for /etc/exports & /etc/quotas on filer
for filer in sorted(mountFiler):
print
print “%s /etc/exports entries:” % (filer.strip(“-nfs”))
for volQtree in sorted(mountFiler[filer]):
print volQtree[1] + ” -rw=” + volQtree[4] + “-nfs:root=” + volQtree[4] + \
“-nfs”
print
print “%s /etc/quotas entries:” % (filer.strip(“-nfs”))
for volQtree in sorted(mountFiler[filer]):
print volQtree[1] + ” tree “ + volQtree[2]
if debug:
for name in dir():
myvalue = eval(name)
print name, “is”, type(name), “and is equal to “, myvalue
myFile.close()