Obsolete Your Idols

It's been a couple years since I did anything useful with the code which JunkBBS runs, itself a mild fork of bbs100. Last Sunday, just for fun, I got it to build on my personal laptop. A few tweaks, some modernization of idioms, and it built. Sweet.

Then for the past 24 hours, I've been gradually merging in bits from later releases of bbs100 than the one I based JunkBBS code on. It's still early in the process, but I'm optimistic this will get me out of the weeds on this project [which has languished for 5 years] and let me get my hands dirty with actual programming once more.

That's what I did last night when I actually left work at a Usual Time. Tonight when I left at an Absurdly Late Time, I sat in front of a fan and wrote this post.

I hope to set up a flow soon so I can blog from my Sidekick (in addition to twittering which it does very well, indeed) perhaps through the tumblelog, perhaps through this blosxom instance.

Sometimes I read the blogs of strangers and it inspires me.

Last night I read a tumblelog called Anarchaia which I think I probably found through some other stranger's blog on my curious staggers through the thoughts of people who code. In particular, the highlighted Thought on that page got me to thinking. So I tried scratching the itch in code and this is what fell out.

 #!/usr/bin/perl
 # splitrot by Shannon Prickett <binder@manjusri.org>
 # rotate vowels separately from consonants

 use strict;
 use warnings;

 #use Smart::Comments;

 my $consonant_string   = 'bBcCdDfFgGhHjJkKlLmMnNpPqQrRsStTvVwWxXzZ';
 my $vowel_string = 'aAeEiIoOuUyY';

 while (<>) {
     chomp;
     for my $letter (split //, $_) {
         my $rotted;
         if ($letter !~ qr{[$vowel_string]}msx) {
             $rotted = rot_n({ character => $letter, offset => 10, string =>
             \$consonant_string  });
         }
         else {
             $rotted = rot_n({ character => $letter, offset => 3, string =>
             \$vowel_string });
         }
         print $rotted;
     }
     print "\n";
 }

 sub rot_n {
     my $arg_ref = $_[0];
     my $character           = ${arg_ref}->{character};
     ### $character
     my $offset              = ${arg_ref}->{offset};
     my $letter_string_ref   = ${arg_ref}->{string};
     my $letter_string       = $$letter_string_ref;
     ### $letter_string

     if ($character =~ m{ [[:space:]|[:punct:]] }msx) {
         return $character;
     }

     my $character_index = index( $letter_string, $character );
     ### $character_index
     my $result_index = $character_index + ($offset * 2);
     ### $result_index
     if ($result_index > (length( $letter_string ) - 1)) {
         $result_index -= ($offset * 4);
     }
     ### $result_index

     my $return_letter = substr( $letter_string, $result_index, 1);
     ### $return_letter
     return $return_letter;
 }

That's not as horrible as I had feared it would be when all was said and done. Some examples:

 binder@death:~/src/r13$ ./splitrot 
 A man, a plan, a canal, SUEZ! 
 O zob, o cxob, o pobox, GEUM!
 O zob, o cxob, o pobox, GEUM!
 A man, a plan, a canal, SUEZ!
 You mean it just contradicts me all day?
 Iae zuob yh vegh pabhfoqyphg zu oxx qoi?
 Iae zuob yh vegh pabhfoqyphg zu oxx qoi?
 You mean it just contradicts me all day?
 binder@death:~/src/r13$

See? Reversible. Pronounceable? I don't think so. But you're free to pabhfoqyphg zu if you dare.

Put here because I keep having to search the web for it every time I need to do it:

openssl req -new -outform PEM -out smtpd.cert -newkey rsa:2048 -nodes -keyout smtpd.key -keyform PEM -days 365 -x509

Then point postfix at the key and cert file from main.cf. You'll need to connect with the clients once to tell them to accept this new updated certificate. One-liner swiped from urbanpuddle.

The Courier imapd-ssl provides a mkimapdcert script to do the Right Thing for that side of things.

Just for future reference, if you've already done this

tar xjf linux-source-2.6.15.tar.bz2 linux-source-2.6.15/Documentation/HOWTO

then this

tar xjf linux-source-2.6.15.tar.bz2 linux-source-2.6.15/Documentation/*

probably won't do what you expect.

This, however

tar xjf linux-source-2.6.15.tar.bz2 linux-source-2.6.15/Documentation/\*

will. Silly shell, that wild-card isn't for you!

Unrelated, I finished playing the brilliant video game, Psychonauts, tonight. Very rewarding of the time and effort put into seeing it through to the closing cinematic.

Some combination of motherboards and CPUs on three separate systems at work went belly up this past week. One of them was mine. In an attempt to expedite getting back up and running, systems slated for some users got shifted to be available for the down users.

So the plus is that I have a faster CPU, more RAM and more disk, as I was able to migrate my disks into the new case and with some fiddling recover the data. The only complicated aspect of the operation was that I used the alternate CD ISO of Feisty Fawn and had some device-mapper issues trying to recognize the LVM partition I had been using. Overnight, without changing or updating anything, that issue went away. While that kind of spontaneous self-repair makes me nervous, I was just happy to recover my data.

But it wasn't that big of a deal because I use source control.

In fact, I use two systems.

For work output, the stuff I get paid for, I use the Perforce client since it's a free download for Linux and the choice of source control depot was made before I got there.

But I also use monotone for my personal configurations, ie, my home directory. I've got a script to deploy the pieces into place:

#!/bin/bash -x

cd ~/sharedhome

# deploy .files
for newfile in dot*; do
        oldfile=.${newfile:3}
        if [ ! -L ~/${oldfile} -o -d ~/${oldfile} ]; then
                mv ~/$oldfile ~/${oldfile}.bak
        fi
        ln -sf ~/sharedhome/$newfile ~/$oldfile
done

# system executables
SYSPATH=/usr/local/bin
CRONPATH=/etc/cron.daily
for cronfile in cron*; do
        oldfile=${cronfile:4}
        if [ -f ${SYSPATH}/${oldfile} ]; then
                sudo mv ${SYSPATH}/${oldfile} ${SYSPATH}/${oldfile}.bak
        fi
        sudo ln -sf ~/sharedhome/${cronfile} ${SYSPATH}/${oldfile}
        if [ -f ${CRONPATH}/${oldfile} ]; then
                sudo rm ${CRONPATH}/${oldfile}
        fi
        sudo ln -sf ${SYSPATH}/${oldfile} ${CRONPATH}/${oldfile}
done

# personal executables
HOMEBINPATH=~/bin
cd ~/sharedhome/scripts
for binfile in *; do
        if [ ! -L ${HOMEBINPATH}/${binfile} ]; then
                mv ${HOMEBINPATH}/${binfile} ${HOMEBINPATH}/${binfile}.bak
        fi
    ln -sf ~/sharedhome/scripts/${binfile} ${HOMEBINPATH}/${binfile}
done

# ssh pieces
HOSTNAME=`hostname`
cd ~/.ssh/
for localpiece in *.${HOSTNAME}; do
    LOCALIZED=`basename $localpiece .${HOSTNAME}`
    ln -sf ~/.ssh/${localpiece} ~/.ssh/${LOCALIZED}
done

It takes some pains to make sure it won't trample over something I want to keep while still putting the things I expect to find in the appropriate places. It could probably be smarter but it's sufficient to my needs.

The monotone database is easily replicated. I use the sync over ssh mechanism to do so as you can see here in my collection of aliases. I got this concept from the O'Reilly Hacks books.

# dotbash_aliases

alias realias="${EDITOR} ~/.bash_aliases &&  source ~/.bash_aliases"

alias reperl='perl -de0'

alias gor='mount /mnt/thumb && ~/bin/tclkit ~/bin/gorilla && umount /mnt/thumb'

alias malias='sudo vi /etc/aliases && sudo newaliases'

alias p4i='p4 integrate -b $0 $1'

alias cshare='cd ~/sharedhome && mtn commit .' 

alias sshare='cd ~/sharedhome && mtn sync ssh://fool.manjusri.org/home/binder/monotone/manjusri-configs.mtn "*"'

alias ushare='cd ~/sharedhome && mtn update'

alias ducks='du -cks * |sort -rn |head -11'

alias scan='sudo iwlist eth1 scan'

alias wmii2='export DISPLAY=:0.1 && wmii'

alias cdburn='sudo cdrecord dev=ATAPI:0,1,0 -v '

alias devmono='cd ~/src/monotone && mtn pull && mtn update'

So that just leaves the stuff which is neither in perforce nor in monotone and that data isn't really interesting or vital. But as with any data threat, it's a cue to back up all of my existing data everywhere.

Once upon a time I had a machine with a hard drive in it. It ran for a couple years, running Debian stable. I added a second and then a third [smaller] hard drive to it. At some point I decided I wanted to be more clever about these devices and so I converted the non-swap partitions to LVM physical volumes. That was pretty cool because now I could dynamically resize partitions as I had different file system usage patterns.

But something weird started happening. I went through a couple physical moves, a couple releases of Debian stable, and I couldn't seem to get any newer kernels to be visible to grub.

At one point, I moved to a new place, changed the network configuration, rebooted to re-cable things and when it came up, the network configuration had reverted. In sorting that out, I discovered that I had a file-system which was serving as both my root file-system AND as a physical volume under LVM. OOPS. But I got that sorted out and things went along more or less normally, other than the no new kernel thing, until yesterday.

When I upgraded my monotone client instance on the work laptop [you see, I store my home directory in revision control and deploy it on systems I work frequently with] and monotone being the dynamic and giddy project it is, this upgrade made for a protocol incompatibility with the 'repository' [pronounced: synchronized database] I keep on the quirky system. So in trying to update that instance of monotone to the same version, I faced a choice of: upgrading to Debian testing, upgrading to Debian unstable, building monotone from source, side-grading to Ubuntu 6.06 LTS.

The right answer was building from source.

What I did was the side-grade. Or at least I started it. Somewhere along the way, the lvm2 package upgrade preinst script failed. I re-fired the dist-upgrade a couple times, made some progress, "decided" [here, the word represents sort of a random coin flip of synapses which is similar to the way other people decide things but lacking the illusion they try to dress it up in of it being a rational process] to reboot the system to get the newer installed kernel in play to see if that unstuck the lvm2 upgrade.

Nope. In fact, it very rapidly became worse. Much worse. Booting still only showed the original kernel I've been running on for years. Booting it forced me into maintenance mode. Sometimes it resulted in a runaway init being killed. Oh, bad times.

Luckily I keep a copy of the sysrescuecd in the home. I booted up with it, took a look at what was going on and kept "looking" [here, this word means jotting things down on paper and doodling and staring off into space because while I'm not a visual person I do seem to observe better with paper at hand] until I finally got it. By which I mean, I found a command which I wouldn't have expected to need, which I had never used before, but which made clear the lvm2 problem I'd been having.

vgreduce.

vgreduce? Yes. Because I got to wondering if I was using all of the physical volumes, I remembered how one of my disk partitions had been doing double duty before, and I was thinking along the lines of maybe carving out a partition to be a proper non-LVM /boot and put my kernel stuff in there so I could keep an eye on it, since booting to an LVM /boot seemed to be a sort of sometimes hit but mostly miss proposition for me. So I ran vgreduce with the -t switch and, wouldn't you know it, there were three partitions not being used and, uh, there was one which was being used twice. OOPS. How'd that happen?

So I got that sorted out, rebooted and ... kernel panic. So back to the System Rescue CD boot, scope it out again, do some various tomfoolery with grub and then I realized I had committed the Classic Grub Blunder.

The first partition on the second hard drive seen by BIOS, as seen by grub? That's hd(1,0). Everyone knows that one. The first partition on the first drive? hd(0,0). Third partition, first drive? hd(0,3). BZZZZZT No. No, it's not. hd(0,3) points at the fourth partition. Which, in my instance, was a swap partition and had been since system creation. For years, the only reason it was booting at all was that I page out so rarely that enough of a kernel image was still written there by grub to bootstrap me up to the running kernel. So once I got that sorted out, reboot, see many exciting kernel choices, boot without error AND with network again AND the dist-upgrade completed. PHEW.

That's why vgreduce is my hero of the day. Because until I used that to untangle my very first attempt at an LVM configuration, I couldn't even see the deeper underlying grub issue.

It's possible I just accidentally gave myself a reason to do something I have put off for a decade.

That is, start using Python.

I know, I know. Python is the first choice of many people. It's great for rapid development; just like Perl. It's good with objects; just like Perl. It's got stellar extensibility; just like Perl. It's ubiquitous on the platforms I work with; just like Perl. That's the problem. In the space where I might hang Python off of my tool-belt, Perl has already got a lock. I've been waiting for someone to show me something I can only do in Python to give me a reason to add it to my repertoire.

That hasn't quite happened, yet. But I did run across this startling state of affairs recently.

binder@fortune:/tmp$ valgrind --leak-check=full perl -e '1'
==13518== Memcheck, a memory error detector.
==13518== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==13518== Using LibVEX rev 1471, a library for dynamic binary translation.
==13518== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==13518== Using valgrind-3.1.0-Debian, a dynamic binary instrumentation framework.
==13518== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==13518== For more details, rerun with: -v
==13518== 
==13518== 
==13518== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 1)
==13518== malloc/free: in use at exit: 242,924 bytes in 617 blocks.
==13518== malloc/free: 4,030 allocs, 3,413 frees, 385,702 bytes allocated.
==13518== For counts of detected errors, rerun with: -v
==13518== searching for pointers to 617 not-freed blocks.
==13518== checked 567,912 bytes.
==13518== 
==13518== 5 bytes in 1 blocks are definitely lost in loss record 2 of 10
==13518==    at 0x401C422: malloc (vg_replace_malloc.c:149)
==13518==    by 0x80AC4AD: Perl_savesharedpv (in /usr/bin/perl)
==13518==    by 0x8066831: (within /usr/bin/perl)
==13518==    by 0x8068BB7: perl_parse (in /usr/bin/perl)
==13518==    by 0x805FCB0: main (in /usr/bin/perl)
==13518== 
==13518== 
==13518== 72,329 (1,557 direct, 70,772 indirect) bytes in 11 blocks are definitely lost in loss record 8 of 10
==13518==    at 0x401C422: malloc (vg_replace_malloc.c:149)
==13518==    by 0x80AB5B8: Perl_safesysmalloc (in /usr/bin/perl)
==13518==    by 0x80E7E86: Perl_new_stackinfo (in /usr/bin/perl)
==13518==    by 0x806182F: Perl_init_stacks (in /usr/bin/perl)
==13518==    by 0x8061C28: perl_construct (in /usr/bin/perl)
==13518==    by 0x805FC6E: main (in /usr/bin/perl)
==13518== 
==13518== 
==13518== 131,716 bytes in 1 blocks are possibly lost in loss record 10 of 10
==13518==    at 0x401C422: malloc (vg_replace_malloc.c:149)
==13518==    by 0x80AB5B8: Perl_safesysmalloc (in /usr/bin/perl)
==13518==    by 0x80B5A4B: Perl_reentrant_init (in /usr/bin/perl)
==13518==    by 0x8061E46: perl_construct (in /usr/bin/perl)
==13518==    by 0x805FC6E: main (in /usr/bin/perl)
==13518== 
==13518== LEAK SUMMARY:
==13518==    definitely lost: 1,562 bytes in 12 blocks.
==13518==    indirectly lost: 70,772 bytes in 586 blocks.
==13518==      possibly lost: 131,716 bytes in 1 blocks.
==13518==    still reachable: 38,874 bytes in 18 blocks.
==13518==         suppressed: 0 bytes in 0 blocks.
==13518== Reachable blocks (those to which a pointer was found) are not shown.
==13518== To see them, rerun with: --show-reachable=yes

Well. That's not great. Not terrible but not great. Then compare.

binder@fortune:/tmp$ valgrind --leak-check=full python -c '1'
==31768== Memcheck, a memory error detector.
==31768== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.  
==31768== Using LibVEX rev 1471, a library for dynamic binary translation.
==31768== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==31768== Using valgrind-3.1.0-Debian, a dynamic binary instrumentation framework.
==31768== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==31768== For more details, rerun with: -v
==31768== 
==31768== Conditional jump or move depends on uninitialised value(s)
==31768==    at 0x807F4EE: PyObject_Free (in /usr/bin/python2.4)
==31768==    by 0x8079DD5: (within /usr/bin/python2.4)
==31768==    by 0x8081146: PyString_InternInPlace (in /usr/bin/python2.4)
==31768==    by 0x8082C35: PyString_InternFromString (in /usr/bin/python2.4)
==31768==    by 0x808E8A9: PyType_Ready (in /usr/bin/python2.4)
==31768==    by 0x808F8BA: PyType_Ready (in /usr/bin/python2.4)
==31768==    by 0x807D930: _Py_ReadyTypes (in /usr/bin/python2.4)
==31768==    by 0x80DA1F5: Py_InitializeEx (in /usr/bin/python2.4)
==31768==    by 0x80DA788: Py_Initialize (in /usr/bin/python2.4)
==31768==    by 0x80553CE: Py_Main (in /usr/bin/python2.4)
==31768==    by 0x40AD31E: __libc_start_main (libc-start.c:237)
==31768== 
==31768== Use of uninitialised value of size 4
==31768==    at 0x807F4F7: PyObject_Free (in /usr/bin/python2.4)
==31768==    by 0x8079DD5: (within /usr/bin/python2.4)
==31768==    by 0x8081146: PyString_InternInPlace (in /usr/bin/python2.4)
==31768==    by 0x8082C35: PyString_InternFromString (in /usr/bin/python2.4)
==31768==    by 0x808E8A9: PyType_Ready (in /usr/bin/python2.4)
==31768==    by 0x808F8BA: PyType_Ready (in /usr/bin/python2.4)
==31768==    by 0x807D930: _Py_ReadyTypes (in /usr/bin/python2.4)
==31768==    by 0x80DA1F5: Py_InitializeEx (in /usr/bin/python2.4)
==31768==    by 0x80DA788: Py_Initialize (in /usr/bin/python2.4)
==31768==    by 0x80553CE: Py_Main (in /usr/bin/python2.4)
==31768==    by 0x40AD31E: __libc_start_main (libc-start.c:237)
==31768== 
==31768== Invalid read of size 4
==31768==    at 0x807F4E4: PyObject_Free (in /usr/bin/python2.4)
==31768==    by 0x8090265: (within /usr/bin/python2.4)
==31768==    by 0x80903A4: (within /usr/bin/python2.4)
==31768==    by 0x808E01D: PyType_Ready (in /usr/bin/python2.4)
==31768==    by 0x808F8BA: PyType_Ready (in /usr/bin/python2.4)
==31768==    by 0x807D958: _Py_ReadyTypes (in /usr/bin/python2.4)
==31768==    by 0x80DA1F5: Py_InitializeEx (in /usr/bin/python2.4)
==31768==    by 0x80DA788: Py_Initialize (in /usr/bin/python2.4)
==31768==    by 0x80553CE: Py_Main (in /usr/bin/python2.4)
==31768==    by 0x40AD31E: __libc_start_main (libc-start.c:237)
==31768==  Address 0x41F5010 is 120 bytes inside a block of size 384 free'd
==31768==    at 0x401CFCF: free (vg_replace_malloc.c:235)
==31768==    by 0x8079DD5: (within /usr/bin/python2.4)
==31768==    by 0x808EBA1: PyType_Ready (in /usr/bin/python2.4)
==31768==    by 0x808F8BA: PyType_Ready (in /usr/bin/python2.4)

[and so on until]

==31768== Invalid read of size 4
==31768==    at 0x807F4E4: PyObject_Free (in /usr/bin/python2.4)
==31768==    by 0x80F2CD7: PyGrammar_RemoveAccelerators (in /usr/bin/python2.4)
==31768==    by 0x80D8F35: Py_Finalize (in /usr/bin/python2.4)
==31768==    by 0x805579F: Py_Main (in /usr/bin/python2.4)
==31768==    by 0x40AD31E: __libc_start_main (libc-start.c:237)
==31768==  Address 0x422B010 is 56 bytes inside a block of size 916 free'd
==31768==    at 0x401CFCF: free (vg_replace_malloc.c:235)
==31768==    by 0x8055D94: (within /usr/bin/python2.4)
==31768==    by 0x80D9D1D: PyRun_StringFlags (in /usr/bin/python2.4)
==31768==    by 0x80A966D: (within /usr/bin/python2.4)
==31768==    by 0x80B63C6: PyEval_EvalFrame (in /usr/bin/python2.4)
==31768==    by 0x80B713A: PyEval_EvalFrame (in /usr/bin/python2.4)
==31768==    by 0x80B781E: PyEval_EvalCodeEx (in /usr/bin/python2.4)
==31768==    by 0x80B7A64: PyEval_EvalCode (in /usr/bin/python2.4)
==31768==    by 0x80D027E: PyImport_ExecCodeModuleEx (in /usr/bin/python2.4)
==31768==    by 0x80D0605: (within /usr/bin/python2.4)
==31768==    by 0x80D1420: (within /usr/bin/python2.4)
==31768==    by 0x80D1648: (within /usr/bin/python2.4)
==31768== 
==31768== ERROR SUMMARY: 811 errors from 103 contexts (suppressed: 19 from 1)
==31768== malloc/free: in use at exit: 654,054 bytes in 46 blocks.
==31768== malloc/free: 3,504 allocs, 3,458 frees, 1,560,141 bytes allocated.
==31768== For counts of detected errors, rerun with: -v
==31768== searching for pointers to 46 not-freed blocks.
==31768== checked 874,748 bytes.
==31768== 
==31768== LEAK SUMMARY:
==31768==    definitely lost: 0 bytes in 0 blocks.
==31768==      possibly lost: 0 bytes in 0 blocks.
==31768==    still reachable: 654,054 bytes in 46 blocks.
==31768==         suppressed: 0 bytes in 0 blocks.
==31768== Reachable blocks (those to which a pointer was found) are not shown.
==31768== To see them, rerun with: --show-reachable=yes

Seriously? Leak-less? OK, sign me up for your crazy British sketch comedy worshiping code-monkey cult.

UPDATE 2007/12/30: Oh, and lest I forget:
binder@fortune:~$ valgrind --leak-check=full ruby -e '1' ==9637== Memcheck, a memory error detector. ==9637== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al. ==9637== Using LibVEX rev 1471, a library for dynamic binary translation. ==9637== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP. ==9637== Using valgrind-3.1.0-Debian, a dynamic binary instrumentation framework. ==9637== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al. ==9637== For more details, rerun with: -v ==9637== ==9637== ==9637== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 21 from 1) ==9637== malloc/free: in use at exit: 335,613 bytes in 7,248 blocks. ==9637== malloc/free: 7,515 allocs, 267 frees, 338,631 bytes allocated. ==9637== For counts of detected errors, rerun with: -v ==9637== searching for pointers to 7,248 not-freed blocks. ==9637== checked 853,556 bytes. ==9637== ==9637== LEAK SUMMARY: ==9637== definitely lost: 0 bytes in 0 blocks. ==9637== possibly lost: 0 bytes in 0 blocks. ==9637== still reachable: 335,613 bytes in 7,248 blocks. ==9637== suppressed: 0 bytes in 0 blocks. ==9637== Reachable blocks (those to which a pointer was found) are not shown. ==9637== To see them, rerun with: --show-reachable=yes binder@fortune:~$ Poor ol' Perl.

A new year. 2007. How glorious and gleaming it looks from behind my dark lenses of hangover evasion.

My goal for this year is to try to make a tiny portion of my world a little bit better. In specific, I'm aiming to provide patches for fifty bugs in open source software. It's still a selfish goal, like my reading of fifty novels in 2006, since I'm almost certainly going to be noticing bugs in the things I tend to use most often and fixing the ones I think I understand. But maybe these fixes will be useful to someone else beside me.

Here's the first one to start the year off and it's one I wasn't sure I should count, as it's a trivial patch and evidently no one else has needed it but me. But this has to start somewhere, so here it is:

--- feedback    2006-07-25 05:33:24.000000000 -0500
+++ feedback.hostname   2007-01-01 12:43:08.000000000 -0600
@@ -801,8 +801,10 @@

     # Load Net::SMTP module only now that it's needed.
     require Net::SMTP; Net::SMTP->import;
+   use Sys::Hostname;

-    my $smtp = Net::SMTP->new($smtp_server);
+   my $hostname = hostname( );
+    my $smtp = Net::SMTP->new($smtp_server, Hello => $hostname);
     $smtp->mail($address);
     $smtp->to($address);
     $smtp->data();

This is a patch to Frank Hecker's feedback plugin for blosxom to address a problem I developed shortly after I made my mail exchanger more restrictive of who it would accept email from. This sets it to something more informative than the Net::SMTP default of localhost.localdomain which might help someone from just having the moderation requests dropped by their MX.

I just converted my workhorse system, which was my desktop machine in 1999 and has had a few minor upgrades since then and now is the core of my home network, from using a mix of reiserfs and ext3 file systems to having only ext3 file systems, managed by LVM. I'm documenting the process because I'm about to do it again to another machine and this probably won't be the last time I do it, either.

Why?

• wanted dynamic control and resizing of file systems
• had three physical devices
• wanted to use only file systems with dump/restore capability
• wanted to be able to take clean, quiescent file system dumps

How?

• identify a partition to start with

  • not in active use by the running OS
  • as big as another partition
  • small enough used that it can be copied off to another file system

• copy the data off of the target partition's file system

• unmount the target partition
• pvcreate on the target partition
• vgcreate using the target partition
• START OF LOOP: lvcreate using all available PE in the volume group; if you know the next partition you'll be working with, name the logical volume appropriately
• mkfs -t ext3 on the logical volume
• mount the new file system in a temporary location

• identify a partition to pivot

  • used space in the file system on the partition must be less than available in new file system
  • smoother if nearly quiescent, so if it's /var or similar, stop as many daemons as you can
  • best choices will be large, mostly unused file systems early on

• cp -a or rsync from the target partition's file system to the temporary mounted file system of the logical volume

• diff or re-rsync to verify currency; log files may well have updated during the replication
• unmount the logical volume from the temporary location
• edit /etc/fstab to reference the logical volume at its future permanent mount point
• if it's a file system in use, reboot; otherwise, unmount the target partition, mount the logical volume in its place
• pvcreate on the target partition
• vgextend using the target partition
• return to START OF LOOP until you've converted everything

I left the swap partitions alone as I already had equal priority swap partitions on each of several drives and it's easy enough to change their size without needing to reboot if the system is lightly loaded. So now I'll be using the lvm snapshot capability to take backups of the point in time state of my file systems. I can also respond to changing system needs in terms of space and file system architecture.

Maybe you've heard of David Allen. Smart guy, wrote a couple books. Here are a couple sites with discussion, debate, tips, tricks and amplifications of explanations. That said, these are the tools I've found most useful for implementing my own instance of GTD.

• Mosuki for calendaring. Friends recommended it and it's OK. Haven't compared it to others [upcoming, eventful, icalshare, et cetera] as it more or less does what I need it to do, it lets me schedule events with varying degrees of privacy and sharing. It may well be obsoleted by reminders from Backpack or my own installation of remind.
• Backpack pages for my SOMEDAY/MAYBE lists.
• The core of the system, Basecamp for Projects and Next Actions[*].

It's a sparse implementation, trading the need for web access to use, versus the very few physical components. It's a work in progress, like everything else, but this captures the essence of it at present.

The only part of it which costs me money is the Basecamp part and that was after I'd tried their Free plan for three months, to make sure it would let me manage the data the way I wanted to manage it. The Milestones and Writeboard features haven't come into play, yet.

* Specifically, the Personal level plan. Each of the three active Projects maps to a Scope [Collaborative, Personal and Professional]. Each To-Do list is a Project. Each Task on a To-Do list is a Next Action. Then the mildly clever bit: Each of several Users is a Context. Tasks are assigned to the User representing the execution Context. This means daily checklists by Context is a simple matter of filtering for Tasks owned by each User and the Weekly Review is done on the level of Projects, scanning the To-Do lists for needed Tasks and assigning them to an appropriate User.

I needed a sequence shuffle and I wasn't finding it at my usual one stop bash shop, nor some cursory web searches. I did manage to find a one-pass algorithm for doing so which could be expressed in two lines. But since I'm writing for other people, it got expanded in the implementation.

#!/bin/bash

# randomN by Shannon Prickett <binder@manjusri.org>
# share and enjoy Bouncy Bubbly Basherage
# arrange the elements 1-N in $RANDOM sequence

RANDOM=$$$(date +%s)

for index in $(seq 1 "$1")
do
     ARRAY[${index}]=${index}
done

for choice in $(seq 1 "$1")
do
     newchoice=`expr $RANDOM % "$1" + 1`
     oldvalue=${ARRAY[$choice]}
     newvalue=${ARRAY[$newchoice]}
     ARRAY[${newchoice}]=${oldvalue}
     ARRAY[${choice}]=${newvalue}
done

for element in $(seq 1 "$1")
do
     echo -n "${ARRAY[${element}]} "
done
echo

In case the intent is not obvious from that, it counts from 1 to whatever argument you give it and then scrambles that set of values in a single pass and spits it out at you. Like so.

stprickett@justice:/yr/mom$ ./randomN 5
2 5 1 3 4
stprickett@justice:/yr/mom$ ./randomN 5
5 4 2 1 3
stprickett@justice:/yr/mom$ ./randomN 20
19 14 9 13 4 2 6 20 7 18 10 8 3 16 12 11 15 1 5 17
stprickett@justice:/yr/mom$

Dear Perlhouse,

I've derived a lot of excitement and satisfaction from reading the letters you've printed over the years but I'd always assumed they were fictional. I never thought that the wild ternary operations or greedy matching regular expressitrixes which form the staple characters of these tales existed in the real world. As you can no doubt guess where I'm going with this, my mind was changed when I had an encounter of the sorting kind.
Now I'm writing in to share my experiences with your readers and to encourage them to read it with an open mind.

I was working late one night on the graveyard hacking shift when I noticed something out of the corner of my eye. It was a big number. Thirteen digits worth, in fact. A time-stamp with microsecond precision, a real looker. And it wasn't alone. There was a whole parade of those hotties. Over 400000, in fact. I could see from the look in their deer-like eyes that they had one thing on their minds.

They were looking to get sorted.

I wasted no time getting down and dirty with those beauties. But soon I noticed they weren't experiencing the anticipated squeals of joy. In fact, they weren't getting any satisfaction at all from my efforts. So I pulled a trick out of my bags and converted the vixens to Math::BigInts. Then I slicked up a subroutine for sort which invoked Math::BigInt->bcmp() and we were back on track, moaning and groaning.

Moaning and groaning for an infernally long time. With no release, no surcease, and definitely no post-coital peace. This wouldn't do. I had a reputation to protect as a guy who could fulfill the trickiest of data.

So I pulled out one of my old tricks for long lists, divide and conquer. I took those hungry leggy integers in groups of 100000 or so, I took the difference between any two of them, and I worked with that. Single pass processing, concentrating only on the juicy core which I really needed to satisfy their needs and my own, and no swap-busting allocations to sweat about.

When I was done, they were spent, I was satisfied, and I had another notch in my bedpost to mark their visit.

Last night I went to the meetup for the San Francisco MySQL Group and it was pretty decent. Nice space and there was a cake, to celebrate the release of 5.0.

When I went to the San Francisco PostgreSQL Group meeting last month, they were excited because 8.1 was entering release candidate status and it, too, has now released.

It's an exciting time to be a DBA, though perhaps less so if one is, for example, rebuilding a RAID array for a SQL Server instance. Or at least differently exciting.

I use a laptop with wireless networking on two networks on a daily basis. I'm also lazy. So I re-invented a wheel with this script:

#!/bin/bash
TMPFILE=`/bin/mktemp` || exit 1
/sbin/iwlist eth0 scan | grep ESSID > $TMPFILE

while read WANT SCHEME; do
     grep $WANT $TMPFILE &> /dev/null
     FOUND=$?

     if [ $FOUND = 0 ]; then
             echo $SCHEME
     fi
done

Which I invoke via this /etc/network/interfaces file:

mapping eth0
     script /usr/local/bin/find-essid
     map lairair eth0-home
     map MoodLogic eth0-work

# The primary network interface
iface eth0-work inet dhcp
     # wireless-* options are implemented by the wireless-tools package
     wireless-mode managed
     wireless-essid MoodLogic
     wireless-key It'sSecret,y0

iface eth0-home inet dhcp
     wireless-mode managed
     wireless-essid lairair

auto eth0

Running on Ubuntu but it should be fine on Debian. I realize the temporary file isn't needed but I like to leave a debug trail if it doesn't work.

I don't really watch a lot of television. Sometimes, I'll watch some SG-1 or SG:A via somebody else's Tivo. But one thing I do pay attention to is the EFF.

So when the Bay Area Debian mailing list started talking about the build-in, I got interested. Not interested enough to actually go to it, mind you. My weekends are about the only time I have to do my own thing and that often devolves into napping and wasting time on purpose. So while I was feverishly interested in the idea of building my own PVR, zealously interested in making some kind of stand against copyright assholes, and generally geeking out with hardware for a Linux project, I didn't think I could get into the idea of working on it with a group.

But I did read the cookbook and I looked at some prices. In particular, I looked at the price for the HD-3000 from pcHDTV. That price looked damned reasonable for what it would let me do. So while there's a TV in my home, used only for playing video games and watching movies, I decided that if I didn't jump on this, I'd be sorry, later. So I bought it.

Then I did some more thinking. I remembered how sometimes I'd buy a key piece for a project and stop there. Not doing anything more with it until it was too late, because I'd lost interest, or lost the key component or broken it. I could just see myself losing track of where I'd put the tuner card until I stepped on it and wasted the opportunity. So. I bought a box to put it in.

I've got maybe half a dozen machines in this house. Only one of them has been powered on since I moved in here. None of them is newer than 1995 era hardware. More to the point, none of them could meet the minimum standards of hardware required for this project. That meant a new machine, which would up the cost but virtually require me to follow through on it because I hate to waste money.

I'd been looking at the shuttle form factor for a few years [yes, I'm very slow to adopt technology] and I had seen some good prices at iBoxPC. So I optioned out the smallest system which would reach the bar set by doing digital TV decoding and storage and it ended up just around a grand. If that link has gone stale, it's a Checker P4 PC. I can't really recommend it, though.

The machine has distinct advantages in this role.

• Small footprint, so it can be near the TV
• Surprisingly quiet, for all the disk chugging
• Nice selection of ports and bays

As well as a few drawbacks, some particular to my environment.

• That tiny footprint comes at the cost of slots. There's one, count it, ONE slot of the PCI variety. Which is where the digital TV capture card goes. Which wouldn't be a huge problem, except...
• Go back and look at the HD-3000 card. See that blue cubic jack on the same side as the PCI pins? Where it wants to be, there are plastic encased jumpers with labels indicative of this being the on-board sound pins. These objects can't occupy the same physical space. Which meant...
• Plan A was to research USB sound devices for Linux, which went about as well as one might expect. For bonus laughs, try doing the same search for firewire. Yeah, that went nowhere, fast.
• So, Plan B. I used a pair of pliers and I gently shattered the plastic sheathes on four of the leads. Yeah, I ain't proud. In my defense, it did the trick. With the hard plastic gone, there was enough give in the wire to let everybody work together.
• Well, almost everybody. See, I've recently gone wireless in the Lair. Yes, partying like it's 2-0-0-0, here. But no more internal slots and no built in wi-fi make Homer something something. But have no fear, this is a solved problem for Linux. Well, solved if you can grit your teeth, hold your nose, and live with Windows cruft on your system. Which I did.
• Only, using it on my system is a thoroughly manual process. I mean that in the literal sense. Here's the sequence.
  1. Disconnect the [wi-fi dongle][wfd]
  2. Power on the system
  3. Log in to a console
  4. Plug the dongle into the system
  5. modprobe ndiswrapper to load the already stored driver
  6. ifup wlan0
  Any attempt to automate any stage of that only results in tears. Or, rather, resulted. Just today I updated to R5A15 but haven't completed that install so maybe the newer kernel has made this less tedious.

Previously, I had gotten it to the point where I could drive the NVIDIA [yes, yes, boo, hiss, I agree, but it was recommended by the cookbook and I wanted to minimize my effort; I do wish that NVIDIA would make it possible for people who find Windows intolerable to use their cards with full features in a way that didn't require OCD levels of hand washing afterward] card, capture TV, schedule capturing of any of the three channels Zap2It actually provides listings for in my area, watch something stored with full glorious sound and, most importantly, play Frozen Bubble. For my purposes, it was feature complete and a really cool project.

Then I messed it all up by updating via apt-get. Bad, bad, idea. In the future I'll only upgrade via KnoppMyth ISOs. I lost the ability to use ndiswrapper, NVIDIA cards, and mythfrontend, itself. I was down to Plan D for recovery by the time I found the newer KnoppMyth release and did the salt-the-earth 'upgrade' it offers. Nothing I've hit has been beyond my abilities and all the frustrations were minor in scope.

I gather my compadre Matthew has recently branched out from his appreciation of Tivo and had a tougher path by going the 64-bit architecture route, meaning much more building from source for him. He's doubtlessly more industrious than I am.

I wanted something a little lighter than stow for managing a group of symbolic links. I quite often find myself building an intricate tree of files somewhere on a filesystem and then wanting to be able to combine them in a single directory for some purpose without name collision. This isn't the smartest way to do it, but it saves me from recreating the pipeline one-liner I've been using each time I needed to do this up until now.

#!/bin/bash

# mklinks by Shannon Prickett <binder@manjusri.org>
# Makes unique symbolic links from files under source directory in target directory.
# $Id: mklinks,v 1.1.1.1 2004/01/19 19:30:51 binder Exp $

SRCDIR=$1
DESTDIR=$2

FIND=/usr/bin/find
LN=/bin/ln
READLINK=/bin/readlink

if [ -z "${DESTDIR}" ];
then
    echo "   Usage: $0 <source directory> <target directory>"
    echo "   Create unique symbolic link for each file under <source directory> tree in the [flat] <target directory>."
    exit 1;
else
    IFS=$'\t\n'
    echo "Searching ${SRCDIR} for files."
    for targetfile in `${FIND} ${SRCDIR} -type f` ; do
        targetpath=`dirname ${targetfile}`
        targetprefix=`basename ${targetpath}`
        targetname=`basename ${targetfile}`
        if [ -L ${DESTDIR}/${targetprefix}+${targetname} ];
        then
            linkend=`${READLINK} ${DESTDIR}/${targetprefix}+${targetname}`
            echo "Found link to ${linkend}"
        else
            echo "Linking $targetfile to ${DESTDIR}/${targetprefix}+${targetname}"
            ${LN} -s ${targetfile} ${DESTDIR}/${targetprefix}+${targetname}
        fi
    done
    echo "Complete."
fi

But the patient died. This page of game design post mortem comes to me from Doug in response to a query of mine about how I might learn about game design for computers and consoles. Here's an itch I've an urge to scratch. Nothing tangible enough to be on my TODO list, yet, but it's starting up again.

I've had a Sanyo SCP-4900 for a while now. I originally chose that model from Sprint PCS specifically because of a page I found while searching the web for "Linux" and "cell phone". My dream was to have a laptop, running Linux, and using some sort of no-hotspot-needed wireless.

It took me this long to actually get it working because the data cable I needed is seemingly extremely elusive. I couldn't find the original manufacturer or even most of the knock-offs available online. In the end, I found this. It arrived today, so I went back to Nate Carlson's page, excerpted below for people who don't like to click away from the story so far. You can go to his page for the straight poop.

Nate says:

I chose Boxwave because it was actually in stock at the time I ordered, but it was possibly a net win additionally because it is retractable and can work as a charger for the phone, which helps someone like me who forgets to plug the phone in to recharge when not in use. But those factors might not matter to you if you chose to reproduce this.

Nate says:

My plan gives me Unlimited Vision for $15/month. Until I score a laptop, I don't anticipate using this enough for it to matter. So even if my Vision isn't genuinely Unlimited, I don't care enough right now to track it down.

Nate says:

This step worked like a dream. I had built my last kernel with modular support for all kinds of USBisms with the expectation I'd be adding my USB Happy Hacker Keyboard to this system and eventually wanting to test my phone's USBability.

Nate says:

I had to cd to /dev and run ./MAKEDEV usb to get /dev/ttyACM* nodes but that was a no-brainer. My Debian stable chroot shows signs of having had the right devices but not my Debian unstable desktop. No big whoop.

Nate says:

/etc/ppp/peers/sprint:

# You usually need this if there is no PAP authentication
noauth
# The chat script (be sure to edit that file, too!)
connect "/usr/sbin/chat -v -f /etc/chatscripts/sprint"
# Set up routing to go through this PPP link
defaultroute
# Use remote DNS
usepeerdns
# Default modem
/dev/ttyACM0
# Connect at high speed
230400
local
novj

pppconfig added these options for me:
noipdefault
ipparam sprint

and I put in the novj and usepeerdns as suggested in Mr. Carlson's script.

Nate says:

/etc/chatscripts/sprint:
TIMEOUT         5
ABORT           '\nBUSY\r'
ABORT           '\nERROR\r'
ABORT           '\nNO ANSWER\r'
ABORT           '\nNO CARRIER\r'
ABORT           '\nNO DIALTONE\r'
ABORT           '\nRINGING\r\n\r\nRINGING\r'
''              \rAT
TIMEOUT         12
OK              ATD#777
TIMEOUT         22
CONNECT         ""

Mine looks more like this because of pppconfig's output. I ignored the comments about not removing the comments because I don't intend to retune it with pppconfig. Minimalism rules.

# Shannon's chatscript
ABORT BUSY ABORT 'NO CARRIER' ABORT VOICE ABORT 'NO DIALTONE' 
ABORT 'NO DIAL TONE' ABORT 'NO ANSWER' ABORT DELAYED
'' ATZ
TIMEOUT     12
OK-AT-OK "ATDT#777"
TIMEOUTE    22
CONNECT ''

But other than superficial differences, it seems to be about the same. I added in the TIMEOUTs, because pppconfig didn't.

Nate says:

TIMEOUT         5
ABORT           '\nBUSY\r'
ABORT           '\nERROR\r'
ABORT           '\nNO ANSWER\r'
ABORT           '\nNO CARRIER\r'
ABORT           '\nNO DIALTONE\r'
ABORT           '\nRINGING\r\n\r\nRINGING\r'
''              \rAT
TIMEOUT         12
OK      "ATZ"
OK      "ATE0V1"
OK      "AT+IFC=2,2"
OK              ATD#777
TIMEOUT         22
CONNECT         ""

My ping times look like this.

binder@fool:~$ ping -v 68.28.33.136
PING 68.28.33.136 (68.28.33.136): 56 data bytes
64 bytes from 68.28.33.136: icmp_seq=0 ttl=251 time=350.5 ms
64 bytes from 68.28.33.136: icmp_seq=1 ttl=251 time=530.0 ms
64 bytes from 68.28.33.136: icmp_seq=2 ttl=251 time=393.9 ms
64 bytes from 68.28.33.136: icmp_seq=3 ttl=251 time=532.7 ms
64 bytes from 68.28.33.136: icmp_seq=4 ttl=251 time=350.6 ms
64 bytes from 68.28.33.136: icmp_seq=5 ttl=251 time=533.2 ms
64 bytes from 68.28.33.136: icmp_seq=6 ttl=251 time=410.1 ms
64 bytes from 68.28.33.136: icmp_seq=7 ttl=251 time=532.3 ms
64 bytes from 68.28.33.136: icmp_seq=8 ttl=251 time=539.8 ms
64 bytes from 68.28.33.136: icmp_seq=9 ttl=251 time=704.5 ms
64 bytes from 68.28.33.136: icmp_seq=10 ttl=251 time=494.5 ms
64 bytes from 68.28.33.136: icmp_seq=11 ttl=251 time=683.2 ms
64 bytes from 68.28.33.136: icmp_seq=12 ttl=251 time=433.0 ms
64 bytes from 68.28.33.136: icmp_seq=13 ttl=251 time=640.6 ms
64 bytes from 68.28.33.136: icmp_seq=14 ttl=251 time=474.7 ms
64 bytes from 68.28.33.136: icmp_seq=15 ttl=251 time=613.5 ms
64 bytes from 68.28.33.136: icmp_seq=16 ttl=251 time=390.3 ms
64 bytes from 68.28.33.136: icmp_seq=17 ttl=251 time=582.1 ms
64 bytes from 68.28.33.136: icmp_seq=18 ttl=251 time=413.0 ms
64 bytes from 68.28.33.136: icmp_seq=19 ttl=251 time=551.9 ms
64 bytes from 68.28.33.136: icmp_seq=20 ttl=251 time=357.3 ms
64 bytes from 68.28.33.136: icmp_seq=21 ttl=251 time=514.2 ms
64 bytes from 68.28.33.136: icmp_seq=22 ttl=251 time=390.3 ms
64 bytes from 68.28.33.136: icmp_seq=23 ttl=251 time=535.0 ms
64 bytes from 68.28.33.136: icmp_seq=24 ttl=251 time=333.9 ms
64 bytes from 68.28.33.136: icmp_seq=25 ttl=251 time=450.6 ms

So it looks pretty comparable here in Shoreline.

I wrote a stupid bash script to automate some of the things I found myself doing time and again as part of the process of publishing blogs.

#!/bin/bash

# publish by Shannon Prickett (binder@manjusri.org)
# Promote a .tst to a .txt for Blosxom to grab.
# $Id$

declare -a TARGETS

BASEDIR=/scratch/www/blosxom
BLOSXOM=/usr/lib/cgi-bin/blosxom
PASSWORD="suuuuuure"
ARGS="-password=${PASSWORD} -quiet=1 -all=1"
SEDSCRIPT=${BASEDIR}/transforms

BASENAME=/usr/bin/basename
DIRNAME=/usr/bin/dirname
FIND=/usr/bin/find
ISPELL=/usr/bin/ispell
MV=/bin/mv
RM=/bin/rm
SED=/bin/sed

TARGETS=`${FIND} ${BASEDIR} -type f -name \*.tst`

for FULLFILE in ${TARGETS[@]}
    do
        echo "Found target: ${FULLFILE}"
        FILEPATH=`${DIRNAME} ${FULLFILE}`
        FILENAME=`${BASENAME} ${FULLFILE} .tst`
        WORKFILE="${FILEPATH}/${FILENAME}.sed"
        DESTINATION=${FILEPATH}/${FILENAME}.txt
        ${ISPELL} -h -x $FULLFILE && ${MV} ${FULLFILE} ${WORKFILE}
        ${SED} -f ${SEDSCRIPT}  ${WORKFILE} > ${DESTINATION} &&
            ${RM} ${WORKFILE}
    done

echo "Publishing to site."
${BLOSXOM} ${ARGS}
echo "All done."

You'll notice that I took perverse joy in abstracting all my executables to variables and that while I intended to check it in to CVS, I haven't yet done so and that the sed I use doesn't allow inplace editing. I had originally envisioned writing a Makefile which would let me do all this jazz but that turned out to be overkill and I was spending more time guarding against things I didn't want being done than I was writing things that I did want done, so bash saved some of my hair.

Speaking of sed, here's a subset of that transforms file I use to allow me to mark up my blog entries with lazy tags instead of typing the freaking URL every time for things I often reference. If I were very smart, I'd have some way to automatically extend this. It was, however, The Simplest Thing Which Could Possibly Work.

[transforms excerpt removed until I extend publish to entity-tize my html on request so that I can actually show what it does]

I fully expect to add more transforms and perhaps even more exciting sed operations, especially if Crag does the heavy lifting for me by doing his proposed STL-engine-in-sed.