Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

cbrooks (3267)

cbrooks
  (email not shown publicly)

Journal of cbrooks (3267)

Thursday June 09, 2005
08:22 AM

Shameless plug: podcast any RSS feed

I thought I'd plug my new project on use.perl as some of you might find it interesting.

Talkr is a service that allows you to create a podcast of any RSS feed.

If you point Talkr to an RSS feed (your own or someone else's), Talkr will give you an RSS feed that points to an mp3 file for each article. You can configure your computer to download those audio files as they become available and drop them directly onto your iPod (or other mp3 player).

Here are links to a couple of BoingBoing articles if you want to hear what it sounds like:

Talkr runs on mod_perl, of course. If you are interested in trying Talkr out with your own blog, you can sign up here.

<full_disclosure>
    Talkr is not affiliated with BoingBoing in any way. They just write about interesting stuff.
</full_disclosure>

Friday February 21, 2003
09:55 AM

Consumer VOIP

We just signed up for Vonage's consumer voice over ip service, and I thought I'd give it a quick review

The good:
  • $39.99 a month, flat fee, unlimited usage anywhere in the US and Canada. That includes call-waiting, voice mail, caller id, etc. They have a $25 plan as well, which has some limitations on the number of minutes you can use.
  • Configuration was simply plug and play. The only problem that I had was that I left the ringer off on the phone that I initially used -- so I mistakenly thought that I was not receiving incoming calls.

The bad:

  • You need broadband internet access. A minimum of 90 kbps up and down is required.
  • Although they can let you keep your existing phone number in some cities, we couldn't keep ours. In fact, although we were able to keep our area code, we couldn't even keep our local exchange. (Their customer support folks claim that this will change in the next few months.)
  • You have to dial all 10 digits to dial any number (even local calls).
  • They don't support 911.
  • I occassionally notice a slight echo of my own voice, and (occassionally) you can't hear sounds from the other end of the line while you are speaking. (I have 1.4 mbps down / 300 kbps up.)
  • I have to buy a new wireless expansion phone, because my LynkSys switch is on the 3rd floor, and we want a phone on the 1st floor.

That's a substantial number of drawbacks, but it's still worth it to me, considering that our phone bill was $170 last month.

Wednesday February 12, 2003
08:10 AM

Security bug in CGI::Lite::escape_dangerous_chars()

It doesn't look like anyone else has posted this to use.perl yet. A security flaw in CGI::Lite was posted to bugtraq yesterday. Essentially, the escape_dangerous_chars() method fails to escape a number of metacharacters.

The impact statement says:

If the CGI::Lite::escape_dangerous_chars() function is used within (for example) a web CGI script, a remote attacker may be able to read and/or write local files on the attacked web server and/or may be able to gain shell-level access to the attacked web server, via the CGI script, as the user-id under which the CGI script is executed (typically, but not always the `nobody' user).

The potential exists for remote root compromise (or other privileged access) if a CGI script using CGI::Lite::escape_dangerous_chars() is installed as set-uid (root) or set-gid.

As noted by the white hat who found the flaw, CGI::Lite's maintainer has not responded with a patch (and the lastest version of the module available on CPAN is from 8-20-2000).

Thursday January 23, 2003
12:06 AM

Cross-Site Tracing (XST) security vulnerability

Ilya has already mentioned this, but he has comments disabled, so I thought I'd make an entry as well.

XST is a newly-reported vulnerability that combines traditional XSS (cross-site-scripting) attacks with HTTP TRACE. The traditional XSS part of the attack is that a blackhat could embed a malicious JavaScript (or other client-side scripting language) function in a web application that does not properly screen client input. The JavaScript function in turn can craft an HTTP TRACE request (using the XMLHTTP ActiveX control in IE or XMLDOM in Mozilla). HTTP TRACE simply echoes your HTTP request, including HTTP Basic Authentication strings (which are passed as base64-encoded cleartext), which apparently makes this new attack noteworthy. XSS attacks could not previously access Authentication strings.

The tone of commentary on Bugtraq generally seems to downplay the severity of this vulnerability for two reasons: (1) very few people have ever seen XSS attacks in the wild; and (2) the vulnerability would be largely mitigated were Microsoft to simply remove XMLHTTP's ability to send an HTTP TRACE request.

If you use HTTP Basic Authentication on your server, you can protect against this method of reading authentication strings by disabling TRACE requests. Disabling TRACE on Apache is trivial, assuming that you have compiled Apache with mod_rewrite:

##########################
#
#This rule is in response to http://www.cgisecurity.com/whitehat-mirror/WH-WhitePaper_XST_ebook.pdf
#
RewriteEngine On
#Test the server variable REQUEST_METHOD to see if it matches the pattern ^TRACE
RewriteCond %{REQUEST_METHOD} ^TRACE
#
#Match 0 or more of any character, No substitution, return FORBIDDEN (403)
RewriteRule .* . [F]
#
#########################

(Comments are mine, mod_rewrite directives are from the white paper mentioned above.)
Wednesday January 22, 2003
08:10 AM

The future of computer programming in the US of A

Scott Kirsner is the author of @Large, a weekly tech column for the Boston Globe. In this week's column he suggests that a substantial portion of the technical and programming jobs currently performed in the United States will be outsourced to India and Russia over the next 15 - 20 years. This would leave mostly management and project management positions here in the states. Good news for those English-speaking progammers abroad, not so good for those of us here....

This cheery news brings to mind a long-standing question that I have: Most of the perl programmers that I know work for someone else. A few work for themselves as consultants / trainers, but I don't know anyone who uses Perl to run a non-technical business for themselves.* I mean, do any techies (for example) start bike shops and then use Perl to automate everything and allow order fulfillment over the web? I have the impression that techies tend not to be entrepreneurs -- is that impression correct? If yes, why? Are the talent sets required for entrepreneurs simply at odds with technical talent sets?

* Hmmm. Actually, that's not quite right. I met a guy at YAPC this year whose primary source of income came from a running a skate-boarding website.
Thursday January 16, 2003
05:09 PM

Build and Release System, version 0.2.0

As mentioned in my previous journal entry, I am developing a build and release system for use at work. I had two goals for the first stage: (1) to develop and document a version numbering system; and (2) to publish the current stable version in the script that we use to report tech stats. This journal entry relates some of the issues that I ran into while implementing this first step.

Motivation
Before diving into the details, you might well ask why developing a version numbering system is an interesting enough (or difficult enough) task to justify more than 10 minutes of thought (let alone a journal entry). Why not simply place the site under CVS, and track versions using CVS version numbers? In fact what does it mean to have a single number that represents a website's "version" at all?

The most useful analogy is probably the concept of a Linux distribution. If you think about RedHat Linux version 7.3, for example, it contains thousands of files. Although each of those files maintains an internal version, the end user (the sysadmin) one rarely cares about the version of the individual files. (They certainly care if their OS has a journaling filesystem, and they care if it incorporates a version of ssh that has the latest patches applied, but who wants to be bothered with tracking the individual versions of the thousands of files that live underneath /?). RedHat can talk about "Sunsetting version 7.0", managers can talk about upgrading to version 8 when it stabilizes: the distribution number is an important shorthand to summarize the versions of thousands of individual files. Aside from a means to summarize the state of many files, a version number can provide hints or clues to its users. Many programmers, for example, discourage production use of their code until it achieves a 1.0 release. (For example, they may not guarantee that an API will be stable before 1.0.) Furthermore, Perl (and many other open source projects), declares that odd-numbered minor releases are development versions, while even-numbered minor releases are production-worthy. Finally, a number that summarizes the overall state of a website's code can link to a CVS tag so that you can automate the checkout of a particular version of each of your CVS projects. All in all, applying a thoughtful and consistent version numbering system allows you to summarize a great deal of information in a very concise manner.

Design
So, enough introduction, what issues did I run into during design? The first issue was simply to choose a set of rules for how and when a version could increment. The version is the concatenation of three integers: one representing the major version, one the minor version, and one the release. The version string is generated by concatenating major.minor.release. The initial version, for example, is 0.0.0. The major version number is incremented to indicate the release of major new features and enhancements. The minor version number is incremented to indicate minor features and enhancements, and the release is incremented to indicate bug fixes. Major and release version numbers begin at 0 and increment by 1. The minor version level begins at 0, but an even minor version indicates a stable production release, while an odd minor version indicates a development release. So, it is perfectly reasonable to say "I am developing a feature for 2.3", and anyone hearing that statement would understand that the feature is intended for the production version 2.4. Each version number can increment past 9 (for example, 2.6.22 is a valid version).

The second issue was whether to incorporate the concept of a build. Builds are typically used with compiled languages since compiling the source code into a binary can take a non-trivial amount of time. Since we do not distribute our perl code in binary form, the build process is not strictly necessary. However, there are a number of other concepts associated with a "build" that make its inclusion quite handy. For example, smoke tests ensure that the codebase as a whole continues to compile as developers check in new code. This integrates nicely with the goal of running all of the unit and regression tests on a nightly basis. Another goal for this system is to package the entire codebase into a single file on a nightly basis, for ease of compression, distribution and release. So some of the original meaning of the "build" is retained. All in all, recording the status of the nightly build seems like a good idea.

So, if the build is recorded in addition to the version, how are the two numbers related? Although I am not entirely settled on this issue, I have decided that the build number will be closely tied to the release version. That is, every time the release version increments (say from 1.4.5 to 1.4.6), the build will re-set to 0. Along with recording the build number, the nightly build will also record whether or not the smoke tests were run, and whether or not they all passed.

Implementation
The final issue was simply defining the interface to set and retrieve the version. I decided on a simple OO class with a few straight-forward methods:

my $ver = new Company::Version;

$ver->set_version( '2.0.4' ); #returns old version
$ver->increment_build #returns old build

print $ver->get_version; #prints 2.0.4
print $ver->get_major; #prints 2
print $ver->get_minor; #prints 0
print $ver->get_release; #prints 4
print $ver->get_build; #prints, for example, 43.
print $ver->get_qa_tested; #prints 'Y' || 'N'
print $ver->get_qa_passed; #prints 'Y' || 'N'
print $ver->get_increment_date( '1.2.0' ); #prints 10/15/2001, for example

The class is simply a wrapper around a database table. Each build (and its associated version numbers) is a row in that table. The get_version() method simply returns the most recent row that has an even minor release version. This means that rollbacks will require deleting the latest row, but it makes for a straightforward implementation, and doesn't introduce any awkward corner cases. There are other methods that will need to be implemented at some point, for example, to connect a particular version to a CVS tag, to delete versions, to increment the major, minor and release versions, to rollback to a previous version, etc. However, this was enough functionality to consider the first iteration of the Version class complete.

Anyway, if you made it this far, thanks for reading. Version 0.4.0 integrates nightly smoke testing. I would appreciate any feedback.
Wednesday January 08, 2003
03:19 PM

Implementing a Build and Release System

For the better part of the last year, I have intended to implement a build and release system for the company at which I work. I wanted to do this project the "right" way -- i.e. specify the functionality; design the modules, define the interface to CVS, etc. However, I have finally accepted that I will never start this project (much less finish it) if I build it this way.

So, In the middle of December, I decided to take a different approach. I decided that the only way I would actually finish the project was to break it into small, incremental steps. I wouldn't have to understand exactly how the entire system works before beginning -- it is enough to build the pieces that I do understand, and gradually fill in the gaps.

The project will be successfully completed when the following two criteria are met:
  1. I can copy a stable version of our code base from our staging server to our production server and release it without an interruption in service.
  2. I can roll-back to the previous stable version without an interruption in service.

I then defined the following intermediate versions:

  1. Version 0.0.0: Current codebase
  2. Version 0.2.0: Versioning (01/01/03)
    • I would establish and document a version and build numbering system.
    • I would begin logging version changes
    • I would report the current production version in our tech_stats.pl script, which is the central dashboard for stats for the Tech Department.
  3. Version 0.4.0: Testing (01/15/03)
    • Run nightly smoke tests
    • Ensure that all tests pass
    • I would report the previous night's testing status in the tech_stats.pl script
  4. Version 0.6.0: 95% sync-up between production and staging (02/15/03)
    • Sync all CPAN modules on staging and production server
    • Make sure all application files are the same on the staging server and the production server
    • Make sure there are no files that exist on the production server but not the staging server
  5. Version 1.0.0: First automated stable release (03/31/03)
    • Integrate w/CVS tags using trunk and branches methodology
    • Remove deprecated functionality and database tables

I am happy to report that I released version 0.2.0 on 01/03/03, and am proceeding on version 0.4.0.

Wednesday December 18, 2002
09:43 AM

Bill Joy and the Apocalypse

I have been following Bill Joy's comments on a coming biotech / nanotech / robotics apocolypse for the last year and a half -or so. His premise is that the potential for extreme violence on a global scale is extremely high, due to the democratization of destructive power that these three technologies offer.

I have to say, first, that I find his comments disquieting, at a minimum. Essentially, he believes that the only realistic option open to us is to choose not to develop these technologies. What I find interesting, though, is the similarity between these arguments and an older apocalyptic argument presented in The Limits to Growth (1972) and Beyond the Limits (1992). These books draw an analogy between over-population-induced die-offs in other animals and (so they claim) a soon to arrive massive die-off in the human population caused by ever-expanding resource consumption. The weakness in the Limits argument, I think, is that they inadequately model the influence of technology, and ironically, I think Bill Joy makes the same mistake. He (reasonably) points out the immense dangers that these technologies make possible, without pointing out that they also make possible immensely more powerful defenses.

So, for example, when he talks about the possibility of very nasty terrorist-engineered viruses which could hone in on weaknesses in the human genome, he doesn't talk about the possibility of white-hat engineered monitors that could scan for, and counter, these viruses in real time. (Norton Anti-Virus personal edition 2025 might have a very different meaning.)

I suppose I have to hope that those sorts of developments will happen, because I firmly believe that widespread rejection of these new technologies has a vanishingly small probability of occurring. We do not live under the kinds of economic and political systems that would make that a feasible solution to the dystopia that Joy envisions.
Thursday December 12, 2002
09:12 AM

Using XWindows remotely

As part of my new eternal commute, I have started working a bit from home -- an hour here, an hour there. One of the annoying aspects of this is managing email. If I check company email from home, I either see the same messages twice (once from home and once at work), or I can choose to have the messages deleted from the server as soon as I delete / move them at home. Neither approach is ideal.

My new approach is to have a single mail client located at work. I fire up Cygwin on my Windows box at home, invoke ssh, and interact with Mozilla remotely using X Windows. (Actually, I'm fibbing a little. So far, I have a Linux box at work running my mail client, and I'm just shelling in from my Windows / Cygwin box across the office.) But, it's a neat trick, and it will greatly simplify any future work from home.

Here are the steps that I took, in case anyone is interested in trying something similar.
  1. Get Linux configured on the box that you are going to run your mail client on. (You could also run Cygwin on top of Windows, I suppose.) Make sure that you can use X on this box, and make sure that you have the latest openssh daemon configured and listening to port 22. I suppose you should also make sure that you can access this box remotely over ssh.
  2. Install Cygwin on the Windows box that you will use remotely. (Cygwin, BTW, is free as in speech, and free as in beer for most purposes.) The Cygwin installation process is quite straightforward, once you get used to its somewhat non-intuitive interface. You can download the setup program from www.cygwin.com. I chose to install the following packages (which take up about 250 MB of disk space):
    • All: Default
    • Doc:Man and Texinfo
    • Editors: ed, mc and vim (because vi is the one true editor, okay, the two true editors, because it uses ed under the hood)
    • Net: Openssh and rsync (rsync isn't necessary for this project)
    • Text: more, less
    • XFree86: pretty much everything. Make sure you have fvwm and base
  3. Follow the instructions at www.shadlen.org. I found these to be pretty good, although the startwin.sh script mentioned was actually in /usr/X11R6/bin/startxwin.sh, rather than /usr/X22T6/bin/startwin.sh.
  4. At this point, I was able to shell into the Linux box that would host my mail client. However, when I tried to invoke a utility (such as xclock) that relied on X, I got the following error: "authentication failed - cannot start X server - perhaps you do not have console ownership". I was able to fix that thanks to the following usenet post: http://archives.neohapsis.com/archives/pam-list/2000-10/0064.html.
  5. I made a minor change to my ~/.ssh/config file (on the remote Cygwin box), setting "ForwardX11 yes". (You could also just pass the "-X" option to ssh at the command line.)
  6. From cygwin, I fire up an xterm, and invoke ssh with the following options: "ssh -n ip_addr mozilla", and lo and behold, Mozilla pops up.

I have not been able to launch startx successfully (remotely), though I can launch it on the local box directly. I think that my ssh client is configured correctly, which makes me suspect that the problem has something to do with inadequate libraries on the Cygwin box. (For example, if I shell from the local Linux box back to itself, I can run startx successfully.) I believe this suggests that if the remote box had the correct software, the X11-forwarding would work correctly. Does anyone have any brainstorms?

Friday December 06, 2002
09:24 AM

The Future of the Operating System

I just moved to NH and earned myself an excruciatingly long commute to work (approx 3 hours a day). The upside is that I have a lot of time to listen to books on tape and, most recently, MP3s. If you are in a similar situation, check out technetcast.com -- the site is presented by Dr. Dobb's Journal, and it has lots of (free) technical conference presentations available on MP3.

Yesterday, I listened to Richard Rashid, VP of Research at Microsoft give a very interesting presentation on the future of the OS. He argues that the essential concepts of the modern OS: hierarchical file systems and user-initiated execution of programs are artifacts of the computers that we had 30 years ago. Computers today are some 40 million times more powerful, and storage capacity is approaching the point that a person could record and store every conversation that they have from birth to death on a single hard drive. Rashid envisions a shift towards interacting with your OS primarily through google-style query mechanisms, rather than a filesystem. The OS would be smart enough to initiate programs on its own, rather than waiting for user input. Obviously, OSes would have a host of new (or at least dramatically improved) input devices, such as speech recognition and handwriting recognition. Perhaps more dramatically, broad spectrum language parsers would have an increasing role in the background to prioritize and categorize documents, email, spoken conversations, etc., and then react without an explicit request from the user.

Anyway, it is a thoughtful talk -- certainly worth a listen.