Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Matts (1087)

Matts
  (email not shown publicly)
I work for MessageLabs [messagelabs.com] in Toronto, ON, Canada. I write spam filters, MTA software, high performance network software, string matching algorithms, and other cool stuff mostly in Perl and C.

Journal of Matts (1087)

Friday June 07, 2002
10:40 AM

Net::Server nightmares

[ #5482 ]

Writing network daemon's is hard.

That's why I prefer not to do it. So for PPerl, I decided to have a go with Net::Server (after having originally written the daemon code using the stuff in the perl cookbook, and failed miserably). It worked really well, and despite being a little slower than my own custom daemon, it seemed really reliable and stable.

That was until I tried it on a real live system. I ended up with defunct processes left right and center, and finally the whole system just locked right up. Bah.

So what could I do? Well, I decided to bite the bullet and re-write it. So I grabbed code from Lincoln Stein's Network Programming with Perl book - the pre-fork web server in there, and adapted it to my needs. After much grappling it seems to be mostly working, though there are some extremely wierd things going on:

- My changing of STDERR to go to a file isn't working right for warn()'s unless I add in a $SIG{__WARN__} handler to print directly to STDERR.
- Occasionally it'll die with a "Connection reset by peer".
- Sometimes the <ARGV> magic fails for unusual reasons.

I guess I need to copy a bit more of the code from Net::Server to see where I'm going wrong (because now I don't have defunct processes any more).

Anyone got good experience with this kind of thing, or an alternate framework to Net::Server?


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Net::Server nightmares 13 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
    • Always hit preview ;)
    • Asynchronous is inappropriate for PPerl - otherwise I would have used POE, and saved myself all these headaches.

      Plus much as Stem looks neat, it's not CPAN installable.
      • you can install it by running "perl install.pl" as root.

        Asynchronous isn't an issue. You can have a synchronous server with asynchronous code internally. Stem comes with a module, Stem::SockMsg that implements a socket-based listener. You use that and it feeds a message into your real server, which then replies. All the synchronous bits are managed by the Stem::SockMsg module.

        Having written such a thing (to interface to a client that is _not_ Stem-based) I can tell you its pretty darn easy to do.
        • PPerl needs to fork. It just can't work any other way - yes I guess the socket itself could be select based and fork after the connection and multiplex in the results, but that would be slower than a prefork model.
          • You can do prefork with Stem too.

            But you still need a single process listening on the socket that hands off the connection to the preforked processes, right?
            • No, you create your socket (binding it to the port), fork as many children as you hope to need, then call accept in each of the children. The OS decides which process to hand the connection to.
              • stem has a neat module Stem::WorkQueue that makes a prefork server easier. you can prefork a bunch of processes and send all the incoming requests to the queue which first-come/first-serves them to the process farm. or you can look at the existing inetd demo which is pretty much what you want and it requires no new coding. it forks on demand but unless you need to fork massively often it should be fine. if you must have a preforking server, it should be easy to code up with the process and SockMsg modules a
                • My major concern with using either Stem or POE with PPerl is that for PPerl I need access to the raw socket so that I can bind STDIN/ARGV and STDOUT directly to the socket. While I know I can do that with POE, it kind of circumvents the way POE is supposed to work. I'm not sure about Stem.
  • You must have skipped my journal entry [perl.org] about Net::Server. I never had any luck with it either. :(

    • I use Net::Server as an option in my RPC::XML package, an alternative to using the accept-loop in HTTP::Daemon. I also have a set of tests in the t/ directory for it, in which I manipulate the settings at run-time to control the log file and such. I haven't had any problems, but to be fair I probably have run anything yet that's on the scale of PPerl.

      --

      --rjray

  • At ValueClick we wrote a daemon that started out as "nonforker" from the Perl Cookbook. It has been used for many many billion messages since then without any problems. If your protocol is reasonably linebased and the system can work as a non forking daemon, then maybe I can release the code. (It's from my pre-ValueClick time and we just extended and optimized it (heavily) there).
    --

    -- ask bjoern hansen [askbjoernhansen.com], !try; do();

    • PPerl isn't line based - it passed the STDIN/STDOUT over the socket byte by byte. And it needs to fork (otherwise there's very little point - I would do it in POE if I didn't need it to work like this).

      On the other hand, I've just "solved" or hacked around the ARGV magic stuff. Not sure why what I did worked, but that's what you get with wierd stuff like I'm trying to pull off.
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.