But what if nothing returns on the command line? The parent dies, killing any child that may be running. This is bad. This is very bad.
When I set up my Net::SCP::Expect module, I set up a 1 second (default) timeout to check for any error messages. I also used a 'soft_close()' followed by a 'hard_close()' in the event that the former failed. The significance of these things will be made clear shortly.
In testing, I never tried copying any particularly large files and so I never had any problems. However, I recently received an email by someone using my module that he was having trouble copying large files. It seemed that he was only getting partial copies. So, I tried it myself - he's right. When I tried copying a rather large log file over, only a portion of it was copied before the program exited.
The next step was to just use the Expect module on its own. Same problem - the program dies before the whole file is copied over. What do I discover? That the only thing keeping the parent going at all is the timeout I used for error checking and the lag caused by using the 'soft_close()' method (which always seems to slowly fail). Once I removed those, even *less* of the test file was copied over. However, this small timeout plus the lag was always enough time to copy the small files I used in testing, so that's why I never noticed until now.
So far as I can tell, if Expect doesn't get anything back on the command line (and it shouldn't since I'm running scp in quiet mode), the parent exits and kills and child process that may be running. The only workaround is to set a high 'timeout_err' value in my module, so that it has enough time to copy before the program exits.
Ouch. Big fucking ouch. An email from Roland Giersig wasn't of much help. So, what do I do now in the event there's no way to deal with this? Do I rip out the Expect module completely and replace it with one of the Term modules? I may very well have to.
UPDATE: jdavidb was kind enough to provide a fix. Hooray! Thanks jdavidb!
Thoughts (Score:2)
I think this behavior is similar to what you would have had under TCL/Expect. Not that that's a good reason. :)
Can't you expect eof? Under TCL/Expect, if you wanted to run a command that produced no output, you would do an expect eof with a long (or -1) timeout to wait until it ended.
Are you using the scp command-line program? If so, doesn't it produce a progress bar that would give you some output to expect()? At first I didn't understand why you were using Expect, but then I read the docs and I se
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Re:Thoughts (Score:2)
Ooooh - good idea. I'll give it a shot.
doesn't it produce a progress bar that would give you some output to expect()?
Not in quiet mode. Normally you would get a '#' sign, but I wasn't sure how portable, or reliable, it was to check for '#'.
Still don't understand why you have to do soft_close() and hard_close().
I don't have t
Re:Thoughts (Score:1)
Still don't understand why you have to do soft_close() and hard_close().
I don't have to. The docs indicate that soft_close() was the polite way to close and hard_close() was the impolite way so my implementation was arbitrary based on that.
No, I mean I don't understand why you call those routines at all. Usually you would only have to call them if you were ending the command early. Once you get to the point of eof either close is redundant, I think.
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Re:Thoughts (Score:2)
Re:Thoughts (Score:2)
Okay, this is goofy, but it appears you can't just
$exp->expect($timeout, 'eof')
Otherwise, it tries to match the literal string 'eof' (hence the -ex in your verbose output). You actually have to use the arrayref syntax to expect(). The arrayref syntax is
$exp->expect($timeout, [pattern => code], [pattern => code], ...)
So you can do
$exp->expect($timeout, ['eof' => sub { exit }])
I tried leaving out the => sub { exit } and it didn't seem to work, although I seem to have w
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Re:Thoughts (Score:2)
Ah! Found out that I only need the => sub { } on special things like eof because they use special code that didn't take the possibility of not specifying a coderef into account.
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Re:Thoughts (Score:2)
The fundamental flaw with Expect.pm (and perhaps expect in general?) is that it kills its own children before letting them finish. Perhaps there was a reason for this, or perhaps they never counted on anyone performing an op that didn't re
Re:Thoughts (Score:2)
You can specify an unlimited timeout. It's hidden in the docs somewhere. Use undef instead of a value. In the original TCL/Expect, it was -1. (Had trouble finding it in the docs back then, too, as some of my old programs attest.)
Here's my code to scp a file from my machine to its own /tmp directory:
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Fix (Score:2)
Boy, you go to great lengths to get someone to install your module. :)
I installed Net::SCP::Expect and tinkered with it a bit. I was able to reliably duplicate the problem by attempting to scp the largest file on my machine to my /tmp directory. I made the following changes, and it seemed to fix the problem:
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Re:Fix (Score:2)
Thank you very much. Should I be irritated that I ultimately received the answer from you and not Roland? Hmm...nevermind. I may end up in the same boat some day. That, or he thought I was an annoying nit and
Re:Fix (Score:2)
Should I be irritated that I ultimately received the answer from you and not Roland?
Speaking as an active member of the expectperl mailing list, Roland sees a lot of FAQs asked many, many times. We also have a lot of people who are trying to learn Expect.pm before they learn Perl! (It doesn't help that some have never seen TCL/Expect, either.) It seems like every week there's someone trying to automate ssh in a way that indicates they would do better to try a different method than Expect (such as sys
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Oh? (Score:1)
Re:Oh? (Score:2)
Perhaps a couple of FAQ entries would be in order. I'll draft and send to the mailing list.
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Re:Oh? (Score:1)
Thanks, I'll put them in and also take a look at the 'eof' issue.
TMTOWTDI (Score:2)
Austin Shutz (the original author of Expect.pm, I think) offers this other way to do it:
$exp->expect(undef);This will basically just wait until the command is done; expect() will return when it gets the eof. If you really needed to explicitly see the eof at that point, you could check the error string:
if ($exp->error() =~J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers