Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

This Week on perl5-porters - 18-25 January 2009

posted by grinder on 2009.02.01 9:19   Printer-friendly

"Are these names considered to be a public non-changeable interface by virtue of having been placed in these directories for a while, even though they were never documented as far as I can tell? Or are these files considered to be an internal implementation detail of Perl, not for external use, and hence subject to change or deletion?" -- Karl Williamson, waging war on two fronts simultaneously, Perl and Unicode.

Topics of Interest

Interesting git-based changelog workflow

Aristotle Pagaltzis mentioned a workflow he had heard about regarding git. The aim was to ease the burden of producing the Changelog and perldelta files, which are huge timesinks as it involves sifting through thousands of commit messages to look for the big-ticket items. The solution is to create a more incremental effort.

Dave Mitchell said he hoped to produce a one line summary for perldelta for each day of reviewing and merging patches from blead to maint.

A number of porters highlighted the difficulty of being able to know precisely when a bug was fixed, since smokers of CPAN on blead tended to uncover problems a long time after the "final" patch had landed. David Golden suggested that a add-perldelta tool might be a step in the right direction to keep track of the changes at an appropriately high meta-level of changes.

  navigating the delta
  http://xrl.us/beeifz 

disabling suidperl in Configure

Nicholas Clark put forward a patch to prevent suidperl being configured in 5.11 and beyond and wondered if there was a better approach. Andy Dougherty suggested another way, but it had the disadvantage of increasing the workload for H.Merijn Brand, the Configure pumpking, and would also not produce a useful error message if someone attempted to try and build suidperl in 5.12 anyway.

As it was, H.Merijn applied Nicholas's patch.

  it's dead jim
  http://xrl.us/beeif3 

Later on, Nicholas axed it altogether, reasoning that if there was some brave maintainer who wanted to resurrect it, all they had to do was to reverse out one change from their local git repository.


  let sleeping dogs
  http://xrl.us/beeif5 

mingw libperl broken

Reini Urban reported a discrepancy between mingw and cygwin as to what public functions were exported to the world at large. Nicholas Clark thought that mingw (not exporting three particular functions) was correct, and it was cygwin that was being lax. And further, modules outside the core had no business in defining PERL_CORE in order to call functions to which it had no right.

This problem is not limited to the Windows platform. AIX also has a strict policy on what is and isn't exported.

Dmitry Karasik noted that the problem had been found a long time ago and a patch had been produced, and wondered why it had never been applied. This allowed Nicholas Clark to recycle his standard remark about volunteer workers and companies using Perl not making resources or money forward in order to improve the situation.

  plus ça change
  http://xrl.us/beeif7 

Reini revived cpan bug #28912 to fix B::Generate and deal with its breakage. He said that three functions, Perl_pad_alloc, Perl_cv_clone and Perl_fold_constants are exported anyway, and so should be published in the API as such. Nicholas Clark was a lot more reticent, saying that while it may indeed be useful for them to be exported, were they documented, testable, stable and supportable.

  a gentleman's agreement
  http://xrl.us/beeif9 

LSB testing issue with Test::Harness

Stew Benedict reported a problem with the LSB project's handling of perl's test infrastructure. They currently use Test::Harness version N-1, and run into problems when that is upgraded to the current version N.

Yves Orton thought that the best way forward was to deal with the pain now and upgrade to the current version, even if there were backward compatible shims available for those who wished to keep working with N-1.

  duty now for the future
  http://xrl.us/beeigb 

Later on, Stew gave a status report on the second round of Perl modules included in the LSB. He made a list of modules that had been pulled out of their core distribution, a list of modules that might yet be pulled, and a list of those modules that remain, along with their counts as pre-requisites. Stew explained that it was easier to pull things out now, possibly to put them in later, rather than the other way around. Thus, it would pay to be as aggressive as possible.

Yves Orton thought that removing parts of the testing infrastructure was definitely a bad idea. Nicholas Clark pointed out that while CGI has been in core for a long time, FreeBSD removed it in a recent release, apparently without much ill effect. He also responded with a large list of modules that have achieved ubiquity (and thus must be present) as well as other modules that also must be present even though we all wish they would curl up in a corner and die.

  freebsd decoring++
  http://xrl.us/beeigd 

Need advice on gotchas on upgrading unicode db to 5.1

Karl Williamson is currently working on bring the Unicode database in Perl up to date with the latest and greatest, version 5.1.

The first problem he encountered was that the previous update didn't upgrade the Property Value Aliases, which is a hassle, because upgrading it now may introduce curious backward compatible bugs.

H.Merijn Brand thought that the problems Karl was raising were sufficiently far down in the basement that the general public shouldn't be aware of them, let alone rely on them, and that anything that allowed Perl to track Unicode with a minimum of intermediate layers and mappings was a worthy goal. He also put forward his idea of the list of files in the distribution that the general public was likely to be aware of, and using.

Karl went ahead after receiving additional clearance from Nicholas, and CYAed himself with a "these files are subject to change without notice" header.

  "just change it" works for me
  http://xrl.us/beeigf 

As part of this work, Karl added the Unicode file NameAliases.txt, which fixes up some of the problems that have arisen over time.

  http://xrl.us/beeigh 

use bytes and regular expressions

Karl is also continuing to wade through the crocodile-infested swamp of bytes, characters and regular expressions and was wondering what exactly should be happening when a regular expression was being compiled when use bytes was effect. The main problem being a complete lack of guidance insofar as the test suite makes no attempt to specify what should be happening one way or another.

  http://xrl.us/beeigj 

glob() on VMS totally different to Unix

John Malmberg was stunned to learn that glob('foo') on VMS is the equivalent to glob('foo.*'), as it can lead to some pretty nasty cross-platform side effects. And even more curiously, there was explicit code to make things behave differently on VMS if 'foo' were in fact a directory. He wondered what the appropriate fix would be. The goal was to get Module::Build working correctly on VMS.

Craig Berry wasn't too sure either, since the observed glob behaviour had been like that since time immemorial, and it also happened to reflect the native behaviour in VMS anyway.

John countered that in actual fact it was the utilities that were doing the expansion, not the shell.

Peter Prymmer voiced the opinion that things are working according to spec already, and pointed out that unlink only removes the current version of the file (since the RMS filesystem on VMS allows multiple versions of the same file to exist simultaneously -- a concept quite alien to Unix where Perl was initially developed).

Peter also thought that Module::Build really ought to be doing the opendir/readdir/closedir dance, since differences in glob behaviour had been observed in the past among different Unices.

After some more work, John produced a final patch that cleared up the remaining Module::Build test errors. He then spotted a different error, that Craig Berry identified as being fallout from the Y2038+ code that entered blead recently. Craig promised to get it sorted out if no-one else wanted to.

  it's a bird... it's a plane... it's a directory!
  http://xrl.us/beeigm 

John forwarded some patches to start clearing this up.

  http://xrl.us/beeigo
  http://xrl.us/beeigy 

lock() on a code ref?

Jerry D. Hedden wondered why we check for code references when locking scalars, since it is meaningless to do so. He wanted to remove it since he felt it was superfluous. Dave Mitchell suggested putting an assertion in its place.

  next up, signed coderefs
  http://xrl.us/beeigq 

And thus it came to pass.

  let my code free
  http://xrl.us/beeigs 

A plan for ext/

Nicholas Clark would like to clean up ext/ by hoisting anything that pertains to the build process per se, such as make_ext.pl so that all that is left are the extensions themselves.

He also wants to flatten the directory structure so that all module build directories can be globbed with ext/*.

The Windows and VMS platforms in particular need some refactoring done so that everything can be driven off a generic make_ext build process. Some modules will need to have a Makefile.PL synthesised as well. Another problem to address is the dependency chain: ensuring a modules's prerequisites are built before it is.

Craig A. Berry remembered his classes and noted that ext/ used to represent modules that contained XS components, and lib/ those that didn't. While he didn't have a problem with Nicholas's idea, he pointed out it would be a good idea to look for cruft in %Config and installperl that such a change would create.

And then dual-life modules can migrate from ./lib to ./ext.

Stay tuned next week.

  sounds like a... oh wait
  http://xrl.us/beeigu 

A plan for pod/

Nicholas also had some good ideas about POD. For instance, pod/perlapi.pod, pod/perlintern.pod and pod/perltoc.pod are currently maintained under revision control, but could just as easily be generated dynamically by ./miniperl during the build. Similarly, the man pages could be built before the install phase, thus avoiding another step that currently performed in a task run with super-user privileges.

One benefit is that it reduces the amount of data shipped in an perl tarball, and helps reduce the speed at which the generated documentation drifts out of date.

  generate the pod bay doors please
  http://xrl.us/beeigw 


TODO of the week

Make Config.pm cope with differences between built and installed perl

Quite often vendors ship a perl binary compiled with their (pay-for) compilers. People install a free compiler, such as gcc. To work out how to build extensions, Perl interrogates %Config, so in this situation %Config describes compilers that aren't there, and extension building fails. This forces people into choosing between re-compiling perl themselves using the compiler they have, or only using modules that the vendor ships.

It would be good to find a way teach Config.pm about the installation setup, possibly involving probing at install time or later, so that the %Config in a binary distribution better describes the installed machine, when the installed machine differs from the build machine in some significant way.


Patches of Interest

Avoid confusing gcc with -2147483648

Robin Barker wanted to play by gcc's rules in relation to LOCALTIME_MIN but there appears to be problems with having things work correctly with C89-compliant compilers.

  time is not on my side
  http://xrl.us/beeig2 

Run make_patchnum.pl before configpm

Jerry D. Hedden wanted to ensure that ib/Config_git.pl was available before configpm was right, but this turned out to be surprisingly difficult in the light of parallel make environments and stamping out the possibility of race conditions that could arise therein.

  first things first
  http://xrl.us/beeig4 

Fix mktables time stamp test

Karl Williamson tried to cut through sedimentary layers of code checks designed to prevent mktables from being run repeatedly during the build process. Just when he thought he had things solved he ran into another problem. Rafaël suggested that the remaining problem was also some sort of bogus optimisation and so maybe it too should be brute forced.

  one thing at a time
  http://xrl.us/beeig6 

File::Copy and permission bits

Charles Bailey stumbled across a pot of tuits and spent them on learning git and cleaning up File::Copy, specifically, the management of permission bits. The result was to make cp more POSIXy and copy more native.

Abigail added a few more tests to look at how the copy dealt with setuid and sticky bits, and pointed out that there was now a potential world visibility race condition in the code for anyone in a Unix session with a 022 umask (all too common). Charles thought that File::Copy's security mandate didn't stretch that far (the paranoid are invited to look at a hand-rolled solution involving sysopen).

  do you copy
  http://xrl.us/beeig8 


New and old bugs from RT

updated reproducible perlmodlib.PL with bugfix for Test::Tutorial's entry (#53000)

Richard Soderberg tossed out some code to attack a TODO, but Rafaël Garcia-Suarez wasn't able to make use of it, and it prodded Nicholas Clark to wonder whether our git-configure-fu was sufficiently strong. Richard said he'd try again some day.

  to do later
  http://xrl.us/beeiha 

sqrt without EXPR and Math::Complex always 0 (#62412)

Frank Wiegand found a problem with sqrt failing to operate on $_ if Math::Complex was loaded. Abigail found a way to use use Scalar::Util 'set_prototype' (suggested by Rafaël and have things work correctly before and after 5.10.

  the root of complexity
  http://xrl.us/beeihc 

Dies in recursion on some regexes (#62444)

Vany wondered why (1 x $_) !~ /^(1{2,}?){2,}$/ blew up with a segfault when $_ became large. Abigail had a look and replied that "This is to be expected given the regexp."

  EWONTFIX
  http://xrl.us/beeihe 

Fcntl exported constants become tainted under 5.8.9 (#62502)

Mark Martinec discovered that under perl 5.8.9, some O_* and other constants as exported by module Fcntl (or IO::File) become tainted under certain circumstances. Both Nicholas Clark and Ben Morrow were able to confirm seeing the same behaviour but were none the wiser as to why it was happening.

  and you don't make no sense
  http://xrl.us/beeihg 

Insecure dependency error in sprintf under -T (#62512)

Eric Promislow reported another easy bug to track down: problems with a sprintf format string being tainted... when run inside the Komodo debugger. Rafaël thought that if it couldn't be reproduced in a stock perl there wasn't much chance of it being fixable.

  heisentaint
  http://xrl.us/beeihi 

warnings::register regressions introduced in 5.10 (#62522)

Peter Rabbitson filed a report about a regression in 5.10 and backed it up with comprehensive set of tests to circumscribe the problem. Alas, it attracted no attention.

  http://xrl.us/beeihk 

Perl 5.8.8 (Tainting) vulnerable to CWE-732 attacks (#62526)

The attack in question is due to the fact that perl will happily allow a program to require a file (and thus, run code) even if said file happens to be world writable. There was some talk of making taint deal with this (refuse to load the file, à la openssh) but Abigail asked how one would go about untainting it.

Yves Orton thought that a new paranoia mode, activated by a new command-line switch, would be a better alternative.

  oh I give up, can I trust you?
  http://xrl.us/beeihn 

Method interpretation in A::B->C (#62584)

Peter Scott noted that what is called by A::B->C depends on whether or A::B and/or A::B::C are defined. This then leads to program behaviour that can be difficult to predict.

Eric Brine pointed out that A::B::->C is the preferred way of disambiguating the situation, and Ben Morrow explained why it is unlikely that a favourable resolution will made of the matter.

  there's one right way to do it
  http://xrl.us/beeihp 

Maximum string length with substr (#62646)

skylar reported running into grief with substr being asked to chop away at a string larger than 2147483648 bytes (give or take a byte). Nicholas Clark explained that the 1998-vintage implementation uses a 32-bit quantity which is clearly starting to become a limit in the 21st century.

  sign of the times
  http://xrl.us/beeihr 

Override documentation (#62648)

Jerrad Pierce would love to see override have its own documentation, or at least be able to get to it easily from overload, which does have its own documentation.

  overboard
  http://xrl.us/beeiht 

UTF8 failure with sprintf () (#62666)

H.Merijn Brand has managed to get the UTF-8 cache confused and panic with nothing more than a sprintf.


  doctor it hurts
  http://xrl.us/beeihv 

Perl5 Bug Summary

  http://xrl.us/beeihx
  http://rt.perl.org/rt3/NoAuth/perl5/Overview.html 


New Core Modules

podlators 2.2.2

Russ Allbery pushed out a new version, the main improvement being the ability to deal correctly with verbatim paragraphs that contain lines with only whitespace.

  http://xrl.us/beeihz 
Encode 2.27

Dan Kogai released a new version of Encode that clears up a number of reported bugs.

  http://xrl.us/beeih3 
Module::Load 0.14

Jos Boumans synced the latest version of Module::Load that corrects a reported bug and streamlines the test suite. Module::Load::Conditional was also upgraded to clear up some bugs in the test suite itself, smoked out by Win32 and VMS.

  sweet
  http://xrl.us/beeih5 
Archive::Tar 1.44

Jos also released a bug fix for Archive::Tar.

  http://xrl.us/beeih7 

In Brief

Some consting goodness for time64 was applied.

  http://xrl.us/beeih9 

Renée Bäcker thought it would be great to show -E in error message when called with -E, and on the surface it is. Except it breaks tests like $0 eq '-e' and to Do The Right Thing in light of that was so hard that all hope was abandoned.

  http://xrl.us/beeiib 

Slaven Rezic improved the warnings if forks fail in Perl_my_popen .

  http://xrl.us/beeiid 

Karl Williamson is getting close to landing code to solve the problem of the Unicode ordinal range 128-255 and so he delivered a skeleton legacy.pm to bring lay the way for genuine Unicode goodness.

  http://xrl.us/beeiif 

Ben Morrow spotted some curious behaviour when calling -t, -T on IO refs and a variation on the theme that tripped an assertion failure. He promised to write a patch after he had audited the rest of the codebase for uses of isGV().

  http://xrl.us/beeiih 

Ricardo Signes wrote a rough patch to remove indirect method calls from documentation. This was applied, but he admitted that other examples no doubt lie around in obscure corners of the documentation.(I thought chromatic wrote an exhaustive patch for this a couple of years ago --ed).

  http://xrl.us/beeiij 

Reini Urban pulled off a gnarly trick to use MSVC6 Project files to build perl.exe, which reduces the edit-compile-debug cycle on windows down to keypresses F7-F5.

  faster debugging
  http://xrl.us/beeiim 

Last week's summary

There was none. I needed a break. As much as I enjoy doing these summaries, it's a huge chunk of tuits each week.

About this summary

This summary was written by David Landgren.

Weekly summaries are published on http://use.perl.org/ and posted on a mailing list, (subscription: perl5-summary-subscribe@perl.org ). The archive is at http://dev.perl.org/perl5/list-summaries/ . Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation or attending a YAPC to help support the development of Perl.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I thought chromatic wrote an exhaustive patch for this a couple of years ago --ed

    I did, but it was too good: it patched the documentation of several dual-lived modules, and I felt exhausted when I considered contacting all of them separately.