Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Journal of nicholas (3034)

Monday February 08, 2010
06:04 AM

... unless you're a Sikh

I didn't know about that one. I knew that a practising member of the Sikh faith is allowed to wear a turban instead of a helmet when riding a motorcycle. I wonder how many other such exemptions are carefully written into UK law. I need a list - Wikipedia you fail me! :-(

Sunday February 07, 2010
11:20 AM

Memory usage

There were a few things that caught my attention in Facebook's presentation on HipHop, their PHP to C++ converter. It sounds like it relies on static analysis of the entire program's source, hence why they can't support eval, create_function etc. (22m25s in). I suspect that that sort of restriction would be, um, "interesting", in a general CPAN using environment, as a lot of modules build on various low level code that encapsulates eval, such as the traditional way h2xs did constants via AUTOLOAD. Also, as it's different runtime from Zend, so extensions need to be ported to it (19m in).

However, the most interesting part was a an early slide about memory usage, at 6m20. Transcribed:

150MB

for ($i = 0; $i < 1000000; $i++ ) {
      $a[] = $i;
}

700MB

for ($i = 0; $i < 5000000; $i++ ) {
      $a[] = $i;
}

(700M - 150M) / 4,000,000 = 144 BYTES

Does PHP really consume 144 bytes per integer value? Is that on a 32 bit or 64 bit machine?

For comparison, here is Perl:

$ perl -le 'for ($i = 0; $i < 1000000; $i++ ) { push @a, $i; }; print `cat /proc/$$/statm` * 4 / 1024'
22.4765625
$ ./perl -le 'for ($i = 0; $i < 5000000; $i++ ) { push @a, $i; }; print `cat /proc/$$/statm` * 4 / 1024'
118.44140625

which works out at 25.155 bytes per integer value, or under 20% of their figure for PHP. The odd number of bytes will be the malloc overhead spread across all the structures allocated from the same arena.

I have no idea what the usage of Python or Ruby are like, but there's a comment in the Unladen Swallow wiki:

Here at Red Hat we use Python for a lot of things. What we've observed is that execution performance is not the main issue (although it improving it would be greatly appreciated), rather it's the memory footprint which is the problem we most often encounter. If anything can be done to reduce the massive amount of memory Python uses it would be a huge win. I would encourage you to consider memory usage as just as important a goal as execution speed if you're going to tackle optimizing CPython.

Tuesday January 26, 2010
11:27 AM

Phished by Visa/3D insecure

Steven Murdoch and Ross Anderson systematically demolish 3D secure/Verified by Visa. Shame that the banks don't employ the smart people, just the security researchers and the malware authors.

Meanwhile, at work, we continue to love the banks. Such as a large UK bank who will authorise a Euro transaction on Maestro (which must have been with 3DS/VbV) yet only reject it at settlement time because you can only use (UK) Maestro in sterling. Another authorised a card, but then rejected it at settlement because it was Electron rather than Visa Debit, and that merchant wasn't allowed to accept Electron. I'd love to be big enough to have the clout to tell banks "if you authorise it, the only reason to subsequently refuse to settle it is because it was reported as missing/fraudulent in the meantime. Otherwise, you honour your authorisation", and bear the cost of (fixing) your own bugs.

And the coda on the second one - said large UK bank then admitted that its own binranges had an error. If the banks can't get that right, what hope has anyone else?

Thursday December 31, 2009
10:38 AM

spontaneous mysql UTF-8 encoding

Dear Lazyweb,

Has anyone seen the following - perfectly happy MySQL connection, with:

$VAR1 = [
          [
            'character_set_client',
            'latin1'
          ],
          [
            'character_set_connection',
            'latin1'
          ],
          [
            'character_set_database',
            'latin1'
          ],
          [
            'character_set_filesystem',
            'binary'
          ],
          [
            'character_set_results',
            'latin1'
          ],
          [
            'character_set_server',
            'latin1'
          ],
          [
            'character_set_system',
            'utf8'
          ],
          [
            'character_sets_dir',
            '/usr/share/mysql/charsets/'
          ]
        ];

(i.e. the connection is consistently latin1), and data that is Latin 1. Yet, sometimes the server will return the characters of Latin 1 encoded as UTF-8. Same query. Same table. Nothing changed.

And, yes, I know that it's the server, or at least the client libraries, because I have "instrumented" my copy of DBD::mysql:

--- dbdimp.c~   2008-04-29 00:36:17.000000000 +0100
+++ dbdimp.c    2009-12-31 14:30:58.000000000 +0000
@@ -3489,6 +3489,18 @@
     if (DBIc_TRACE_LEVEL(imp_xxh) >= 2)
       PerlIO_printf(DBILOGFP, "\t<- dbd_st_fetch, %d cols\n", num_fields);

+    {
+      char buffer[256];
+      sprintf(buffer, "/tmp/DBD.%d", getpid());
+      PerlIO *fh = PerlIO_open(buffer, "a");
+      if (fh) {
+       do_sv_dump(0, fh, av, 0, 32, 0, 0);
+       PerlIO_close(fh);
+      } else {
+       croak("open '%s': %d", buffer, errno);
+      }
+    }
+
     return av;
   }
   else
@@ -3601,6 +3613,17 @@
         (void) SvOK_off(sv);  /*  Field is NULL, return undef  */
     }

+    {
+      char buffer[256];
+      sprintf(buffer, "/tmp/DBD.%d", getpid());
+      PerlIO *fh = PerlIO_open(buffer, "a");
+      if (fh) {
+       do_sv_dump(0, fh, av, 0, 32, 0, 0);
+       PerlIO_close(fh);
+      } else {
+       croak("open '%s': %d", buffer, errno);
+      }
+    }
     if (DBIc_TRACE_LEVEL(imp_xxh) >= 2)
       PerlIO_printf(DBILOGFP, "\t<- dbd_st_fetch, %d cols\n", num_fields);
     return av;

"mysql_st_internal_execute MYSQL_VERSION_ID 50134", DBD-mysql-4.007, DBI-1.609, perl 5.8.8, x86_64 GNU/Linux

Wednesday December 30, 2009
04:49 AM

What we should thank the X-Factor for

Some people seem sufficiently upset about the X-Factor "always" being Christmas Number 1 that they did something about it.

However, I think that we forget that we should be thankful for X-Factor number 1s - at least it stops Cliff getting it. (And the wannaby trendy UK might be ashamed to know that Cliff isn't the hardest working pop act out there - it's actually The Quo.)

Friday December 18, 2009
04:30 AM

some vehicles serve no purpose

So, this morning on the way in, the (4 wheel drive) Suzuki in front of me was trying to pull away from me at some traffic lights, but was having trouble because its right rear wheel was spinning. Me, on a racing bike behind it had no problems.

Some vehicles serve no purpose.

Context - It snowed last night. There's a couple of inches* of snow, and unlike last time, none of the roads have been gritted. Grr.

* For people outside the UK, the correct unit conversion for snow is 1 inch == 1 metre. This will make everything make sense.

Friday December 11, 2009
02:55 PM

Some things are not what they seem or what they say they are

So I have a bottle. A green bottle. The label says "Aspall Crisp Draught Suffolk Cyder". Most of that is true. Apart from the Draught part. Clearly it's a bottle. So, um, why use that word?

But that's not actually the most disconcerting thing about it. The rather confusing thing about it is that it's made of plastic. Yes, this rather nice high-end cider is being sold in a bottle, that at first inspection seems to be rather trog. You pick up a full bottle, and you have to keep checking that it's really 500ml, not 330ml, because it doesn't feel right for a half litre of booze.

But I can see why they're doing it. This bottle weighs 32g. A glass bottle (admittedly technically a beer bottle, as everything else went with the recycling yesterday morning) weighs 321g. So this bottle is a tenth of the weight, and reduces the container from 40% of the total shipped weight to 6%.

Hmm, I've been here before.

05:47 AM

Escaping the pantopticon

It seems that if moving to Switzerland isn't feasible, another way to escape the prying eyes of Google Street View is to head underground. It seems that the spycars only go so far down tunnels. (Or one did, and never came back, so the others avoid dark places). I wonder who has been waiting in tunnels and mugging black Opels.

05:33 AM

Random interesting geology link

A Lake Divided - A History of the Southern Pacific Railroad Causeway and Its Effect on Great Salt Lake, Utah

Well, I thought that it was interesting. I assume that this and this are the ends of the 1950s causeway described in the article.

Thursday December 10, 2009
08:48 AM

oиgoiиg busway fuи

The (expensive) farce continues - (temporarily) illiterate workmen managed to get an N backwards in some busway signage.