Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

ethan (3163)

ethan
  reversethis-{ed. ... rap.nov.olissat}

Being a 25-year old chap living in the western-most town of Germany. Stuying communication and information science and being a huge fan of XS-related things.

Journal of ethan (3163)

Wednesday December 17, 2003
04:02 AM

magical insanity

[ #16385 ]

The recent discussions on the porters-list about tieing in general and in particular some mumblings about its deficiencies made me play around a little with it. In essence, some people regretted that tied variables weren't implemented on top of a full virtual table. The virtual table is a C-structure holding pointers to C functions. When a variable is tied, eventually two different virtual tables are involved. The one of PERL_MAGIC_tied refers to two functions, whereas PERL_MAGIC_tiedelem refers to three other functions.

When you have a tied hash and do a print $hash{key} unexpected things happen. Perl creates a dummy scalar variable and flags it as PERL_MAGIC_tiedelem and puts the virtual table PL_vtbl_packelem into it. After that, perl would do a copy of this element (which happens in sv_mortalcopy which triggers (eventually) sv_setsv_flags. It's at this point that perl figures out that the dummy scalar has magic attached to it and triggers mg_get. So perl does it sort of backwards: when you would expect FETCH to be triggered (namely when the lookup of the key happens in the hash) it returns a magic scalar variable. The lookup happens when this scalar is copied.

The problem with this approach is that perl needs to stuff an incredible amount of information into the dummy scalar, because the FETCH method is invoked on the object backing up the tied hash. So this object needs to be stored in the dummy as well plus the key to be looked up.

It gets even more insance when you do a $hash{key} = "val". The same sort of deferred lookup happens here, only this time followed by storing "val". Now you need even more information: the dummy needs the object, the key and the value. This is all crammed into one dummy scalar. This scalar of course still has the virtual table so it's a really huge scalar.

All the above is just a tiny picture of the real problems. Many of the methods of tied variables aren't even looked up in a virtual table, such as SPLICE and many more. In this case, the OP checks whether the variable is magic and then triggers the respective method from the tieing class. There is no virtual table involved here. In short, the whole thing is a mess.

Initially I thought it would be rather easy to make a proper virtual table with maybe 15 or even more pointers to functions so that every tieing method could be looked up in this table. The above should make clear this that this is not so easy. So far I managed to make EXISTS, FETCH and STORE work for hashes. My impression however is that it doesn't address the problem of this odd dummy variable that is packed with information. A normal STORE operation has all its required information (($self, $key, $val)) still packed in one scalar, which looks quite impressive when dumping it. The below shows the innards when doing $hash{test} = "bla":


SV = PVLV(0x8176030) at 0x817834c
    REFCNT = 1
    FLAGS = (TEMP,POK,pPOK)
    IV = 0
    NV = 0
    PV = 0x81763f8 "bla"\0
    CUR = 3
    LEN = 4
    MAGIC = 0x8175fd0
        MG_VIRTUAL = 0x81629e0
        MG_TYPE = PERL_MAGIC_tied(P)
        MG_FLAGS = 0x02
            REFCOUNTED
        MG_OBJ = 0x816615c
                SV = RV(0x8175934) at 0x816615c
                    REFCNT = 2
                    FLAGS = (ROK)
                    RV = 0x8166120
        MG_LEN = -2
        MG_PTR = 0x8178358 => HEf_SVKEY
                SV = PVIV(0x8166760) at 0x8178358
                    REFCNT = 2
                    FLAGS = (POK,pPOK)
                    IV = 0
                    PV = 0x8175fc0 "test"\0
                    CUR = 4
                    LEN = 5
    TYPE = T
    TARGOFF = 0
    TARGLEN = 0
    TARG = 0x8173838

MG_OBJ = 0x816615c becomes $self, MG_PTR = 0x8178358 => HEf_SVKEY becomes $key and the whole thing is $val when invoking STORE. The above also shows that I no longer distinguish between PERL_MAGIC_tied and PERL_MAGIC_tiedelem. Current perl would show MG_TYPE = PERL_MAGIC_tiedelem(p) in a dump like the above. I had to change it because the virtual table of a tied element is too small.

Sorry that this day's journal was sort of technical. I am really just storing the things I found out during the past three days for later look-up. Christmas is coming closer and I wont be sitting before a computer for about two weeks. Hence there is a realistic danger that I'll have forgotten all my discoveries after that.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Duh. And I thought this was a post about Jamiroquai.

    Hey, want to take over the p5p summaries ? *ducks*

    • Duh. And I thought this was a post about Jamiroquai.

      Sorry for being misleading. I just realized that "virtual insanity" would have been an equally fitting subject.

      Hey, want to take over the p5p summaries ? *ducks*

      No chance. You need to be a spindoctor to write those summaries. But I only understand a tenth of all the discussion on p5p. And half of this tenth I get wrong.

      You're doing a far too good job on the summaries, Rafael, so I wont raise my hands. :-)
      • I just have to point out that when I began the summaries I wasn't understanding much discussion on P5P either. Writing summaries was a great way to learn. But learning is time-consuming :)
        • I guess it must be very time-consuming. I often catch myself in the act of skipping one or the other thread because I either find it less intriguing or incomprehensible right from the start. I suspect skipping whole threads isn't affordable once you are responsible for the weekly summaries.

          On the other hand you could probably talk me into this although I am aware of how much I'll regret it later. :-] Remind me again a week after new year's eve when I'll have returned from my parents. Meanwhile, I'll try to