Whiteknight's Journal http://use.perl.org/~Whiteknight/journal/ Whiteknight's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:31:53+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 Whiteknight's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~Whiteknight/journal/ Cessation of Use.Perl blogging http://use.perl.org/~Whiteknight/journal/38911?from=rss <p>This is going to be my last post to this use.perl journal. From now on, all my Parrot- and Perl-related posting will happen on my <a href="http://wknight8111.blogspot.com/">Blogger blog instead</a>. There are a number of reasons for this:</p><p>1) My wife and I are going to do all our personal family-related blogging on a dedicated blog, so my current personal blog is able to be completely devoted to technical topics.<br>2) I maintain 4 other blogs at Blogger on various topics, so it makes more sense to keep everything together.<br>3) The interface there is a lot better then the interface here, and I'm pretty sick of having to write out HTML tags by hand anymore.<br>4) Since I'm going to the blogger website daily anyway, I'm more likely to remember to write regular posts then I am here.</p><p>All my Parrot- and Perl-related blogs will be labeled with "Parrot" or "Perl", so people interested in subscribing to the crap I have to talk about can filter out the unrelated stuff. So, without further adieu, I bid use.perl goodbye forever.</p> Whiteknight 2009-05-02T02:23:48+00:00 journal JIT on Parrot http://use.perl.org/~Whiteknight/journal/38827?from=rss <p>The JIT system on Parrot is a little bit of a mess, and one that has been mostly ignored for a while now, except for the increasingly-frequent occasions when it breaks.</p><p>So far back as <a href="http://groups.google.com/group/perl.perl6.internals/browse_thread/thread/9b61366be89b4333">2004</a> people in Parrot were talking about libJIT but decided it made more sense to use a custom-made JIT backend solution that was targeted to the idiosyncracies of Parrot "from day 1". Several years later, with the release of 1.0.0 slowly fading into the distance behind us, our custom-built system is not only no better then libJIT or LLVM for our specific needs, and is more likely to need to be completely scrapped and rewritten for it to be useful at all. Plus, JIT is only implemented on i386, which is only a small portion of our target systems.</p><p>Parrot's JIT system is a complicated one. Parrot hands over the current bytecode stream to the JIT engine, which in turn compiles the bytecode into native machine code and executes it. It does this by maintaining a large list of Regex-based definitions for opcodes, and writing the glue code necessary to pass between them and the other parts of Parrot (the PMC system vtables, for instance). This system requires that we write, from scratch and with minimal abstractable overlap, a new JIT engine for each system we want to support.</p><p>An engine like LLVM or libJIT (or even GNU Lightning, although I don't know as much about that project immediately simplifies everything. We can write the solution once, and have it just work on all platforms where the JIT backend system is supported (which, at the moment, is almost a superset of all the platforms that Parrot supports, for both engines). Plus, for both engines, we suddenly get all the benefits while only having to write one interface: automatic machine code generation and execution, cross-platform stability, and code optimization. It's like a win-win for us!</p><p>Tewk++ submitted a very interesting application to GSOC to implement an LLVM-based JIT system for Parrot. I don't know when winning projects get announced, but if he's in that group I'm sure I'll post a short hallelujah here. I've also been doing some looking into libJIT myself, and might dabble with that concurrently. Having a good JIT engine in Parrot would be very good, but having a sane interface that would support multiple JIT cores would be great.</p><p>In the meantime, my opinion is that our current JIT system should be ripped out wholesale. We don't really have a working system currently, just the illusion of one. It doesn't work on all our target systems, and where it does work it is fragile and messy and unreliable. It's better to admit to ourselves that we don't currently have an acceptable JIT solution. Hopefully, that realization will galvanize us into implementing a real JIT post haste.</p> Whiteknight 2009-04-18T22:59:49+00:00 journal Project updates http://use.perl.org/~Whiteknight/journal/38681?from=rss <p>A few miscellaneous updates:</p><p>1) Matrixy now has an assortment of TAP-related testing functions: plan(), ok(), nok(), is(). We also have functions start_todo() and end_todo() to mark blocks of tests as TODO tests. This morning I went through the test suite and updated most of our tests to use these routines. It isn't a perfect TAP implementation, and not all the expected functionality exists, but it's a start.</p><p>Blair has been doing more great work on the various library functions, and I'm much closer now to getting BLAS and LAPACK working like normal on my system (He's on Win32, I'm on Ubuntu-x86_64, so it's been hard to get all the NCI stuff properly working on my system)</p><p>Our test suite is really growing by leaps and bounds, which helps to put into perspective how much work we have done (a lot) and how much we still need to do (also a lot). It's a testament to Parrot and PCT that this project is progressing so quickly.</p><p><a href="http://code.google.com/p/matrixy">Matrixy</a></p><p>2) The Perl 6 Wikibook is still developing, albeit slower then it had been. Matrixy has been taking up a lot of my attention recently, and I am trying to strike while the iron is hot. I've been adding snippets of material here and there. I'm planning to get a fresh Rakudo release soon to facilitate some testing about concepts that I am not too familiar with yet. I'm hoping to get a good amount of work done this week, and I would appreciate any feedback that people have.</p><p><a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6 book at Wikibooks</a></p> Whiteknight 2009-03-21T17:01:23+00:00 journal Function Definitions http://use.perl.org/~Whiteknight/journal/38667?from=rss <p>Continuing with my series about M, I'm going to talk today about defining functions today. I discussed how to call and use functions last time, and today we are going to learn how to make our own.</p><p>Functions are defined simply with the "function" keyword, and are terminated with the "endfunction" keyword. Here's an example:</p><p> &nbsp; function x()<br> &nbsp; &nbsp; &nbsp; disp("Hello World!");<br> &nbsp; endfunction</p><p>It's worth noting here that semicolons should probably be used to terminate all statements inside a function, because the values of statements will still be printed to the console otherwise.</p><p>Parameters can be defined as expected:</p><p> &nbsp; function foo(a, b, c)<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; printf("values: %d, %d, %d", a, b, c);<br> &nbsp; endfunction</p><p>Where things get a little different from what we (Perl programmers mostly) are used to is in return values. The return values of a function are defined in the function signature itself. The "return" keyword, while still useful for exiting a function prematurely, does not take a value itself.</p><p> &nbsp; function s = sum(a, b)<br> &nbsp; &nbsp; &nbsp; &nbsp; s = a + b;<br> &nbsp; endfunction</p><p>Here is almost the exact definition of the pi() function in the Matrixy repo:</p><p> &nbsp; function p = pi()<br> &nbsp; &nbsp; &nbsp; &nbsp; p = 3.141592653589<br> &nbsp; endfunction</p><p>M allows multiple return values too:</p><p> &nbsp; function [a, b, c] = bar()<br> &nbsp; &nbsp; &nbsp; &nbsp; a = 1;<br> &nbsp; &nbsp; &nbsp; &nbsp; b = 2;<br> &nbsp; &nbsp; &nbsp; &nbsp; c = 3;<br> &nbsp; endfunction</p><p>All input and output parameters in M are optional. Even if we define 3 named parameters, a caller could pass only one or two (or 4 or more!) without causing a problem. It's the function's job to count the number of values that it has received and modify it's behavior accordingly. This is done simply through the "nargin" and "nargout" keywords:</p><p> &nbsp; function [a, b] = baz(c, d)<br> &nbsp; &nbsp; &nbsp; &nbsp; if nargin == 1<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; d = 0<br> &nbsp; &nbsp; &nbsp; &nbsp; elsif nargin &gt; 2<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; error("too many args passed!");<br> &nbsp; &nbsp; &nbsp; &nbsp; end<br> &nbsp; &nbsp; &nbsp; &nbsp; if nargout &gt;= 1<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; a = c + d;<br> &nbsp; &nbsp; &nbsp; &nbsp; elsif nargout &gt;= 2<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; b = c - d;<br> &nbsp; &nbsp; &nbsp; &nbsp; else<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; error("too many output args!");<br> &nbsp; &nbsp; &nbsp; &nbsp; end<br> &nbsp; end</p><p>There's no such thing as multidispatch in M, if you want a function to do different things with different numbers and compositions of arguments, you have to code the dispatching logic yourself.</p><p>If you want to account for argument lists of indeterminant length, you can use the slurpy "varagin" and "varargout" keywords:</p><p> &nbsp; function [c, varargout] = bazooka(a, b, varargin)<nobr> <wbr></nobr>...<br> &nbsp; endfunction</p><p>varargin and varargout are special data types called "cell arrays" that we haven't discussed yet, so we won't see them used here.</p><p>That's a brief introduction to functions in M. We don't have all of this implemented in Matrixy yet, but we are making some pretty amazing progress. Come check it out.</p><p><a href="http://code.google.com/p/matrixy">Matrixy on Googlecode</a></p> Whiteknight 2009-03-19T23:20:57+00:00 journal Matrices and Functions in M http://use.perl.org/~Whiteknight/journal/38646?from=rss <p>Continuing my short tutorial about M, today I'm going to talk about variables and functions in M. This is, simultaneously, one of the hardest parts of the Matrixy compiler for us to get right, and we've just made our second attempt at it.</p><p>Functions in M are named and called very similarly to how functions are called in other languages like C or Perl:</p><p> &nbsp; foo(1, 2, 3);</p><p>Again, as we covered last time, omitting the trailing semicolon will print the return value of the function to the console, if any. As we'll see later, functions always take variadic argument lists by default, and it's up to the function itself to keep track of it's own input arguments. Functions can also return multiple values:</p><p> &nbsp; [a, b, c] = foo(1, 2, 3);</p><p>And again, it's up to the function to recognize how many return values are expected and alter it's behavior appropriately. There is no multiple dispatch in M, the function implements it's own logic internally to determine the configuration of input and output parameters and adjust it's behavior to that.</p><p>Matrices, as we saw previously, are very important to M. It originated as a linear algebra mathematics pack, after all. We index into a matrix variable in the same way that we call a function:</p><p> &nbsp; x = [1 2 3 4];<br> &nbsp; x(1) % ans = 1</p><p>This causes obvious problems with our parser at compile time because we have to determine whether "foo(1)" is a function call or a variable index. To make things a little more difficult, a bare identifier could also be either a variable or a function call with no arguments:</p><p> &nbsp; help; % Calls the "help" function with no args<br> &nbsp; x; % the variable x. store it's value in ans</p><p>So, as you can see, there is a lot of ambiguity that needs to be resolved at runtime when the parser finds an identifier bar:</p><p>1) If a variable bar has been defined locally, treat it like a variable and return it's value. Variables of the same name always overshadow functions.<br>2) If we have a user-defined function bar in the current scope, call that.<br>3) If there is a builtin function bar, call it<br>4) If all else fails, search the path for a file named "bar.m". this file should contain either a defintion for function bar, or it should be a "script file" which takes no parameters but is executed directly.</p><p>This is all made more difficult because M allows function handles to be stored in variables and called as functions:</p><p> &nbsp; y = @foo; % handle to function foo<br> &nbsp; y(1, 2); % call function foo(1, 2)</p><p>or even anonymous functions to be defined:</p><p> &nbsp; x = @(r) 2 * pi * r;<br> &nbsp; x(3); % 2 * pi * 3;</p><p>So even if baz is a variable as determined by rule #1 in the dispatch algorithm above, it could still cause the invocation of a function if it's a function handle.</p><p>Functions also have the ability to be executed using bare word arguments:</p><p> &nbsp; help foo</p><p>is the same as the call</p><p> &nbsp; help('foo')</p><p>The idea behind the ambiguity between variable and function dispatching, at least as I've heard it, is that it facilitates the interchange of pure functions and lookup tables. Either can be implemented, and the caller doesn't have to worry about which type it is.</p><p>Another point to be aware of is that M traditionally doesn't have namespaces, so all functions are visible by default at all time. This results in a large amount of namespace pollution, but also appears to be a driving force behind the multistage dispatch algorithm I showed above: There are far too many identifiers defined by default to prohibit the user from overriding any of them locally.</p><p>One last thing to mention is that if we absolutely need to call a function instead of indexing into a variable of the same name, we can use the feval() function. feval() takes the name of a function and a variadic list of arguments and calls that function, never indexing into a variable:</p><p> &nbsp; x = [1 2 3];<br> &nbsp; feval("x", 2); % function x, not variable x</p><p>So this has been a brief introduction to function and variable use in M. It should demonstrate that there is some significant difficulty on the part of the compiler-designer, but it creates an interesting situation for end users to interchange functions and lookup tables, and to silently override the huge library of builtin functions with their own versions of them if needed.</p><p><a href="http://code.google.com/p/matrixy">Matrixy Compiler for Parrot</a></p> Whiteknight 2009-03-15T14:04:23+00:00 journal Basic statements in M http://use.perl.org/~Whiteknight/journal/38623?from=rss <p>I promised to write a short series about programming in M, to try and show some of it's cool features. For those who don't know, M is the scripting language used by Matlab (proprietary) and Octave (Free). M is designed for linear algebra and engineering simulations.</p><p>Statements in M are very similar to statements like we would see in languages like C and JavaScript. Statements take very similar syntax, and are (mostly) terminated with a semicolon:</p><p>x = 1;</p><p>x = sin(y) + 1;</p><p>When I said "mostly" above referring to the semicolons because they ar optional. If you leave the semicolon off the end, the value of the statement is printed to the terminal:</p><p>Write this: x = (3 + 5) * 2</p><p>And get this: x = 16</p><p>If you don't assign the value to a variable, you get the result stored in the default variable "ans"</p><p>Write this: 3 + (5 * 2)</p><p>Get this: ans = 13</p><p>M is based on matrices, so dealing with them is very easy:</p><p> &nbsp; x = [1 2 3; 4 5 6]</p><p>In the matrix constructor, the semicolons separate rows in the matrix. Elements in the row can be separated by commas or whitespace.</p><p>Matrices are used for working with strings too. Strings in a row are concatenated together:</p><p> &nbsp; x = ["hello ", "world"]</p><p>prints:</p><p> &nbsp; x = hello world</p><p>A matrix can either be treated like a string or like a regular numerical matrix, so writing this:</p><p> &nbsp; x = ["ok ", 49]</p><p>prints this:</p><p> &nbsp; x = ok 1</p><p>We've actually used something very similar to that in the Matrixy test suite.</p><p>Combining matrices together is also very easy:</p><p> &nbsp; x = [1, 2, 3];<br> &nbsp; y = [4, 5, 6];<br> &nbsp; z = [x y]</p><p>prints this:</p><p> &nbsp; z = 1 2 3 4 5 6</p><p>As rows:</p><p> &nbsp; z = [x;y]</p><p>prints this:</p><p> &nbsp; z =<br> &nbsp; 1 2 3<br> &nbsp; 4 5 6</p><p>One caveat about matrices is that they must be uniform rectangular matrices. You can't have rows that are different lengths:</p><p> &nbsp; x = [1 2 3]<br> &nbsp; y = [4 5 6 7]<br> &nbsp; z = [x;y] % error!</p><p>This caveat is actually relaxed when it comes to strings, you can have strings of different lengths on different rows.</p><p>So that's a quick introduction to statements in M. Throughout the week I'll post more information about other aspects of it.</p> Whiteknight 2009-03-09T23:31:12+00:00 journal Progress on Matrixy http://use.perl.org/~Whiteknight/journal/38617?from=rss <p>I haven't been doing much work on Parrot or even the <a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6 book</a> this week, instead focusing most of my efforts on Matrixy. Matrixy, which I introduced in my last post, is an M (Matlab/Octave) compiler for Parrot.</p><p>Progress has been going well, and just this morning I got a few more tricky features working. "working" is, of course, open to multiple definitions. This is especially true for such an idiomatic language as M.</p><p>I got nargin and nargout mostly working, along with varargin. These are the tools for doing variadic argument lists in M. Implementation isn't perfect still, but at least it's a start. I'm still fighting with lots of issues when it comes to function/variable dispatching. I finally have a fix ready that does what we need it to do, but it relies on the use of lexical variables which terribly breaks interactive mode because of issues with PCT.</p><p>Blair has been doing some great work on the libraries, and he's got some NCI stubs in place for interfacing with the BLAS and CLAPACK libraries, and converting our ResizablePMCArray-based matrices into the forms needed for use with these libraries. Linear algebra power is coming to Parrot!</p><p>I realize that a lot of people aren't familiar with M, it's a pretty specialized language that's used primarily for engineering modeling and scientific research. Maybe in my next few posts here I'll give a quick tutorial on the language and try to show some of the weird/cool idiosncracies that simultaneously make it an interesting language to work with and a difficult language to write a compiler for.</p><p><a href="http://code.google.com/p/matrixy/">Matrixy project page</a></p> Whiteknight 2009-03-08T19:52:16+00:00 journal Matrixy: M on Parrot http://use.perl.org/~Whiteknight/journal/38590?from=rss <p>I've finally gotten my M interpreter project off the ground. M, for those not familiar with it, is the scripting language used by Matlab and Octave, and is oriented towards linear algebra and mathematical modeling. I had started idly working on it back when I was still in school as a way to pass the time on long train rides. I gave it up to focus on Parrot internals work for GSOC, and never went back to it because it felt to me like PCT was changing and evolving too rapidly for me to keep up with it.</p><p>Well things are more stable now and I found another interested participant for the project. Things are going very well now, and we're getting far more work done on this now then I was ever able to get done by myself. Blair is working on NCI bindings for the BLAS and LAPACK libraries, which will bring a lot of mathematical muscle to our little compiler and to the Parrot ecosystem as a whole. I've been focusing my attentions lately on getting some of the core syntax and idiosyncratic semantics of M implemented. Some things I've gotten to work now:</p><p>1) The ';' is used as a statement terminator like it is in Perl or C. However, it's optional. If ';' is ommitted from the end of a statement, the value of that statement is printed to the console. So writing "x = 5" will print "x = 5". A bare expression "5 + 6" will print "ans = 11". Writing "5 + 6;" will set the value of the default variable "ans" to 11, but won't print anything to the console. This mostly works.</p><p>2) Matrix indexing and function dispatching both use parenthesis. So "x(1)" is treated as the first element in array x if x is a variable, or is treated as calling the function x with argument 1 otherwise. This is confusing as hell in the parser, although it's an interesting design decision for the M language. You can interchange a function with a lookup table in M without having to change any calling syntax at all. This is partly implemented, although a few thorns are still standing in my way from getting this right.</p><p>3) Functions are typically all defined in their own files, so the function foo() will be defined in "libpath/foo.m". I've written a basic function dispatcher that handles this lookup, if a builtin function hasn't been found. Blair also did some great work refactoring my search path handling to be more correct and less hackish.</p><p>4) I've got basic Matrix definition implemented. You declare a matrix like this: x = [1, 2;3, 4] and if you leave the semicolon off, this will be printed:</p><p> &nbsp; &nbsp; x =</p><p> &nbsp; &nbsp; 1 2<br> &nbsp; &nbsp; 3 4</p><p>So that's looking good. Matrices are constructed by nesting ResizablePMCArrays, which isn't an ideal solution (especially as the dimensions of the matrix increase above 2, and we try to keep things uniform length).</p><p>So that's how things are progressing with Matrixy. I'd like to invite any other interested participants to check out the source code and see what progress we've made so far. There's a lot of work left to do and, especially as pertains to the linear algebra and mathematics work, a lot of features for Matrixy that could benefit the rest of Parrot world.</p><p><a href="http://code.google.com/p/matrixy/">Matrixy at Googlecode</a></p> Whiteknight 2009-03-04T15:31:57+00:00 journal parameter passing http://use.perl.org/~Whiteknight/journal/38545?from=rss <p>All the magic about handling parameters in Parrot, all the slurpy params, flat args, named args, and optional params are all sorted out in the function Parrot_process_args. This function is called most often from parrot_pass_args and parrot_pass_args_fromc. These, in turn, are called from some very interesting places:</p><p>1) It gets called from the get_params opcode</p><p>2) Called from the set_returns opcodes</p><p>3) It gets called from inside the generated code of C-defined methods</p><p>4) a handful of other places such as in exception handlers called from certain places.</p><p>Parrot_process_args depends on certain values from the callers context, such as Keys (that I discussed last time) that can store references to registers instead of their values directly. This can cause a problem because of the various invocation paths and their handling of contexts in different ways:</p><p>1) Parrot_PCCINVOKE and Parrot_pcc_invoke_* functions create a new context to store passed-in params, and then invoke their sub objects.</p><p>2) The invoke vtable of a Sub creates a new context for itself</p><p>3) The generated C code of a C-defined method creates a new context for itself (NCI PMCs do not create one when invoked, so this is the workaround for that)</p><p>So if we are going through a Parrot_pcc_invoke_* call, we're going to be creating two contexts for almost every call: One context created in Parrot_pcc_invoke_* to hold the passed-in params, and one context created in the Sub or METHOD itself. Now when we finally do call Parrot_process_args, it can be either 1 or two levels below the context of the caller, which causes all sorts of problems when it comes to register references. I've conceived of an idea to store the callers context somewhere (possibly inside the CallSignature PMC somewhere) to handle these corner cases, but that might be a job for much much later in the refactoring process.</p><p>I've tried to resolve these references in Parrot_pcc_build_sig_object_from_varargs, but every time I do it seems to create hangs or crashes that I don't quite understand yet. I plan to get to the bottom of that soon.</p> Whiteknight 2009-02-24T14:54:10+00:00 journal Calling conventions refactor: Update http://use.perl.org/~Whiteknight/journal/38523?from=rss <p>I've been keeping busy lately with two projects. The first, a new one, is finally starting to implement the Matlab/Octave-on-parrot compiler. The second, an old one, is my continuing work on the calling_conventions refactor stuff.</p><p>I'll talk about the Matlab/Octave compiler, which is now called "Matrixy" later. For now you can check it out at it's <a href="http://code.google.com/p/matrixy">Googlecode homepage</a>.</p><p>My current cc work involves swapping out all the calls to Parrot_PCCINVOKE with calls to Parrot_pcc_invoke_method_from_c_args (or related variants). I had been trying to update the calls in the file src/io/api.c, but a few weird recursion situations were causing intermittent segfaults or coredumps. I worked on this for a few days, was getting pretty frustrated by it, and decided to do something else. So I picked up a Trac ticket where a segfault was causing problems involving overriding the init vtable method.</p><p>vtable overrides have been being handled by calls to Parrot_run_meth_fromc_args and related variants. While tracing through the segfault, I found that these functions are not very robust in their handling of contexts and were creating poisonous situations if they were used inside an invocation that used one of the different families of functions. I replaced the call to Parrot_run_method_fromc_args with Parrot_pcc_invoke_from_sig_object, and the segfault disappeared.</p><p>So, thinking back to some of the errors I was having with my cc work, I realized that Parrot_run_meth_fromc_args was poisoning recursive calls to Parrot_pcc_invoke_method_from_c_args, and I needed to destroy the former before I could make the switch over to the later. Working ast night and this morning, I finally made the switch. The switchover wasn't perfect because now I'm getting some weird errors in some of the tests, but it's better going then it was. I need to do some tests now to see if the functions in src/io/api.c can be updated now without causing more errors like it was earlier.</p><p>The calling_conventions refactor is large and sweeping, and is being done in multiple parts. Allison is working on another branch now to try and factor out common code from generated methods. I'm trying to unify all the various invocation functions to use common code. Once we get all the common code into one place, we're going to optimize the hell out of it (I already have lot of ideas for that!). The end result is that the system should be both more elegant and faster.</p><p>I don't think all of this work will land prior to 1.0, and in fact none of it might if we can't make the individual updates stable enough. Post-1.0 however, a lot is going to change under the hood, and all for the better.</p> Whiteknight 2009-02-21T21:38:07+00:00 journal Examples Needed! http://use.perl.org/~Whiteknight/journal/38500?from=rss <p>Yesterday I made a few misc changes to the <a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6 book at Wikibooks</a>, although nothing as substantial as I would have hoped. My wife hasn't been feeling well and Parrot is preparing for a big release today (the last one before 1.0 next month) so I've been keeping busy with those things and haven't had a lot of time to spare.</p><p>Yesterday I did add a few sections about feeds, gather/take, and placeholder arguments. I had intended to start writing about exceptions and then segue into a discussion about control exceptions. However, as I started digging through the synopses I realized that there really weren't any good examples of them in action for me to base my work on.</p><p>I know some of the basics about the Perl 6 exceptions implementation, but I don't have a really firm grasp on it just yet. I'm going to have to do some digging to try and find some examples to make sure I'm explaining it right in the book. Once I get exception handling down, I'm going to talk about control exceptions and then the various special named blocks.</p><p>That out of the way, I have very few subjects left to write stub drafts for:</p><p>1) Grammars and regexes<br>2) Threading and Concurrency<br>3) File IO<br>4) Packages/Modules<br>5) Perl Culture</p><p>With all the information in place, it's just a matter of expanding discussion and adding various code examples to fill the book out. I plan to be concurrently expanding both the book on Wikibooks and the book in the Pugs repo at the same time, since they will both be needing a lot of the same work at that point.</p><p>I could really use some good example code, especially code that *should* work and be best-practices Perl 6. Most of the code I can find right now is filled with all sorts of hacks and workarounds because of shortcomings in the various Perl 6 implementations. I have to do some serious digging through the Pugs test suite to get some ideas (I may even help improve/verify some of the tests too, while I'm at it).</p><p><a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6 book at Wikibooks</a></p> Whiteknight 2009-02-17T13:54:50+00:00 journal Perl 6: Junctions, Lists and Metaoperators http://use.perl.org/~Whiteknight/journal/38466?from=rss <p>I've been slacking a little bit on the Perl 6 book recently, because there has been so much other "high priority" work on my todo list. I do it to myself of course, taking on more projects then I can reasonably do in a short period of time. It's part of my mentality where I like having lots of little projects that I can jump back and forth between to keep my interest level high.</p><p>This works great for software projects where "commit early, commit often" leads to smaller changesets. I can make a change and then kick off the test suite, and then do some book work while I wait for that to complete. Yesterday was one of those days, I was doing some calling conventions work for Parrot (which I'll talk about in a different post), and was able to get some good book work done simultaneously.</p><p>Yesterday I added stubs for three chapters: "<a href="http://en.wikibooks.org/wiki/Perl_6_Programming/Junctions">Junctions</a>", "<a href="http://en.wikibooks.org/wiki/Perl_6_Programming/Lazy_Lists_and_Feeds">Lazy Lists and Feeds</a>", and "<a href="http://en.wikibooks.org/wiki/Perl_6_Programming/Meta_Operators">Metaoperators</a>". All of these are big topics, and I've barely scratched the surface with any of them. I always find the hardest part is getting those first few words and headings on a page, after that it's just filling in the blanks and expanding on stuff that's already there.</p><p>I have two blank pages in the middle: "Language Extensions" and "Operator Overloading". These are both meant to follow and build upon the chapters on Grammars and Regexes. I'm thinking I may combine these into a single chapter and talk in a single place about modifying the Perl 6 grammar itself. I'm still not sure about all the particulars of that, so I'll hold off on it for now.</p><p>The other day I put out a call for help on <a href="http://www.parrotblog.org/2009/02/readers-needed.html">ParrotBlog</a> to get some readers, reviewers, and contributors for Parrot documentation. I'd like to put out a similar call or this book: I would like to get some help with the Perl 6 book on Wikibooks. Readers, reviewers, editors, contributors, all are needed. Contributing examples of working Perl 6 code, especially if that code demonstrates best practices, is most necessary. Any feedback I can get, no matter how small, would be very much appreciated. This book is still in development, but things will go much faster if more eyes are reading it and more hands are working on it.</p><p><a href="http://www.parrotblog.org/2009/02/readers-needed.html">Perl 6 Book at Wikibooks</a></p> Whiteknight 2009-02-12T13:37:20+00:00 journal Working on my MediaWiki Bot http://use.perl.org/~Whiteknight/journal/38431?from=rss <p>I've talked about my <a href="https://sourceforge.net/projects/wikiperl/">MediaWiki bot</a> on this journal before. Yesterday and today I've been doing some major work on it because I've had need to use it. That's how development on this project seems to go: it sits dormant until I need to use it, and then out of frustration I have to fix things I find wrong with it.</p><p>My bot, for people who have never seen me mention it before, is an automated MediaWiki client program that operates by applying "transforms" to given pages. A transform is perl subroutine that takes text in as a parameter and returns a modified version of that text.</p><p> &nbsp; &nbsp; $newtext = transform($oldtext);</p><p>The transform itself is written in every day Perl 5 code, not any kind of fancy DSL. You've got the full power of Perl 5 applied to the text of every page in a given list. You can store multiple named transforms together into a queue and apply them all in order to the page text too.</p><p>One of the key things I added today is syntax highlighting and line numbering through the Gtk::SourceView widget. I've also done a lot of refactoring internally to try and add new languages to the transform engine eventually. The eventual goal is to be able to write transforms in other languages besides Perl 5. I'm thinking the most direct way to do that is by interfacing with Parrot::Embed, but there are plenty of other modules on CPAN too that add interfaces to other languages that could easily be used, as soon as I come up with a good plugin architecture.</p><p>The one big thing holding me back right now is adding an installer. Frankly, it's the area of perl development that I'm the worst at, and I've been unable to put together an installer for this application and the various component libraries it relies on. Documentation in this area hasn't been great in my experience, or if there is good documentation I've been unable to find it. As soon as I do get an install script working for it, I'll be able to cut the 1.0 release.</p><p>With all this work I've been doing on the bot, I haven't had much time to work on the <a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6</a> book this weekend. I've got a few other projects I'm poking around on, I'm sure I'll talk about them later.</p> Whiteknight 2009-02-08T19:25:48+00:00 journal Looking at the GC http://use.perl.org/~Whiteknight/journal/38402?from=rss <p>A few problems have come up lately dealing with the mess that is src/gc/system.c (formerly src/cpu_dep.c). This is the file that handles the system-dependent parts of the GC mark phase. This is where the processor registers are stored on the stack and the stack is walked to extract pointers to PObjs for marking.</p><p>There are a number of interesting things in this file, and I've blogged about some of them in the past when I was doing my GC work over the summer. One interesting part, which is turning out to be causing build problems now, is the way the processor registers are retrieved on SPARC systems. Current implementation is to use hand-assembled machine code words, put them into a manually-aligned array on the stack, and execute it. Just imagine all the things that could go wrong with that! Infinoid++ sent me <a href="http://www.sics.se/~psm/sparcstack.html">this link</a> that explains how Sparc works in more detail. Fun read!</p><p>I've also been looking closely at the <a href="http://www.hpl.hp.com/personal/Hans_Boehm/gc/">Boehm-Demers-Weiser collector</a>. It isn't going to be quite as simple as a direct drop-in replacement for malloc in Parrot, I would need to create a dummy GC core to handle some of the expected bookkeeping, but it's something I definitely want to try out in the next few days. I'm also not sure if I'd be able to tune it to handle timely destruction, but it's worth a shot.</p> Whiteknight 2009-02-03T19:27:26+00:00 journal Writing during the Superbowl http://use.perl.org/~Whiteknight/journal/38389?from=rss <p>Like so many other Philadelphia fans, I'd much rather do anything else besides watch two other teams. So, tonight I'm working on some book stuff.</p><p>I've been testing many of my code examples on IRC using some of the creative and helpful bots that inhabit the various channels. You type in a line of Perl 6 code, and the bots will execute them and return the results. This is very handy for most cases, but the limitations in the various incomplete implementations become readily apparent.</p><p>I was trying to introduce the notion of dynamic scope at one point, and tried to test the keywords CALLER and OUTER. Of course, neither of these are implemented yet in Rakudo, so I quickly moved on to other topics. Today I started work talking about code blocks, closures, and pointy blocks. I was in the channel testing some things out and got a lot of help from moritz++. He reviewed some of my work and pointed out some areas where I was wrong, and where I had been right but the spec has now changed to make it wrong.</p><p>moritz also corrected some of my misunderstandings and non-best-practices concerning conditionals and pointy blocks. I don't just want to teach this material to new readers, I want to teach it <i>right</i>.</p><p>So far I've drafted material on quite a few topics. I welcome any feedback people have about any of them:</p><p>1) variables, sigils, context, types<br>2) basic arithmetic and string operators<br>3) if/unless conditionals and loops<br>4) subroutines, blocks, closures, and pointy-blocks<br>5) classes, objects, and methods</p><p>Next I'm going to get started on a basic treatment of Perl 6 regular expressions and grammars. I'll be deferring discussion of the Perl 5 RE syntax to the <a href="http://en.wikibooks.org/wiki/Perl_Programming">Perl 5 Wikibook</a>, and will probably be borrowing some of the grammar discussion from the PGE stuff I wrote in the <a href="http://en.wikibooks.org/wiki/Parrot_Virtual_Machine">Parrot book</a>. Material from all these places will be migrated to the Perl 6 Essentials book in the pugs repo, and the Parrot book in the Parrot repo too.</p><p>I've heard a few complaints about having too many books, but I personally think it's a good thing. More books in more places can attract more readers and more writers. You can approach the same material in multiple ways, which is something I try to make conscious effort to do. Maybe the effort is spread a little bit thin at first, but it's the long-term viability of these projects that I'm most concerned with. Time will tell if my ideas here pan out.</p><p><a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6 book at Wikibooks</a></p> Whiteknight 2009-02-02T02:00:33+00:00 journal VTABLE invoke http://use.perl.org/~Whiteknight/journal/38360?from=rss <p>kjs put in a great little commit the other day that resolved an inconsisency in the way VTABLE overrides were handled in PIR.</p><p>When you define a class in PIR, you get the ability to override the default VTABLE interfaces for that class. These overrides should, in theory, have access to all the same facilities that their C-written breathren have. One of these features is the "self" keyword, a magical variable that contains a reference to the PMC object that the VTABLE override is being invoked on. Before kjs' commit, the "self" parameter was only passed in some situations. Now, it's always passed.</p><p>For a large number of reasons, this causes a failure in the handling of the "invoke" VTABLE method, which is a special case. People want to be able to invoke a PMC class like this:</p><p> &nbsp; &nbsp; $P0()</p><p>However, this raises a number of problems: We don't know what type of object $P0 is until runtime, so IMCC (or PIRC) cannot know to generate an extra instruction to pass an extra invocant parameter at compile-time. With kjs' patch, calling the statement above causes an error in the overridden invoke VTABLE: one parameter is expected ("self"), but none have been passed.</p><p>There are two ways to get around this currently:</p><p> &nbsp; &nbsp; $P0($P0) # pass "self" explicitly<br> &nbsp; &nbsp; $P0.$P0() # IMCC thinks it's a<nobr> <wbr></nobr>:method</p><p>I was thinking that either of these could be easily macroized away behind a compiler directive:</p><p><nobr> <wbr></nobr>.pmcinvoke($P0 [,args]) # $P0.$P0(args)</p><p>or:</p><p><nobr> <wbr></nobr>.pmcinvoke($P0 [,args]) # $P0($P0, args)</p><p>However, this is messy too because every time we want to invoke a custom object we need to write all this nonsense. Remember, what we want to be able to write is this:</p><p> &nbsp; &nbsp; $P0()</p><p>Instead of changing things on the caller's side, which we know we can't really do, we can consider changing things at the callees side instead. As things are, the invocant is passed like a normal parameter, except at the front of the list. What if, instead of passing it implicitly at the beginning of the list, we pass it through some other internal mechanism? In fact, we have an opcode that can do just that:</p><p> &nbsp; &nbsp; $P0 = interpinfo<nobr> <wbr></nobr>.INTERPINFO_CURRENT_OBJECT</p><p>if IMCC generated that op at the beginning of every<nobr> <wbr></nobr>:vtable override, we don't need to worry about passing the invocant and then VTABLE_invoke will just work.</p><p>This is maybe even more heavyweight then we need. If "self" in PIR were just an alias for the interpreter field "interp-&gt;current_object" (maybe a read-only alias for it) it could just exist everywhere (and would only be non-null in<nobr> <wbr></nobr>:method and<nobr> <wbr></nobr>:vtable subroutines)</p><p>In any case, this is a complicated situation and there are no easy resolutions in sight. We may just have to stick with the "$P0.$P0()" syntax that kjs suggests. It's not terrible, but I do think it can be better.</p> Whiteknight 2009-01-29T00:23:21+00:00 journal Perl 6 Release Date http://use.perl.org/~Whiteknight/journal/38344?from=rss <p>While working on the Perl 6 book I find myself referencing the Synopses quite frequently. I gave up on browser bookmarks a long time ago, so I usually rely on google to bring the synopses to me when I want to see them. Every time I look, Google suggests several other popular search terms to me. Here are the popular search terms that Google says other people are trying to find starting with "Perl 6":</p><p>Perl 6 release data<br>Perl 6 news<br>Perl 6 tutorial<br>Perl 6.0<br>Perl 6 status<br>Perl 6 download<br>Perl 6 release<br>Perl 6 regex<br>Perl 6 wikipedia<br>Perl 6 vaporware</p><p>Not exactly an encouraging list of common search terms! Part of me even wonders what kinds of results google will send you for "Perl 6 vaporware"? I doubt you'll find anything in there that you weren't expecting to find in the first place though.</p><p>Perl 6 is very real, and it's making amazing progress every day. There's also a <a href="http://en.wikibooks.org/wiki/Perl_6_Programming">nice book about it</a> that's coming together nicely for all those people searching for "Perl 6 tutorial". </p><p>I've seen the development velocity of the Rakudo project first hand, and I can't possibly believe we're going to have to wait till Christmas for this thing to reach "completion". However, I have joined the Perl 6 language mailing list, and I can tell you that things are still changing and the spec is hardly set in stone right now. I certainly hope Zeno wasn't right about Achilles never catching that tortoise though, because Achilles in this case is moving pretty damn fast.</p> Whiteknight 2009-01-26T00:05:11+00:00 journal Perl 6: Covering the basics http://use.perl.org/~Whiteknight/journal/38332?from=rss <p>Work on the Perl 6 book at Wikibooks is going smoothly. I haven't had as much time to work on it this week as I would like (and Hiveminder won't shut up about it!), but things are progressing well. Some status:</p><p>1) I've drafted, or at least outlined, almost all of the chapters about "basic" features so far. This covers variables and data types, branching, looping, subroutines, classes and objects.<br>2) I still find myself slipping into "Perl 5 mode" sometimes when I write, so there's some serious proofreading required for some of the code examples.<br>3) MediaWiki is using the GeSHi syntax highligher, which has support for "perl", but not "perl6". If we have a perl hacker somewhere who is familiar with GeSHi and is willing to add highlighting for this language, that would be awesome (and make the book look nicer).</p><p>There is some more "basic" information I need to cover before I move onto some of the more advanced topics, and I definitely need to do more reading in the Synopses before I try to tackle things more comprehensively. Feedback is always appreciated!</p><p><a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6 Programming at Wikibooks</a></p> Whiteknight 2009-01-23T16:15:18+00:00 journal Perl 6 Book: Get It Started http://use.perl.org/~Whiteknight/journal/38307?from=rss <p>The Perl 6 book at Wikibooks has begun. This morning I threw down drafts discussing some basic ideas about variables, sigils, and contexts. As much as I wish I could have run Synopses 2 through some kind of POD2Wikitext translator and just posted that.</p><p>I was realizing as I read through S02 this morning that it's really quite a complex document and is pretty inaccessible to people who aren't familiar already with Perl and programming ideas in general.</p><p>The book however really needs to be accessible to all audiences, including people who maybe haven't programmed Perl 5 before, or people who haven't programmed in a dynamic language before, or even people who have never programmed at all. We have to start off with baby steps introducing little terms and little ideas first and then slowly building up to the bigger ideas.</p><p>Writing this book is not going to be a fast process, but I have a handful of pages drafted up so far and would love to get some feedback on them.</p><p><a href="http://en.wikibooks.org/wiki/Perl_6_Programming">The Perl 6 Book at Wikibooks</a></p> Whiteknight 2009-01-17T17:00:44+00:00 journal Newsy Tidbits http://use.perl.org/~Whiteknight/journal/38288?from=rss <p>A few pieces of news today:</p><p>1) Parrot's SVN repo was scheduled to move from svn.perl.org to svn.parrot.org, but got postponed until after the 0.9.0 release on Tuesday</p><p>2) Language projects are supposed to be moving out of the Parrot repo by the 1.0.0 release in March. Particle++ and Coke++ have set up a new repo for some of the smaller and older languages at <a href="http://code.google.com/p/squawk/">Googlecode</a>. This is for languages that aren't popular or mature enough to warrant their own projects yet. This is a great opportunity to get involved by helping move languages to the new project, or even finding language projects that you want to adopt and champion.</p><p>3) The <a href="http://en.wikibooks.org/wiki/Perl_6_Programming">Perl 6 Programing</a> book at Wikibooks has been created! It's basically a bare outline at this point, but I'm hoping that it will grow pretty quickly. There's plenty of things to do there, from writing and editing to providing examples or illustrations. If you want to have a hand in improving the Perl 6 documentation, this is a good place to get started.</p><p>That's the news for today, I'll expand on these later if I have time.</p> Whiteknight 2009-01-14T20:45:36+00:00 journal Perl 6 Microgrant http://use.perl.org/~Whiteknight/journal/38275?from=rss <p>I just received word today from Jesse that my Perl 6 microgrant proposal has been accepted! Here's the post where it's officially announced:</p><p><a href="http://use.perl.org/~acme/journal/38274">http://use.perl.org/~acme/journal/38274</a></p><p>Now, there was some back-and-forth between myself, Jesse, and Allison prior to the application being accepted and the general consensus is that work on the books in the Parrot repo (docs/book/) and in the pugs repo (docs/tutorial/) should probably be focused on since they are licensed under the Artistic 2.0 (which is compatible with Parrot and Perl 6) and we will want to have these two be publication-ready when Parrot and Rakudo hit 1.0 respectively.</p><p>So, pursuant to these desires and the letter of the microgrant, here are the things I will be doing to fulfill this grant:</p><p>1) Create a new "Perl 6 Programming" book on Wikibooks. Write lots of content and try to attract some readers/contributors for it.<br>2) Help improve <a href="http://svn.pugscode.org/pugs/docs/tutorial/">pugs/docs/tutorial/</a> by removing old info, expanding it with new info, incorporating details from the test suite and the synopses and the various implementations, and generally trying to turn it into a publishable book about Perl 6.</p><p>Most of the new content I write for one book will be able to be adapted for the other, so there isn't going to be any wasted effort.</p><p>In addition to this work for the microgrant, I will be continuing work on:</p><p>3) The "<a href="http://en.wikibooks.org/wiki/Parrot_Virtual_Machine">Parrot Virtual Machine</a>" book on Wikibooks, which I started and having been updating occasionally.<br>4) The <a href="http://svn.perl.org/parrot/trunk/docs/book/">parrot/docs/book</a>, which I've also been contributing to regularly and have done a lot of updating for in the past few months.</p><p>The end result of all this work is that we're going to have lots of documentation that people can go to for answers. Different books will cover the same material in different ways and they will evolve separately. The real winners will be the readers and new learners of Perl 6, because they are going to have a wealth of resources available to learn from. Running a google search for "Learn Perl 6" should turn up lots of informative results!</p><p>I will (probably) be using this blog to post updates about this work as I go.</p> Whiteknight 2009-01-13T16:19:37+00:00 journal JIT .h files mess http://use.perl.org/~Whiteknight/journal/38249?from=rss <p>I finally got my jit_h_files branch finished and merged back into trunk yesterday. The branch primarily affected src/jit/i386/*, but there were a few other changes as well. Notably, I had to make a change to config/gen/makefiles/root.in to ensure the new<nobr> <wbr></nobr>.c files were actually built properly. I'm a makefile n00b, so I did't get that right the first time.</p><p>Merged in the branch and that created few errors for other people. Luckily, with some diagnostic help from moritz++ I was able to iron them out quickly. No harm, no foul.</p><p>In other news I've been chipping away idly at the book. I've added some inexplicably missing sections for classes, objects, and exceptions. I also added some stuff about jonathan++'s new annotation system. The book still doesn't contain any comprehensive discussion about how classes and objects are actually used in Parrot, so there is still a lot of work to be done about that.</p><p>I'm looking forward to having some time this weekend to dig back into the GC and figure out where that damn string concatination error is coming from. I suspect that a string object is being prematurely recycled which was screwing with the pointers. Hopefully I'll be able to narrow it down and resolve it ASAP.</p> Whiteknight 2009-01-09T14:21:17+00:00 journal MMD distance calculation http://use.perl.org/~Whiteknight/journal/38224?from=rss <p>I had an idea last night about how to redo the MMD distance calculations. It's easy to say that I was...affected by chromatic's post to the list awhile back about the inefficencies of the MMD system.</p><p>The current system loops over the function signature string to create a type tuple array PMC. This tuple is used with the list of possible candidates to calculate manhattan distances. Once all the distances are calculated, the list is sorted according to this distance and the "best" match (with the smallest distance) is returned.</p><p>I had an idea that using a Levinshtein distance on the signature string directly would prevent us the need to create tuples and then calculate a manhattan distance. Actualy, we could probably use a simple Hamming Distance for most cases, since we don't want to match transposed arguments or cases of incorrect argument numbers.</p><p>If I have some free tuits, I might throw together a prototype subclass of the MultiSub PMC to prove the idea works (and hopefully prove that it's a performance gain).</p> Whiteknight 2009-01-06T19:50:15+00:00 journal Busy Weekend in $reallife http://use.perl.org/~Whiteknight/journal/38218?from=rss <p>This weekend has been much more busy in $reallife then I had expected. I haven't been able to work on any of the things I was supposed to.</p><p>I am continuing to rip out parts of the overly-ambitious IT garbage collector, hoping that a smaller and more modest version will eventually build and start passing tests. After that, there are a lot of places where the algorithm needs to be optimized, but that's a separate task entirely (and not one on my short-term tasklist).</p><p>Hopefully I get more done this week to pick up some slack.</p> Whiteknight 2009-01-04T23:58:31+00:00 journal My 2009 Tasklist http://use.perl.org/~Whiteknight/journal/38204?from=rss <p>Instead of some vague self-help resolution for 2009, I'm going to write out a tasklist of things I want to accomplish in Parrot this year. 2009 is going to see a lot of milestones for Parrot including the 1.0 and 1.5 major release milestones, and the buildup to the 2.0 release in 2010. A lot of stuff is going to have to happen in that time.</p><p>1) Get the new damn Garbage Collector working<br>2) Get the 1.0 release ready and out without any major hiccups<br>3) Get working on my Octave compiler, including implementations of multi-dimensional PMC array and sparse numerical array PMCs<br>4) Do more work on the calling conventions refactors with an eye for optimizing some of the critical paths in the subroutine-calling code and unifying the PIR and NCI calling procedures.<br>5) Take a stab at the asynchronous IO system. I know it's not on the board for a while, but it's a subsystem I'm really interested in participating in.<br>6) Finish up with my JIT refactors, at least on the i386 platform (which is the messiest)<br>7) Improve JIT coverage on the amd64 platform<br>8) Keep working on the Parrot book, and get it ready for publication/distribution for the 1.0 release (or shortly thereafter)<br>9) Participate in one of those in-person meetings or conventions, like a Parrot hackathon, or a Parrot developer conference, or YAPC, or something.</p><p>2009 is going to be an eventful year, and I hope I can get even half of these things done.</p> Whiteknight 2009-01-02T14:17:55+00:00 journal Parrot Purge http://use.perl.org/~Whiteknight/journal/38144?from=rss <p>The parrot people are on a rampage tonight, and some amazing work is getting done in a hurry. The emphasis here is not on adding new features, but on removing old and unused ones. The Polymorphic Inline Cache (PIC) code, which I had just been starting to learn about, is being ripped out by Coke. chromatic also suggested that we remove most of the runcores too, since they aren't often tested and some of them (such as the computed goto and predereferenced computed goto cores) are only available to GCC users. Since we are aiming for platform neutrality and stability with the 1.0 release, removing things that don't work everywhere or things that aren't used often is a good idea.</p><p>Software Transactional Memory (STM) is also on the chopping block, being mostly unused and dependent on Parrot's older threading model. The IMS garbage collector and the GMS garbage collector, which I have never heard record of being functional, are also slated to be pulled out.</p><p>I've started working on the new GC again in the pdd09gc_part1 branch. The focus of that branch is to get a trimmed-down version of the tricolor collector I wrote over the summer working. I'm also working on an old nagging ticket to get executable code out of<nobr> <wbr></nobr>.h files, which has turned into a nightmare in the belly of the JIT beast. On the bright side, I'm learning about another subsystem so I'll be able to hack on it competently and write about it in the book without relying on bullshit.</p> Whiteknight 2008-12-24T01:51:38+00:00 journal NCI dispatcher idea http://use.perl.org/~Whiteknight/journal/38128?from=rss <p>So I've been kicking around the idea for a simplified NCI dispatch mechanism for Parrot. The upside is that it would be versatile and you wouldn't need to write out a large list of NCI signatures before the build. The downside is that it would be written in assembly and highly platform dependent. Here's some pseudocode for the C part of the dispatcher:</p><p> &nbsp; void nci_dispatcher(...) {<br> &nbsp; &nbsp; &nbsp; void * args_array = malloc(sizeof(void *) * num_of_args);<br> &nbsp; &nbsp; &nbsp; foreach (args) {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; args_array[i] = args[i++];<br> &nbsp; &nbsp; &nbsp; }<br> &nbsp; &nbsp; &nbsp; ret = _asm_dispatch(c_func_ptr, args_array);<br> &nbsp; }</p><p>Here's the part for the assembly dispatcher:</p><p> &nbsp; __asm_dispatch:<br> &nbsp; &nbsp; &nbsp; pop args_array<br> &nbsp; &nbsp; &nbsp; pop c_func_ptr<br> &nbsp; &nbsp; &nbsp; mov ecx, num_args<br> &nbsp; loop_top:<br> &nbsp; &nbsp; &nbsp; je ecx, 0, end<br> &nbsp; &nbsp; &nbsp; push args_array[i]<br> &nbsp; &nbsp; &nbsp; inc i<br> &nbsp; &nbsp; &nbsp; dec ecx<br> &nbsp; &nbsp; &nbsp; jmp loop_top<br> &nbsp; end:<br> &nbsp; &nbsp; &nbsp; call c_func_ptr<br> &nbsp; &nbsp; &nbsp; pop eax<br> &nbsp; &nbsp; &nbsp; ret</p><p>Now, I'm sure that I've criss-crossed my MASM and GAS syntaxes, and for clarity I definitely didn't properly declare all my variables. Also, I didn't account for floating point return values, which certainly wouldn't be coming from the same place as an INTVAL or pointer return value would. I don't care because it's pseudocode after all.</p><p>The downside to this approach is that the _asm_dispatch function would have to be written in platform-specific ASM code, which would be a nightmare to do with our current config system and our list of supported platforms. We would need a different version of the code for each supported assembler on each platform (GAS and MASM on i386 for instance). It might be required that we pre-assemble the object files and include them in the repository for people who don't have assemblers or don't have the few assemblers that we're supporting.</p><p>Anyway, it's an idea that I had to help simplify things. I think it has some merit, even if it's going to be difficult to implement and maintain. What I'm sure would happen is that we would need to maintain the current NCI system as a fallback anyway for systems where this approach isn't possible or feasible. And since maintaining the two systems isn't a good idea in the long-term, this idea is probably dead on arrival.</p> Whiteknight 2008-12-22T01:52:54+00:00 journal Next steps http://use.perl.org/~Whiteknight/journal/38121?from=rss <p>I posted the release notice for the 0.8.2 Parrot here on Tuesday, and I'm proud to report that the release went off without too much trouble. There were a few issues that arose because we've been moving and changing all the Parrot infrastructure to new servers and new software, so in many ways I was the guinea pig for that. But, since I've never done a release before, it's not like I was expecting it to go any way besides how it went.</p><p>Allison created the pdd09gc_part1 branch today to take the first baby steps in replacing the ailing mark&amp;sweep collector. The plan for this branch is to take a paired down version of the incremental tricolor collector I wrote over the summer and get it working, without all the troublesome frills that I had tried to add back then. Once we get a basic version working as a drop-in replacement for the current collector, we can start tackling some of the more ambitious ideas that we've been thinking about for the past few months.</p><p>The calling_conventions branch is still in limbo, and Allison created a cc_restart branch to try and incrementally fold in the changes that I had made there to help with the debugging effort. The error we're getting is so amazingly difficult to debug, and I have a feeling that it's connected to some of the other problems people have been uncovering recently with contexts and subroutines. Time will tell, but I can say that the whole subroutine calling mechanism we have now is very fragile and needs major improvements.</p><p>These are just the things I'm working on this month, come 2009 I have a bunch of other projects I want to tackle: Improve the C99 language implementation, write that Octave implementation I started working on idly last summer, and I've also had my eye on Parrot's asynchronous I/O system. I know the asynch I/O system isn't slated to be included until 2.0 in 2010, but I think I want to take a stab at it long before that.</p><p>2009 is going to be a great year for Parrot, and I'm looking forward to all the cool things I'm going to try to do with it.</p> Whiteknight 2008-12-19T23:48:07+00:00 journal Parrot 0.8.2 "Feliz Loro" Released! http://use.perl.org/~Whiteknight/journal/38103?from=rss <p>On behalf of the Parrot team, I'm proud to announce Parrot 0.8.2<br>"Feliz Loro." Parrot (http://parrotcode.org/) is a virtual machine aimed<br>at running all dynamic languages.</p><p>Parrot 0.8.2 is available via CPAN (soon), or follow the download<br>instructions at http://parrotcode.org/source.html. For those who would like to develop on<br>Parrot, or help develop Parrot itself, we recommend using Subversion on<br>the source code repository to get the latest and best Parrot code.</p><p>Parrot 0.8.2 News:<br>- Implementation<br> &nbsp; &nbsp; + fixed lexical semantics<br> &nbsp; &nbsp; + added the 'capture_lex' opcode<br> &nbsp; &nbsp; + added automatic resume for nonfatal exceptions<br> &nbsp; &nbsp; + added multidispatch cache<br> &nbsp; &nbsp; + applied miscellaneous performance improvements, including startup time<br> &nbsp; &nbsp; + fixed several bugs and leaks found by Coverity Scan<br> &nbsp; &nbsp; + removed race conditions from parallel testing<br>- Compilers<br> &nbsp; &nbsp; + IMCC<br> &nbsp; &nbsp; &nbsp; &nbsp; - removed undocumented<nobr> <wbr></nobr>.param int =&gt; syntax<br> &nbsp; &nbsp; &nbsp; &nbsp; -<nobr> <wbr></nobr>.line directive now only takes an integer argument<br> &nbsp; &nbsp; &nbsp; &nbsp; - new<nobr> <wbr></nobr>.file directive to specify the file name being compiled<br> &nbsp; &nbsp; + PCT<br> &nbsp; &nbsp; &nbsp; &nbsp; - properly handles lexical generation and closure semantics<br> &nbsp; &nbsp; &nbsp; &nbsp; - uses<nobr> <wbr></nobr>:subid instead of name lookups to reference PAST::Block nodes<br> &nbsp; &nbsp; &nbsp; &nbsp; - added PAST::Control node type (exception handlers)<br> &nbsp; &nbsp; + PGE<br> &nbsp; &nbsp; &nbsp; &nbsp; - add support for and assertions<br> &nbsp; &nbsp; &nbsp; &nbsp; - Match objects use Capture PMC instead of Capture_PIR<br> &nbsp; &nbsp; + PIRC<br> &nbsp; &nbsp; &nbsp; &nbsp; - add macro handling to PASM mode<br> &nbsp; &nbsp; &nbsp; &nbsp; - disable vanilla register allocation in PASM mode, but do allow optimization<br> &nbsp; &nbsp; &nbsp; &nbsp; - add tests and bug fixes<br> &nbsp; &nbsp; &nbsp; &nbsp; - first bits of bytecode generation. No sub calling/returning yet.<br>- Languages<br> &nbsp; &nbsp; + Rakudo<br> &nbsp; &nbsp; &nbsp; &nbsp; - fixed lexical handling and recursion<br> &nbsp; &nbsp; &nbsp; &nbsp; - refactored subtypes implementation<br> &nbsp; &nbsp; &nbsp; &nbsp; - support for quotes with multi-character delimiters<br> &nbsp; &nbsp; &nbsp; &nbsp; - implemented list slices (Positional role)<br> &nbsp; &nbsp; &nbsp; &nbsp; - list assignment<br> &nbsp; &nbsp; &nbsp; &nbsp; - reduction meta operators<br> &nbsp; &nbsp; &nbsp; &nbsp; - hyper meta operators<br> &nbsp; &nbsp; &nbsp; &nbsp; - cross meta operators<br> &nbsp; &nbsp; &nbsp; &nbsp; - more builtin functions<br> &nbsp; &nbsp; &nbsp; &nbsp; - added Nil type<br> &nbsp; &nbsp; &nbsp; &nbsp; - basic support for protos<br> &nbsp; &nbsp; &nbsp; &nbsp; - iterator on filehandle objects<br> &nbsp; &nbsp; &nbsp; &nbsp; - basic support for exception handlers<br> &nbsp; &nbsp; &nbsp; &nbsp; - warn<br> &nbsp; &nbsp; + Lua<br> &nbsp; &nbsp; &nbsp; &nbsp; - added complex &amp; mathx libraries<br> &nbsp; &nbsp; &nbsp; &nbsp; - merged LuaClosure &amp; LuaFunction PMC<br> &nbsp; &nbsp; + Pipp<br> &nbsp; &nbsp; &nbsp; &nbsp; - added support for a return value from user defined functions<br> &nbsp; &nbsp; &nbsp; &nbsp; - added incomplete implemention of 'require_once'<br> &nbsp; &nbsp; + Ecmascript<br> &nbsp; &nbsp; &nbsp; &nbsp; - parser fixes, parses spidermonkey's top level test/shell.js<br>- Deprecations<br> &nbsp; &nbsp; + PARROT_API is now PARROT_EXPORT<br> &nbsp; &nbsp; + PIR<br> &nbsp; &nbsp; &nbsp; &nbsp; -<nobr> <wbr></nobr>:lexid is now<nobr> <wbr></nobr>:subid<br> &nbsp; &nbsp; &nbsp; &nbsp; -<nobr> <wbr></nobr>.arg is now<nobr> <wbr></nobr>.set_arg<br> &nbsp; &nbsp; &nbsp; &nbsp; -<nobr> <wbr></nobr>.result is now<nobr> <wbr></nobr>.get_result<br> &nbsp; &nbsp; &nbsp; &nbsp; -<nobr> <wbr></nobr>.yield (in<nobr> <wbr></nobr>.begin/end_yield) is now<nobr> <wbr></nobr>.set_yield<br> &nbsp; &nbsp; &nbsp; &nbsp; -<nobr> <wbr></nobr>.return (in<nobr> <wbr></nobr>.begin/end_return) is now<nobr> <wbr></nobr>.set_return<br> &nbsp; &nbsp; &nbsp; &nbsp; -<nobr> <wbr></nobr>.namespace x /<nobr> <wbr></nobr>.endnamespace x syntax is removed<br> &nbsp; &nbsp; + Capture_PIR (runtime/parrot/library/Parrot/Capture_PIR.pir)</p><p>Many thanks to all our contributors for making this possible, and our sponsors<br>for supporting this project. Our next scheduled release is 20 January 2009.</p><p>Enjoy!</p> Whiteknight 2008-12-17T02:37:26+00:00 journal The Tuesday Release http://use.perl.org/~Whiteknight/journal/38057?from=rss <p>My first release is about a week away now, and I'm already getting excited about it. Been running through some of the procedure, going through checklists and getting lots of practice. I've never uploaded anything to CPAN before, even though I've had a PAUSE account for a while. What a lousy Perl coder I've been!</p><p>#parrotsketch this afternoon was great, lots of progress being made on everything. I'm particularly impressed by the Perl 6 developers, they're close to hitting the 5000 spectest mark and might break the barrier before the release. I don't want to jinx them or anything, but I do wish them luck.</p><p>The calling_conventions branch is still failing in perplexing ways. If any hackers out there have some tuits and some solid C-foo, I could definitely use a help with it. I would like to get this added to trunk for the release, but if it's a huge problem I'm not going to force it.</p><p>Other then that, the release is on Tuesday so I could use everybody's help testing and testing again. If you've used Parrot in the past, grab a fresh checkout and run some tests on your system. Submitting bug reports is great. Submitting bug fixes is better. We're not doing any kind of a sprint on it, but if the bug queue shrank a little between now and tuesday nobody would get upset about it.</p><p>I'm sure I'll post an update or two between now and the release.</p> Whiteknight 2008-12-10T00:50:19+00:00 journal