use Perl

Why is PHP so much easier for newbies?

Why does Java have the best IDE tools?

Why is Ruby prettier than Perl?

Why does Perl have the best package repository?

As I've played through Mass Effect 2 over the last few weeks, I see some interesting parallels.

In the Mass Effect universe, human technology is bootstrapped by the discovery of an ancient abandoned alien observation outpost on Mars, and the further discovery that the dwarf planet Charon is really an abandoned but active interstellar jump gate covered in ice.

Other similar species have done the same, resulting in a galactic community of around a dozen civilisations all based around the same basic technological underpinnings.

Despite these civilisations believing a recently (50,000 years) extinct civilisation built the gates, it turns out the technology is perhaps millions of years old.

Every 50,000 years, the synthetic AI race that built them returns from hiding in intergalactic space to wipe out all of the existing advanced species based on "their" technology, and reset the galaxy for the next set of civilisations to rise.

In a conversation between the game's protagonist and one of these old AIs, we are lambasted by the AI for taking the shortcut on their technology. The jump gates and other technology is left in place intentionally, so that each new generation of civilisations take a controlled and predictable development path, making it easier to destroy them.

The AI posits that it is the overcoming of adversity on your own that drives true technological advancement, and that easy routes make you (technologically) weak.

I think you can see something similar in the development of the different programming languages.

Java is long and wordy, taking a long time to type. The need to work around this limitation resulted in the proliferation of powerful IDEs, resulting in the annual 20 million line of code Eclipse release train.

PHP as a web language would have been stillborn if it didn't deal competently and quickly with the need to easily deploy code, the result of which is that you can effortlessly just change .html to .php, add a hello world tag, and upload via FTP as normal (something Perl still can't do well).

Python's need to gain mindshare against an entrenched Perl led to a huge focus on being easy to learn, to a simplification of the language, and to hugely popular things such as the PyGame library and game competitions.

Faced with the lack of truly great package repository, and with a web-heavy community, Ruby became the "prettiest" language. Creating an elegant website is both expected and required if you are going to gain mindshare for an idea.

And Perl's messy syntax and difficulties in the area of maintaining large codebases, combined with a pragmatic sysadmin-heavy community, resulted in an unmatched packaging system that allowed code to be maintained in small pieces, with enormous volumes of support infrastructure around it.

The ease of publishing and trend to smaller package that the CPAN allowed conversely means that the Perl community has never really had the need for pretty and elaborate websites, and the smaller package size means that we lack the giant headline libraries that make the payoff from website work better.

Our bias towards a pragmatic tech-savvy sysadmin userbase means we haven't really provided anything like the focus on learnability that has driven Python's gradual dominance in the mindshare of the young. It takes a certain rigour in your prioritisation to intentionally remove and dumb down existing powerful features so that the language is easier to learn.

Even for Strawberry, which focuses on the userbase with the lowest traditional knowledge, we intentionally have the smallest and most maintainable website possible and we don't even have the kind of introductory screencasts that we really really need (which should be easy but which I never seem to find the time to do).

If you throw a bunch of Perl coders against some PHP coders in a website competition, it is not unexpected that when both sides play to their strengths you will see something like http://geo2gov.com.au/html?location=e.g.+1+Oxford+Street from the Perl coders and something like http://www.hackdays.com/knowwhereyoulive/postcodes/view/2000 from the PHP coders.

The former required a massive amount of data extraction, transformation, aggregation, a gigabyte-sized PostGIS database, and deployment via a Linux virtual appliance to Amazon EC2 to allow for strategic load-shedding.

The latter required the ability to turn data into presentable and understandable information for real humans, and to make it pretty enough that they WANT to look at it.

Driving true technological progress, then, may often be about identifying weaknesses that are hard to solve but aren't completely impossible (and don't have any crippling long-term conceptual flaws at an economic or project-management level).

The three best projects I have driven - PPI, Strawberry, and (in part) Padre - all share this property. All three of these represent hard but not impossible problems, and require an awareness about which issues are intractable and which issues merely exist because there's been no need to solve them any better.

Padre in particular has suffered greatly from issues with Wx quality and threading. But given the low takeup of both threading and Wx it was reasonable to move forward on the basis that these would be fixed once there was something depending on them, and driving a need to fix them.

All of our early problems are gone now, and there is continued pressure to find ways to improve our use of (and the efficiency of) Perl's native ithreads.

Similarly, the creation of Strawberry required a lengthy year-long process of fixing Win32 bugs in all kinds of toolchain and low level modules, because we'd never had a proper working developer feedback loop before.

Similarly, Perl's current push for marketing and blogging and websites is directly resulting from Python's success in mindshare capture.

So my question for you to ponder this week is the following:

What can you see that Perl as a whole struggles to do well, for which a good solution is not impossible, and is only being held back by smaller problems which would go away if there was a working candidate solution put in place that needed those small problem solved.

In the spirit of trying to jam as many speed hacks into Padre as possible, I've finally taken it upon myself to take the awesome work demonstrated in ORLite::Array (which uses ARRAY based objects instead of HASH based objects) and moved it into the ORLite core.

The removal of the need for a hash slice in DBI doubles the speed of SELECT statements, reduces the memory cost of objects, and makes accessors quicker.

I've also integrated support for Class::XSAccessor, so now not only are they faster ARRAY objects (if you want them) the accessors are all XS-accelerated as well.

As a bonus to this I've altered the ORLite::Mirror sub-class to always use ARRAY-based objects by default, which means that all of the ORDB family of modules just instantly got doubled in speed without me needing to do any new releases.

So all up, Padre 0.56 is looking awesome.

To quote one person in the #padre channel, "padre is so fast to start now that the splashscreen is an irritation"

Now that the new resource locking system has been landed for a couple of weeks and some of our worst performance bugs have been resolved, we've been able to uncover and fix a ton of routine performance problems.

Padre 0.55 landed the first pass of these, but the upcoming Padre 0.56 is looking to be incredibly fast and has ditched much of the weight that you expect from an IDE. It's actually starting to feel light like an editor again, instead of heavy like an IDE.

Amoungst the improvements, we have a new tricksy startup mechanism that can apply startup preferences without having to load the user's config file at all (or any of the accompanying weight).

The startup process has become so fast that we're seriously considering ditching the splash screen (because it slows down the startup too much) and open files in an existing Padre has become nearly instantaneous.

Opening files is at least twice as fast thanks to an encoding detection shortcut for simple ascii files. Changing tabs is 5+ times faster due to removal of some filesystem operations and refresh locking. Closing a file is at least twice as fast, and 3-4 times faster if you choose to use the new feature preference to disable Padre's remembering of cursor locations in files.

Closing groups of files "Close This Project" and "Close Other Projects" etc has gained locking and now works incredibly quickly. With the cursor memory off the time between clicking close and the Padre window vanishing is almost instantaneous now.

Finally, we've also added a number of new refresh shortcutting to different tools and widgets, which makes Padre in general much snappier when moving around inside the same project or just doing operations that trigger off refresh cascades within the same file.

The sum total of all these improvements is that Padre feels almost like a whole new editor. It's fresh, snappy, and generally a joy to use.

And even more fancy improvements are in the pipeline. Mattias and Steffen between them have spawned a special new "slave master" threading branch which is already able to save 4-5meg of RAM per background thread by spawning of a master thread very early in the Wx startup process, while our list of loaded modules is very small. As a bonus, because it doesn't have to fork the entire Wx application tree, the slave master branch also fixes the long-time "Leaked Scalar" bug.

0.56 is definitely shaping up as something special and when it comes out I highly commend it to everyone to take a look, especially if you haven't had a look at Padre in the last 10 or so releases.

http://wiki.github.com/frioux/Template-Tiny-js/

Because it works entirely through the creative use of regular expressions, it turns out that it is relatively easy to port as long as you have good enough regex support.

And so that's exactly what fREW Schmidt has done with his Template.Tiny JavaScript port.

Template.Tiny.js contains 600 lines of cloned code to provide an upgrade XRegexp package, and then 150 lines of plain JavaScript to implement the Template::Tiny clone.

Of course, we've already had Jemplate for a while, so it's not like you couldn't use Template Toolkit in JavaScript before.

But this gives you an alternative solution for smaller fragments that doesn't require any server side work at all and allows pure inline client-side templating, a kind of Jemplate.Tiny. :)

While not quite as busy as I would like, the #win32 IRC support channel has so been highly successful. Having fixed most of the obvious flaws in Strawberry uncovered by people in the support channel, things have started to stabilise now.

In this steady state, we are seeing three specific problems surface against and again on a daily basis (for the type of user that clicks on the "Live Chat" link on the front page of a Perl distribution's website).

1. Nobody is there to answer, or not there fast enough.

The pattern we seem to be seeing amoungst the most need users (who don't change their nick from mib_$hexstring to something else) is that they have a typical tolerance for silence of around 3 minutes.

They will join the channel, ask their question, wait three minutes, then leave.

The existing infobot dipsy is almost worse then useless, because about a third of the time it creates the apparent situation where the user thinks that someone IS there to answer their question, but is intentionally ignoring them after the initial greeting.

There's a clear need here for a customer support bot. Something that monitors new arrivals for questions and informs the user that nobody is there right now, and they should lurk for a little bit (and provide information on when the most recent conversation was and how long they should wait).

2. Users expect Perl to come with an IDE.

"I've installed Perl, but I can't find where to edit/run/etc Perl files" is a major complaint. We even get some people that thing that the "CPAN Client" start menu entry is a Perl shell of some kind, then get confused when Perl code they type in doesn't work.

The editor part we plan to fix shortly with Strawberry Professional, but the other half suggests the need for some kind of shortcut to a "Perl Command Line", or at least something LIKE this so at least we can place an obvious-looking Start Menu entry for their perceived need.

I've been playing with Perl::Shell a bit more lately, to turn it into something suitably dumbed down and approachable, so this shouldn't be hard to solve.

3. Can't find Net/LDAP.pm in @INC ( giant wall of text )

The first two problems we can fix at a project and distribution level.

This problem, on the other hand, is a different kettle of fish.

This ordinary Perl error message is an endemic cause of complete failure on the part of the user.

Perl scripts are all over the place, most of them NOT part of a well structured CPAN distribution or PPM package.

The smarter Windows people can survive not having an editor, and they work out how to run the script.

This cryptic error, however, stops them completely in their tracks.

"Can't find Net/LDAP.pm in giant set of paths" (which is the specific case that happened to our Windows admin team in meatspace at work today) actually means "You need to install the distribution perl-ldap, using the PPM tool listed in the start menu entries".

That an error message like this even exists, instead of being at least something like "The Perl class 'Net::LDAP' is not installed on your machine: Can't find Net/LSAP in ..." is the real icing on the cake, because we don't even provide the poor guy running the Perl script with a useful error message that describes the failure in high level terms, let alone provide some suggestion on how to fix it.

For Strawberry Professional, I'm pondering the idea of adding a "Run a Perl Script" start menu program, built using perl and Wx, who's only job is to run a named perl script in a way that turns the cryptic stuff into vaguely useful instructions on what probably caused the problem, and how to fix it (and ideally, to automatically fix it for them).

I end up letting people run Perl by completely preventing them from ever running perl.

But since lots of people don't use the command line (they hit Perl via a batch script or similar) now I might also need to hijack perl.exe entirely to deal with it (except that breaks things that do pass-through code to child Perl's for process isolation reasons).

It's not surprising that Perl has some of the reputation it does, if a Perl coder can't reasonably give a .pl script to an admin and reliably have them be able to run it. Unable to know that they have to install one tiny PPM package, the code becomes "That shitty Perl script".

I posit that there's a whole swath of people that don't hate Perl for it's syntax, since they don't read all kinds of other ugly languages they use day to day.

They have it because running a typical Perl program as an untrained user is hard when it works, and impossible to setup when it is missing dependencies.

But I'm not really sure what, if anything, the core team could do at this point to fix this problem.

With the increasing usage of the xt directory, and with a more robust understanding of the appropriate mechanics behind release tests, I feel the time has come to build support for xt into Module::Install by default.

But before I do so, I'd like to lay out the rules I'll be applying to the implementation for public review and commentary.

-------------------------------------------------------------------

1. The xt directory, if it exists, will automatically be added to the list of tests to be executed if the Makefile.PL is running with either..

1.1. Author mode enabled

1.2. With RELEASE_TESTING true.

1.3. With AUTOMATED_TESTING true.

2. Recursion on the xt directory will be triggered with the same tests_recursive command that triggers recursion on the t directory.

3. The behaviour of the individual test scripts in xt is assumed to be...

3.1. Always run when RELEASE_TESTING is enabled.

3.2. Run during AUTOMATED_TESTING only if the support modules are installed and at a new enough version.

3.3. Run during author mode only if the support modules are installed and at a new enough version. If the test script is unable to M:I's author mode status, the test should be skipped to prevent accidental executing during end user install.

I believe this combination of rules will give us the following scenarios.

Module Authors

Having checked out a module the xt tests will be visible during make test and will run if they can, but if not they will skip without error while telling the author what extra dependencies they need to install.

When building a release tarball, the xt tests will forcefully always run and fail if they are missing dependencies. This will ensure releases are always subject to full Quality Assurance.

CPAN Testers

The xt tests will be included in the list of tests to run, but will only be run when the system already has the appropriate dependencies in place to prevent excessive dependency recursion (which might fail and prevent any testing of the package at all).

End User

During installation, the end user will never see the xt tests displayed on their screen, even if they would otherwise always skip.

-------------------------------------------------------------------

If you have any comments, questions, or suggestions, please reply now.

Assuming no obvious issues arise, my plan is to implement this in the next M:I release some time in the next month.

I've just uploaded a new 0.90_01 dev release of File::HomeDir which contains long-anticipated support for the freedesktop.org specification, meaning that File::HomeDir should now return correct locations for desktop, music, pictures, and user-data directories on modern Unix systems.

If you are interested in these changes, you may want to test the new release.

Please note that this will result in a change in behaviour on many Unix systems, as your backend will automatically switch from the generic File::HomeDir::Unix to the new File::HomeDir::FreeDesktop (based on the existence of the directory /etc/xdg)

In 2008 the Padre project was about building a large development team, with the actual writing of the editor being almost a secondary task (albeit an important one). We knew the actual coding job was utterly immense and too hard for just the few people initially involved to deal with.

A lot of the work on heaviest work on the editor in 2008 was on deep subsystem stuff so that we could more easily recruit more people, like the Config/DB API, the Plugin API, the Locale API, and the basic distribution structure and code layout. And in the process we made enough of the actual editor that people could Understand what we were building.

In 2009 the Padre project took that team building success and built the editor into something we can now honestly call an IDE without our inner pedants cringing.

There are still various problems to solve, lots of rewriting of shitty first implementations to do, and "essential" features to add. An IDE is an Open Problem, and so that is never going to change and there will always be More Things To Do.

But I think it's reasonable to say that we've now built an IDE that the Perl community can call its own, and an IDE that you won't want to leave once you get used to some of the ways in which it excels at Perl development.

In another few months, we should be at something we can reasonable use as a part of the default Strawberry install for the upcoming all-in-one Strawberry Professional distribution.

However, for the most part Padre still lives in a mythical fantasy world where there is only one developer, only one host, and only one desktop. It is still, primarily, a tool for the individual developer. And now we're finally in a position to do something about it.

In 2010 the Padre project will evolve from an IDE into a Collaborative IDE. This isn't something that will happen by fiat. Nothing in Padre happens by fiat, and there is no roadmap.

But if you use Padre on a regular basis, you can see that the place where the most development time is being "wasted" now has now clearly moved from working with Perl code to working with projects, teams, and the internet.

In 2010 you will start to see some of the following:

1. The rise of Policy

One of the biggest themes in Padre is the concept of "Intuition", that your editor should recognise (reliably) what you are doing and adapt to it without having to be told, and in a way that doesn't remove freedom in the process.

Padre already can recognise projects without being told where they are, identify the files and assets in a project without being told where they are, and save files without you saying where. It knows about tabs vs spaces warfare, it knows what language you speak, and it knows how to distinguish things you personally prefer from things you happen to be doing on the current machine.

In 2010 Padre will learn about when it is that the needs of the team outweigh the needs of the individual, and how to selectively apply preferences and intuition differently in different open files, based on the specific preferences of different projects.

You will be able to define, at a project level, rules for your project that Padre will automatically follow, such as spaces vs tabs, perltidy rules, perlcritic preferences, and so on.

This is especially important for making Padre more relevant in corporate environments, where adherence to coding policies and standards are essential.

2. The demise of the host

Even though it hasn't been used to any great effect as yet, Padre has always understood that some configuration is about the user, and some is about the system.

In 2010 Padre will finally start to use that information. A 4th year student team from the University of Western Ontario is currently implementing a synchronisation client/server for configuration data, similar to Mozilla Weave or Google's older bookmark sync.

This will allow all your personal preferences to migrate across multiple installations automatically, without getting mixed up with data that only refers to the machine you are currently working on.

3. The push towards Portable

Another area that I hope to see movement on this year are configuration and GUI improvements to make Padre much more tolerant to changes in the underlying operating system.

While many applications do not support this, I'd like to see Padre automatically adapt to changes in screen position and resolution, mounting and dismounting of drives, and a variety of other crazyness that developers often do in the process of creating applications.

This tolerance and adaptation to screen geometry errors should also allow better configuration of windows and dialog positioning, so that you can more easily tailor the layout of Padre to your particular multi-screen developer setup.

4. The swarming of Collaboration

Finally, the experimental Swarm plugin is (after a long gestation) starting to get closer to producing a usable collaborative communications subsystem that can be used as the basis for chat, file sharing, remote peer review, and other collaborative goodies.

In 2010 Padre should start to take advantage of some of these new capabilities to add new and interesting real-time "Intuitive Collaboration" abilities (which should be quite interesting at conferences and hack-fests).

In summary, it's looking like a great year for Padre, and by the end of the year Padre should hopefully be starting to become the corporate Perl editor of choice.

I take (apparently) an unusual approach to testing.

Apart from regression tests for replicating and fixing reported bugs, I find that my main reason for writing test scripts is to feed the debugger, since I probably spend as much time in the debugger than I do in my editor.

It's just so much more convenient to see what your code is doing, rather than having it tell you (via print statements or logging/tracing etc).

I guess you could call it TDD of a different kind. Test Driven Debugging, rather than Test Driven Development.

However, one of the biggest weaknesses of this debugger-centric development process for me has been string evals and code generation.

Things like ORLite can generate thousands of lines of code, with half a dozen levels of methods nested inside each other. Debugging this is a pain, because unlike regular code the debugger can't show you this eval'ed code.

For this reason, I created Devel::Eval, which exports a 'dval' function that is a (temporary) drop in replacement for eval that writes the code to a temp file, and then does a 'do' of that file. You get the code visible in the debugger, AND you can take a copy of the generated code for later if you want to read through it in more depth.

This has worked great for ORLite, which generates at a package granularity.

But it ran out of steam when I tried to use it on .

After year-long successful development pushes at work on Testing in 2008 and Reliability in 2009, our 2010 development theme is Usability. And I want Aspect-Oriented Programming to play a significant role in this, providing agile hooks into our 250k line application to extract various high volume user-related information (for example, firing UDP syslog messages every single time we try to show a product to a user and it doesn't have a photo, so we can establish a priority of which of our 100,000 product lines need pictures the most).

After encountering some problems with the current implementation, work has given me the low-productivity 3-4 week period over the holidays to rip apart the Aspect.pm internals and rewrite them to bring them up to our required level of functionality and performance.

This is been quite a challenging task, because Aspect is complex in its own right, and it's also built on top of Hook::LexWrap (which does some amazingly crazy/scary things, like using DESTROY-time self-executing blessed code references that modify lexical variables as a way of altering the behaviour of anonymous closures after we've handed them off and don't know where they are any more.

After the first 2 weeks of work, I've managed to drop the Hook::LexWrap dependency and added some specialisation so that optimisable cases work differently (for example, the before { } advice is optimised to use goto internally, while the after { } advice uses Sub::UpLevel instead of Hook::LexWrap's private caller() clone.

But I digress...

The problem with all this, is that the internals now rely on string-eval generated closures that use variables created OUTSIDE the string eval. So the "do $filename" approach is now untenable.

Figuring that the temp file approach was tapped out, I headed to #p5p as my "Perl help channel of last resort" to see if there were any REALLY exotic solutions to my problem.

To my great relief, not only is this problem fixed, it's fixed properly and at the source (and it has been for a while). It's just that someone forgot to document a feature that has been in Perl since 5.8.9/5.10.0!

In the debugger, the $^P variable is a bit mask used to control behaviour and flag which Perl content the debugger should save the content of for later reference.

The perlvar documentation lists 11 bits this variable (0x1 through to 0x400).

But this not true!

There's actually two more undocumented control bits (and a third in 5.10.1), and the first of these (0x800, which is off by default) is exactly what I need.

Adding the 0x800 flag to $^P will cause the debugger to save a copy of everything that it does a string eval on, so that when you later step into the generated code, you can see it in the debugger as you would any other code.

The idiomatic usage of the flag looks like this.

do {
        local $^P = $^P | 0x800;
        eval $perl_string;
};

The new Devel::Eval 0.02 release will continue to use the "do" approach on older Perls, but when it detects that it is on Perl 5.8.9 or later, it will automatically switch to this $^P approach.

Both approaches can be accessed directly via Devel::Eval::fval (for the file approach) and Devel::Eval::pval (for the $^P approach).

http://padre.perlide.org/trac/ticket/1

Our Christmas release of Padre 0.53 is done now, and it's looking really really awesome.

My contribution to this release is a shiny new resource locking system, which has significantly improved the speed of most startup, shutdown and file operations.

A couple of people have asked me to post about how it works, so here's the short and simple version.

First, some background on resources in Padre and Wx.

When managing performance and blocking issues in Wx, the main players are the visibility state (changed via ->Show and ->Hide), the update state (changed via ->Freeze and ->Thaw) and the busy state (changed via various mechanisms).

Visibility is most important during startup and shutdown.

When starting up you want to delay the appearance of the window until the event loop is bootstrapped and you are able to take user events on the window, but you can't wait for things like opening files because this may take quite a long time and cause the editor to appear slow and "bloaty". It's a trade off between performance, and PERCEIVED performance (which is almost as important).

When shutting down, things are a bit clearer. As soon as you are sure that the editor will no longer need to interact with the user in any way, you can proactively ->Hide the window and finish shutting down while invisible (letting the user get on with their next task).

The update state is applicable across the entire lifetime of the application.

If updates are enabled, every action results in a paint event (or will be captured by the next paint event). This creates the appearance of things happening, but changes to the application take longer because of the cost of repainting.

Also, often you don't want to be faster in this way. If you are working with a list box, during the process of deleting all the values and generating a new (slightly different) set of values you don't WANT the user to see the box empty and incrementally refill. It acts a form of "flicker" (one of the great enemies of GUI applications). What you want is for the list box to instantly transition from one filled state to the next filled state creating the impression of incremental change even where the underlying code isn't incremental.

The way you solve this problem in Wx is to explicitly disable repainting via the ->Freeze method as late as possible, quickly make your changes to the GUI structure, and then as quickly as possible re-enable painting via ->Thaw.

This must be fast, because if the users are doing anything they will notice after a quarter to half of a second. Also, after 2-10 seconds in some cases the operating system will start to get concerned about your application. Windows for one will sometimes spontaneously ask Wx to repaint your menu after 5-10 seconds. If painting is disabled, the result is a blank white bar where your menu used to be.

Finally, for Padre specifically, we also need to be concerned about expensive refresh operations. If you open a file, we need to regenerate the directory tree, the function list, outline, menu, toolbar, title, recent files list, and directory list and some of those things (like the directory tree) are expensive.

We need to be sure that we avoid updating elements that don't need to be updated (opening a new file in the currently project context shouldn't result in a project file list refresh, for example) and that we avoid or delay any pointless operations (like updating the GUI during a multi-file open) until after the last file is done.

Unfortunately, the update and refresh issues are problematic when most of your application is event-driven, but all your refresh operations are imperative.

When you first start writing Wx code, you usually do something like this.

    sub open_file {
            my $self = shift;
            my $file = shift;

            $self->Freeze;

            my $panel = $self->create_editor;
            $panel->load_file($file);
            # 10 more lines of setup code here...

            $self->refresh_all;

            $self->Thaw;
    }

This kind of thing works, but it will only work if called directly from the user event and only if it doesn't throw an exception.

What if $panel->load_file errors due to file permissions, and we don't get a chance to call ->Thaw?

Or what if we want to open several files?

sub open_files {
    my $self = shift;
    foreach my $file ( @_ ) {
            $self->open_file($file);
    }
}

In this case, open_files will freeze and thaw repeatedly, slowing down the loading process, and will pointless refresh all the GUI resources after each file.

Or what if we want to open a named group of files (a "Session").

sub open_session {
    my $self = shift;
    my $name = shift;
    my @files = $self->get_session($name);
    $self->close_all;
    $self->open_files(@files);
}

Now we are at two levels of indirection.

Or we want to have a ->next_session method (three), or to support arbitrary scripted "macro" actions contributed by the user (four)?

The real problem with imperative management of these resources is that they don't encapsulate cleanly. You can't nest them inside each other ad infinitum and have code calling your not have to care how you are implemented.

Wx takes care of part of this problem for you, and hints are the right general solution.

    my $guard_object = Wx::WindowUpdateLocker->new($window);


This built in "locker" class generates objects that fire ->Freeze on the parameter window at constructor time, and ->Thaw at destructor time (i.e. when they fall out of scope).

Importantly, these objects also nest properly so that the ->Thaw only occurs on the outermost destructor. This provides the encapsulation we need so badly.

Unfortunately, the ORLite-based SQLite database classes in Padre::DB we rely on quite heavily don't have this ability (and Wx native asynchronous SQLite support isn't available yet).

Worse, our resource refresh logic doesn't follow this pattern at all. Often you can't just "lock" the refreshing of the whole window or the directory list, because you have very specific times that you want the refresh to fire.

For example, if the user has two files open and is switching between one and the other, we want to initially lock updates while we change the editor panel to the new file but we DON'T want to refresh the other tools. Once we release the update lock to show the new file, we know it will take around a second for the user to notice and adjust to what they are seeing. So we want to use THAT time to refresh the other tools, applying another update lock during the changes to prevent screen flicker and avoid distracting the user with unexpected secondary movement.

To resolve all these problems, I've added a new locking API based on a similar guard object principle to the internal Wx one, except that it is able to control several locking states at the same time. More importantly it's also able to understand the interplay between the different lock types.

The canonical usage looks something like this.

    my $lock = Padre::Current->main->lock('DB', 'UPDATE', 'refresh_menu', 'refresh_title');

This indicates that we should "lock" changes to the SQLite database (causing all database operations to take place in a transaction), lock painting updates, and that some time in the future we'll need to do a refresh of the menu structure and the window title.

To simplify and add extensibility to the implementation, the lowercase refresh locks all match directly to methods on the main window class. So anyone can add a new "lock" type just by adding a refresh_something method.

When the lock handle expires, the lock manager follows a specific release plan. First, release the update lock. Second, fire any refresh events that have accumulated. Thirdly, commit any pending database statements and release our database connection (again, taking advantage of the user needing a small amount of time between screen updates and their next action).

Where the value of a custom lock manager comes in is the Padre-specific semantics.

For example, locks on the database and update state don't interact with each other, but refresh method locks persist beyond the scope of the lock if the release occurs within a higher parent lock.

Take the following example.

SCOPE: {
        my $lock1 = $main->lock('UPDATE', 'refresh_menu');
        SCOPE: {
                my $lock2 = $main->lock('refresh_directory');
        }
        SCOPE: {
                my $lock3 = $main->lock('DB');
        }
}

In this example, the database lock will clear at the end of the nested scope, and a commit will occur. However, the 'refresh_directory' method will NOT fire in the nested scope because there are other active related locks in effect in a parent scope. Instead, the 'refresh_directory' will be transfered up to the higher lock.

When $lock1 releases, it will fire BOTH of the refresh events.

This is a basic implementation of the concept (less than 100 lines) but it does work quite nicely. However, it does have some problems remaining to be solved in the next version.

The syntax is still a bit clunky. To fire a refresh event immediately (if you are at the top level) you still need to do the following.

$main->lock('refresh');


This will do an immediate ->refresh, but remains lock-aware so that if you are in a higher level lock it will add refresh to the list of things to fire later. It would be nice to have a cleaner syntax for this case, or do be able to have the ->refresh method itself inherently know if it is in a lock and just shortcut return true.

Another problem is that the refresh methods come in a natural heirachy that the lock manager doesn't know about.

For example, the top level ->refresh will itself call ->refresh_menu, ->refresh_title and so on. But the lock manager isn't aware of that.

If two nested scopes ask for 'refresh' and 'refresh_menu' locks it will always fire both refresh methods, even those doing the first one means we don't need the second one.

And it won't fire the locks in a reliable order either. Even if ->refresh was smart enough to automatically clear the 'refresh_menu' lock, we can't be sure that ->refresh_menu won't fire first, before the higher one.

This problem discourages the creation of finer-grained locks, because it would increase the number of collisions and pointless double/triple/etc refreshing.

By discouraging finer locks we end up using bigger locks more often, resulting in more waste due to pointless refreshing of GUI elements.

Ideally, the locker would contain information on the relationships between refresh events, so that the refresh events can be fired in the most "correct" order, with pointless methods filtered out.