Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

doubi (9222)

doubi
  (email not shown publicly)

Journal of doubi (9222)

Thursday August 13, 2009
09:54 PM

New version Wx::WebKit RFC

Submitted for your perusal, as the outcome of my Google Summer of Code efforts hitherto:

http://www.thinkollab.net/socghop/Wx-WebKit-0.01.tar.gz

The first (somewhat) working draft of a shiny new module. It's for putting a browser in your Wx apps... with a few temporary caveats.

First, it doesn't work on Windows, because wxWebKit only compiles with MSVC at the moment. So Strawberry's right out, and my ActivePerl install is GCC-built too. I'm not sure if all AP versions are (I think they used to be MSVC?) but there you go. We're going to start work on porting to GCC / MinGW, but I don't know how long that'll take.

Also, there's a bug that's only in the Gtk layer at the moment which makes it pretty unusable on Linux too wxWebKit git repo if you have the skillz and inclination to apply it yourself (although sneakily, the new build system is in fact waf, not scons.)

Building the wxWebKit binaries was a steep learning curve for me personally; it might be a lot easier for people who know what they're doing, but I've put up a Linux (Ubuntu 9.04) binary for those who want the least possible faffing about (75MB tarball mind):

http://www.thinkollab.com/socghop/wxwebkit_linuxBinary.tar.gz

Now, if anyone's still with me, there are a number of things that I'm still working on / need help with:

  1. General coding style of /Wx/DemoModules/wxWebKit.pm, suitability / readability / completeness of the pod, etc.

  2. Testing on Mac / BSD.

  3. For some reason, when I tried to change sub OnAddressBarEnter to also call SetFocus on the Wx::WebView object, it crashed with the error: "Can't locate object method "SetFocus" via package "Wx::WebView" at blib/lib/Wx/DemoModules/wxWebKit.pm line 275." (you'll see I have the line still there commented out). I went and looked through the .pm and .xs files of a number of controls in the Wx core package and couldn't see anything special or explicit that enabled them to use functions inherited from wxWindow. wxWebView is a public wxWindow as well. Anyone know what I'm missing here?

  4. The general wxWebKit API class is called wxWebView, so that's what I ran through h2xs and that's what I started hacking on at the beginning. I always thought that it'd be more convenient for users of the module to call Wx::WebKit->new, so this morning I tried to change it to that. I changed the package name in WebKit.xs, and every occurrence of 'WebView' I could find in Wx/WebKit.pm and Wx/DemoModules/wxWebKit.pm. test.pl seemed to load fine, but when I input a URL and OnAddressBarEnter eventually called $browser->LoadURL, the app crashed with the error "variable is not of type Wx::WebView at ..." and the line number of the call. I grep'd through my sources and couldn't find 'WebView' outside of the .c and .xs files, as I thought it had to be, since that's the C++ class name it's calling, so I couldn't figure what else to change (I did try changing it in the .xs out of desperation, you can guess how well that worked). When I grep'd for that error message in the Wx sources I found it in four different places in helpers.cpp, at least one of which was very near by something marked 'magic'. I've tried hard to get the inner wisdom of Wx but... it's a jungle in there. If anyone could shed some light on this one too I'd be very grateful.

  5. I haven't figured out what's involved in exposing wxWebView's special events yet. I'm going to look at Wx::ActiveX for how to do that; hopefully I'll understand what's going on, if not I'll hit it with a wrench for a while before giving up and asking here again.

So, if anyone can help with any of the above, I'd be very much obliged. Big thanks to Mattia and the wxperl-users list for all your help so far.

By the way, I've called this a 'new' module, and labelled my effort 0.01. I've made repeated reference to Dan Sugalski's existing CPAN module. I haven't been in direct contact with him about it yet but if my one works on Mac at the minute then it should be as functional as the existing one. Mac users please let me know.

Ta for now!

Saturday August 08, 2009
09:30 PM

Wx::WebKit: Linux imminent, Windows postponed, testing want!

As I mentioned last time, there are lots of little things that are only obvious once you know them, and if you don't it's quite difficult to find the snippet of critical information that'll let you move on. I've come across a few more in the past couple of weeks, for which the attendant error messages were entirely unhelpful.

For instance, mingw32-make won't play nice with (Wx) MakeMaker on Windows, you need dmake. You can't use (C-style?) comments below the package / module declarations in XS files (the error this generates misdirects attention to the next function after the comment). The requirement for __declspec() in function declarations on Windows; this demanded an extra -Ddefine in MakeMaker.PL, which I'd thought was handled internally by wxWidgets. Finally, I was thrown off for a while by the fact that Strawberry expects library files in the format lib.a - sure I can just rename my .libs, but to have to do so on Windows it seems a little counter-intuitive. None of these are things that take any time to fix, but they're quite confounding until someone taps you on the shoulder and tells you your shoe is untied.

perlxstut, well written as it is, leaves a rather steep learning curve for getting to grips with wxPerl, which I feel I have yet to do. I spent some time working off the example of Wx::ActiveX, which was a fascinating process, but eventually outdid me. I'd adapted the following from PlActiveX.h:

wxPliClassInfo wxPliWebKit::ms_classInfo((wxChar *) wxT("wxPliWebKit"),
&wxWebKit::ms_classInfo, NULL, (int) sizeof(wxPliWebKit),
(wxPliGetCallbackObjectFn) wxPliGetSelfForwxPliWebKit);

The only place I could find wxPliGetSelfFor[...] elsewhere was in helpers.h in the main Wx package in the WXPLI_IMPLEMENT_DYNAMIC_CLASS_ macro. My function wasn't playing ball and I thought that macro should have been the key, but I couldn't find it used anywhere in the Wx::ActiveX package... which befuddled me. So I threw out my carefully constructed Wx::Ax mimicry and went back to working off Dan Sugalski's OSX-only Wx::WebKit, tweaked a function call name at random which fixed an &Wx::WebKit::constant is undefined error I'd been getting before, and badda-bing, Wx::WebKit is now working under Linux! :-)

That's the good news. The bad news is that wxWebKit compilation on Windows currently only supports MSVC, meaning that it's currently impossible to create wxWebKit .dlls compatible with Strawberry Perl on Windows. They could in theory work with some ActivePerl installs I suppose if they've been made with Visual Studio, but my own recently acquired version seems to have been built on Vista with GCC, meaning it should be incompatible too.

So what's to be done? Well, wxWebKit is on the verge of transfering to a new version control system (waf) which the maintainer has suggested could make the process of porting it from MSVC to MinGW a much easier process. However, I've never attempted anything like that before; I don't know what's involved (not that I'm not looking forward to learning) and it's probably not feasible for me to do in the last few days of GSoC.

So in the remaining time I propose to get a *nix-happy version finalised and docs written. I should have a version available for download by Monday night PST, which I'll announce here. Help with testing would be very much appreciated, especially for BSD, Mac (I think I can get one to test on this week but I'm not sure) and Windows !~ /XP/.

Monday July 20, 2009
11:20 PM

MinGW wrangling

I finally got wxWidgets to compile on Windows with gdiplus using MinGW. I didn't realise GCC actually looks for libs in a completely different way to MSVC. Had to copy over and edit some headers from the MS platform SDK, since MinGW doesn't come with gdiplus yet.

I don't know what the legal position would be on packaging up the files with the edits I made and making them available for download. The originals are freely available in SDKs from download.microsoft.com... anyone have any advice on this?

I've also read through the majority of perlxstut. I actually spent a long time looking for anything saying explicitly how to use external libraries; all the examples in perldoc I looked at were based on making one's own .c/.h files. Fair enough, it turned out to be delightfully simple, but these things are only obvious when you know the answer (thanks to Zaxo for revealing the elegant truth).

I've also been examining the wxWebKit headers and example program, along with the documentation on the wxWebKit site. It looks fairly manageable. I'd hope to have a demo working within a couple of days (although I'm going to my graduation this week so won't get to work on until the weekend).

The part that still seems tricky is setting everything up to install automatically, like the Alien::wxWidgets currently does. The wxWebKit source is currently hosted at gitorious.org, so I don't know if LWP will be able to pull it down from there easily. I can worry about that later though.

Tuesday July 07, 2009
05:41 PM

sub convert_to_mingw( "WebKit" )

Thanks to a conversation this morning with the wxWebKit maintainer (who doesn't have a homepage, the cad) and the current Strawberry Perl maintainer I now have a much better idea of what needs to be done in the immediate. I think.

On Linux (and I assume Mac) the wxGC option in wxWigets necessary for wxWebKit is enabled by default, meaning a Wx::WebKit module should work out of the box with any default (unicode) wxWidgets pack installed on those platforms. The only issue there is a rendering bugfix in GTK+ which has yet to be rolled out, but should be available in the next few days.

On Windows, wxWebKit can work without wxGC set. I've been examining the Alien::wxWidgets installer and I'm not sure yet if it has the ability to make the necessary amendments to the file /build/msw/setup.h before building from source - I think it should do, and if so adding the ability to specify --enable-graphics_ctx shouldn't be any bother.

The bigger issue on Windows is making wxWebKit, which currently requires MSVC, compilable with MinGW, so that Strawberry users can use it with no fuss. I was completely wrong last week when I said A::wx requires a gcc 'widgets build, as mattia pointed out. It just wasn't finding my MSVC-build libs because it takes its cue for compilers from the running perl's Config.pm.

Work on mingw support on the wx port at least is apparently scanty, and I doubt I have the C++ know-how to do it myself. Hopefully someone will have done some work towards that end on one of the other toolkit ports of WebKit (Qt, Cairo etc). I'd certainly be interested in working on it long term, but as far as the Summer of Code goes I'd say I might have to put up some binaries for people to use in the meantime.

I think the best thing to proceed with now is Wx::WebKit on Linux to show it working, and we can worry about how exactly to make it easy to use on Windows later.

After some frustrating weeks I think everything's starting to come together :-)

Tuesday June 30, 2009
08:44 PM

wxPerl, wxWebKit, Win32 woes

I can't believe I've spent so long just trying to get wxWebKit working on Windows. The build is finishing at the moment, but some JSCore tests are failing and the example app is being colicky about some dlls and how they're loaded. I'm about ready to shove all this for now and go get working on Linux.

This whole process has been delayed by the fact that there is literally one guy who 'does' wxWebKit. Since the wx port was taken off the Apple buildbot, the folks in #webkit, helpful as they are, don't know anything about it. In #wxwidgets too I'm generally referred to this one fellow who knows the project.

All I feel I've contributed through the project so far is revealing wild inaccuracies in the wxWebKit documentation which held me up for days (the instability of the main WebKit trunk for wx, the existence of the git repo and the inability to compile successfully with MSVC2005), as well as fielding some more basic questions on the wxwebkit-users list and on IRC. As I said in a previous post, being new to so much of this meant not wanting to bug people with my problems before being really sure I wasn't just doing something wrong (especially since there was only one maintainer - didn't like the idea of pissing him off!)

Current status is thus:

  • wxWebKit issues
    It builds, but example app won't run. Author doesn't recognise the symptoms at all:
    • App asks for dlls from a version of MSVC different to the one I compiled it with (msvcp80.dll, msvcr80.dll; I'm using MSVC9/2008) .
    • The dlls should in fact be available as they are in a subfolder of the WINDOWS\WinSxS.
    • Copying them into the app's directory by hand just leads to other errors.

    Pursuing two courses of action:

    1. Use detect.exe to find where the wrong msvc version dependency is coming from.
    2. Try to build on another Windows machine.
    3. Leave Windows for now and get it working on Linux (not a positive course of action, really).
  • wxPerl issues
    • Alien::wxWidgets gives the option to use one's own compiled wxWidgets library, but it requires a gcc build of the wxWidgets library. Ironically, the only way I can currently get wxWidgets to build on my Windows machine is using MSVC 2008's nmake.
    • The reason I wanted to use my own wxWidgets compilation was that wxWebKit requires a wxWidgets build with non-standard settings, specifically wxGraphicsContext. Although the Alien::wxWidgets install script supports the specification of some basic options, the vast majority of WxWidgets options, including wxGC, are unsupported in A::wx's automated build process.

    At first I was just annoyed about this, but the wxWebKit maintainer pointed out that in order to gain any traction, wxWebKit ought to be supported by stock wxPerl installs; Perl devs don't want to muck about building their own support libraries when they're used to CPAN doing it for them. This I suppose points to a new deliverable for my project:

    • Patch Alien::wxWidgets (and possibly Wx itself) to improve flexibility of settings for automatic building of wxWidgets.

The new requirement will in effect have me modifying the existing wx modules on CPAN to enable them to make a wxWidgets build I can use to make wxWebKit... seems a little convoluted, but I guess a simple install method was always going to have to be part of the project. I just imagined having wxWebKit built and working before dealing with those kinds of 'details'.

From here I need to start discussing the feasibility of such a modification on wxperl-users. I've stepped through the code and pretty much get the relevant parts, but there might be wider implications or underlying design decisions I'm unaware of.

In other news, in response to my last post tsee kindly brought this to my attention. Looks to be very helpful :-)

Comments about any topics touched upon in this post and how best to move ahead with the project in general are royally welcomed.

Thursday June 18, 2009
04:06 PM

A wrapper for a wrapper for a mad horse

After more than a week of working in earnest, simply trying to get a WinXP build of wxWebKit I can work with, early last Friday morning I finally met with the pasty, unfamiliar apparition of success. I think. Maybe.

I hadn't expected this preliminary stage to take nearly as long as it has (and the sample app still perplexes me). It took me a long time to convince myself that I wasn't making terribly obvious mistakes and seek out help from the wxWebKit maintainer. When I did, I was informed that the wx build is no longer included in the main WebKit buildbot, and recent changes to the trunk frequently break it. I was directed to a git repo for the wxWebKit project that was new as of last month (and therefore wasn't referenced on the wxWebKit site -_- ). The git repo doesn't seem any more smoothly buildable than the trunk though.

I'm now examining Dan Sugalski's OSX-only wxWebKit module on CPAN. It looks like Dan didn't get terribly far with his OSX implentation, and by now it's pretty old. I'm also reading the perlxs & perlguts pages, and looking at some wxPerl example apps with a view to getting a demo working.

I found an old blog entry of Dan's where he bemoans the inability of wxMozilla to keep up with the underlying Mozilla project. This was a problem for him in 2006, and seems to still be the case for wxWebKit today.

Also, I've come across people referencing the future possibilities of wxHTML2, which would provide a a unified API to drop in whatever browser engine one likes, be it WebKit, Mozilla or IE. From the sounds of that GSoC project suggestion, it would either render wxWebKit irrelevant, or of central importance (if it ends up being based off of it). Either way, as far as I know it's not likely to turn up soon, so my project should remain relevant for a while.

I took some solace from the concluding clause of their project spec: "part of the project will be to ensure that downloading and building the web browser code is made as easy as possible, given that resolving dependencies can be challenging." I feel it's something of a validation of the problems I've been having thusfar. Also indicates an important deliverable for me to think about for my project.

In other news, I'm moving home from uni next week, and the eejits at Be (or possibly a housemate's inaccurate specification of dates) have seen my internet cut off early, which makes things a bit more inconvenient. I've been on campus all day downloading everything I might possibly need to work on tonight (mostly wxPerl and updated WebKit sources). Must leave soon to forage.

Friday June 05, 2009
06:46 AM

Project: Cross-platform bindings for wxWebKit

While waiting for the WebKit source tree to check out, I thought it might be good to record here what it is I'm actually working at. The following is the interesting bits from my submission to TPF for GSoC. Any comments, feedback or advice would be welcomed, especially on the relative pros and cons of XS, which would mean my working from Dan Sugalski's Mac-specific module, or SWIG, which I imagine would let me make use of Python's wxWebKit implemenation.

==================================================

Name: Ryan Jendoubi

Email: ryan [dot] jendoubi [] gmail [] com

Project Title: Cross-platform Perl Bindings for wxWebKit

Synopsis

I will create cross-platform Perl bindings for the full functionality of the wxWebKit library, implement the new bindings in an existing Perl project as a proof-of-concept, and provide all necessary documentation and tutorial material to allow people to start using the new components quickly and easily.

Benefits to the Perl/Open Source Community

The WebKit engine is used in all kinds of applications: browsers, media players, web development apps, RSS readers, kiosk apps, IM & email clients, and more [1]. Having a cross-platform browser widget to play with would open up new possibilities to Perl programmers. For example, grappling with separate browser widgets for different platforms took up a lot of unnecessary time for dotReader [2], a cross-platform ebook reader project. With a cross-platform wxWebkit binding, the time taken would have been greatly reduced. Digsby [3], a Python app, shows the potential of using wxWebkit and other wxWidgets via a scripting language. I want to offer those same possibilities to the Perl community.

Further, wxWebkit itself has benefited greatly from the Digsby development effort. I hope that new cross-platform bindings for Perl will also raise interest in the wxWebkit project, so that it may benefit from testing and code contributions from our community.

Finally, the Perl / Open Source community will benefit from integrating a new member with enhanced understanding of GUI development and integrating external libraries with Perl, and the desire to apply my new knowledge in further projects as well as share it in an accessible way to give others an in-road to the community and FLOSS development.

Deliverables

  1. Common Perl bindings for wxWebKit proven to work on at least Linux, Mac and Windows.
  2. A thorough test suite.
  3. Documentation and example code.
  4. A proof-of-concept, integrating WebKit into dotReader [2] using the new bindings, with documentation of the process.
  5. Updating the POD viewer in Padre, the Perl IDE to use WebKit, as an additional use-case with documentation.

Project Details

There exists a Wx-WebKit CPAN module by Dan Sugalski [4] designed only for the Mac, WebKit being the Mac Apple API for WebCore. The first phase of my project will involve examining this module and adapting its concepts to the cross-platform API. I may also examine for reference the existing wxPython bindings for wxWebKit [5], although in contrast to the Wx-WebKit CPAN module these are implemented in SWIG rather than XS.

There are certain WebCore features which have not yet been implemented in wxWebKit, namely plugins, frames, cookies and printing support. Aside from these features then I plan to expose the full functionality of WebCore for Perl programmers.

In addition to standard POD for the new modules, I plan to use my knowledge gained in the proof-of-concept phase to write some introductory documentation for applying wxWebKit to an existing project. I would like to contribute this documentation to the wxPerl pages [6], to make it more easily accessible to newbies and community outsiders.

Project Schedule

  • Documentation to be made continuously throughout the project
  • Now - May 23rd: planning phase
    • Familiarize myself with previous work in the area: WebCore, wxWebKit, the wider wxPerl project, wxPython's WebKit bindings, Dan Sugalski's Wx-WebKit module.
    • Attempt to transfer Mac Wx-WebKit to Linux, use experience to inform course of action from there.
    • Comprehensively map current wxWebKit API to establish extent of required bindings (existing module not updated since 2006).
    • Discussion with mentor and others on how best to proceed.
    • Beginning to work ahead on some coding if possible, as the first three weeks after the official start date overlap with my university exam period.
  • May 25th - June 18th
    • Continue working through up-to-date wxWebKit API specs established in planning phase.
    • Write tests in parallel with new code.
    • Relatively low activity period, but expect to make decent progress.
  • June 18th - July 13th
    • Finish initial implementation.
    • Make sure tests are passing on all systems.
    • Make sure POD is complete and accurate.
  • July 13th - mid August
    • Publicise code available for review by the community.
    • Work new code into dotReader.
    • Work new code into Padre's POD viewer.
    • Write a 'How to' for embedding and interacting with WebKit in a wxPerl application.
    • Complete the project!

References and Likely Mentors

I have discussed this proposal with Eric Wilhelm and Kevin Ollivier, who believed it to be both worthwhile and achievable within the timescale. Their advice has been exceedingly helpful in guiding my planning for the project so far, although neither of them feel able to be the official mentor for the project.

Gabor Szabo has said that the Padre team would be willing to help me update their POD viewer.

Bio

I'm an almost-22-year-old final year undergraduate student, studying Japanese and Politics at the University of Sheffield in South Yorkshire, UK. I've used PHP and Javascript (and AMOS, believe it or not...) which taught me the fundamentals of programming. I've also looked at C++, so I understand some more intricate language features than are commonly found in scripting languages. This experience will be critical in helping me get to grips with XS/SWIG for this project.

I only began to look at Perl in-depth in the past six months, but the character and quality of the Perl community has made a big impression on me. My long-term goal is to create a set of applications for academic collaboration which could be a 'mini-BioPerl' for arts and social science disciplines, and I believe Perl's flexibility and diversity make it the ideal language in which to invest. In taking on this GSoC project I am very much thinking in terms of future uses for the resources I'm going to create. I believe this will give me the drive to see the project finished to the high standards the community demands.

[1] http://trac.webkit.org/wiki/Applications%20using%20WebKit/

[2] http://dotreader.com/

[3] http://www.digsby.com/

[4] http://search.cpan.org/dist/Wx-WebKit//

[5] http://trac.webkit.org/browser/trunk/WebKit/wx/bindings/python/

[6] http://wxperl.sourceforge.net/documentation.html/

[7] http://socghop.appspot.com/document/show/program/google/gsoc2009/faqs#graduate/

Created on: 2009-04-02 09:21:10.122604

Last Modified on: 2009-04-03 08:40:39.942025

==================================================

I didn't anticipate the amount of uni work I'd have for my last few projects and exams, so I'm behind on my schedule. Before getting swamped I spent a few days trying to get WebKit and wxWidgets compiling on WinXP - not having written up my exploits at the time, all I remember now is a strong feeling of frustration. I've uninstalled everything and am starting again, writing up as I go. I'll be putting up a concise and verbose account / how-to when I've got things working. I'm starting on Windows as Kevin Ollivier indicated it would be the trickiest to work with. Hmm, come to think of it, maybe it'd make more sense to get going on Linux first & transfer that knowledge over... oh well.

Thursday May 28, 2009
02:30 PM

Ready, Set... damn.

Last Saturday, being the 23rd of May, marked the start of the real live coding phase of the Google Summer of Code programme. Around the world, hundreds of student coders are getting stuck into their projects, bringing heaps of libré software love to the world.

Apart from me though. Unfortunately, the final two exams of my university career are glowering at me from just beyond the coming weekend. I dunno if North American uni's tend to finish earlier for the summer, but I'm sure there are other participants in the world with lose ends to tie up before being able to dive into their projects properly. I've calculated that despite losing this week and a half, plus a conference I'm presenting at, my graduation and a pre-booked excursion all coming up later, I'll be able to make up the time for the project by working weekends. No-one's gonna be short-changed ;-)

In other news, I noticed that the use of (wx)WebKit is being discussed #padre in the image linked from Alias's latest journal entry. When Gabor Szabo originally asked me to add Padre integration to my deliverables, it seemed like it would just be 'a nice thing to have' - bit of a surprise to see other folk discussing it! Helps fuel the fire though :-)

As much as I'd like to go with the excitement and get started, the only thing I'll be hacking on for the next six days is Japanese vocab.

See you on the other side...