I have nothing against Mouse, but I'd like my app to run with either one of Mo[ou]se. For those applications wanting the lighter side of things, could we please make it a standard to use Any::Moose instead?
Really, I just want to choose which backend to use myself, rather than having to deal with two of them.
Please?
P.S., I've written Module::Install::Any::Moose for this kind of thing.
Last year in a sudden fit of madness, I went and started talking to a publisher in Japan, asking if I could write a book about Perl.
Well, there was a brief moment of no activity, some crazy time trying to move my company to a new office, and then a stormy winter where I kept writing and writing and writing...
and finally, my first book, "Introduction to Modern Perl" is coming out this next week. I got the first printed version today. yey.
For this book, we decided to aim solely on the intermediate level perl programmer. So there's almost no talk about syntax. Lots of stuff about writing classes using Moose, job queues, profiling, debugging, performance tuning, etc. Basically we included pieces of material that we want our colleagues to have when working on a project together.
This is part of the material that we'd like to promote when Japan Perl Association comes alive. We want to teach people how to be efficient using perl. To that end, I think this book's going to be a pretty good intro, and I'm pretty proud I took the opportunity.
The only thing is... it's in Japanese. So... who's writing the English version?
While the enlightened folks (http://www.enlightenedperl.org/) are working steadily towards their goal in the Western world, us folks in the Far East have something of our own that's being cooked up by the community.... and so here I am, pre-announcing the Japan Perl Association.
We're essentially a bunch of folks that want to make sure people know Perl Is Good For Your Business. Like chromatic said (http://www.modernperlbooks.com/mt/2009/01/why-modern-perl.html), we're in 2009, not 1999. But a lot of people, including some major businesses are branding Perl as un-manageable. Those of us who decided to follow the Good Practices know that this isn't true: We know that Perl can be an excellent tool to build large, complex systems and still be extremely easy to manage. But have we done enough to promote Good Perl?
JPA will focus on those issues in Japan. Of course, if EPO can do it for us, that's good too, but there's always the language barrier. So JPA will (in Japanese
Oh, and all this isn't just talk: JPA has already been incorporated. Yes, we have an identity already! We've managed to gather officers from some major Japanese IT players. We're currently discussing funding, targeting the coming fiscal year (Japan's fiscal year traditionally starts on April), and planning some activities for April and May.
We haven't yet come up with a website. We'll be getting one pretty soon-ish.
Also, you should know that, among other things, we will be running YAPC::Asia Tokyo (http://yapc.asia/) from this year on. In 2008, we had a whopping 570 registered users. This year we want to extend this to somewhere around 700.
Also, we're planning to host YAPC::Asia Tokyo some where around September this year, mainly due to the transition of hands. Haven't been to Japan yet? We'll be waiting this autumn!
That's it for now. We'll have more details soon. Please contact me (lestrrat @ twitter, or dmaki on CPAN) if you have any questions.
I've been trying to profile Memcached::libmemcached for a while.
Before I start, I'm not going to claim that I'm a C expert, so take my writings with a grain of salt. Ok?
I've run a few benchmarks on Cache::Memcached::libmemcached/ Memcached::libmemcached against Cache::Memcached::Fast, and have found that get_multi() seems to be consistently slower for Memcached::libmemcached. At this point I'm inclined to think that the bottleneck is within the underlying libmemcached library.
As the rules of optimization says, I wasn't going to do anything before I got a profile output, so I started digging around gprof.
Now, gprof looked much like my beloved Devel::DProf, so I was thinking this would be an easy task.... turns out not.
So the first thing to note: to enable profiling, you must enable -pg on both compilation and linking. in a nutshell, this is what you do:
shell> gcc -pg foo.c
shell>./a.out
# executing./a.out (and a successful call to exit())
# creates a gmon.out file
shell> gprof
So first I created foo.c.
#include <libmemcached/memcached.h>
#include <stdio.h>
memcached_return
fetch(memcached_st *ptr, memcached_result_st *result, void *context)
{
fprintf(stderr,
"%s => %s\n",
memcached_result_key_value(result),
memcached_result_value(result)
);
}
int
main(int argc, char **argv)
{
int i;
memcached_st *ptr;
char *keys[] = { "a", "b", "c", "d", "e", "f" };
size_t key_length[] = { 1, 1, 1, 1, 1, 1 };
uint32_t flags = 0;
memcached_return error = 0;
memcached_return (*callbacks[])(memcached_st *ptr, memcached_result_st *result, void *context) = { fetch };
ptr = memcached_create(NULL);
memcached_server_add(ptr, "localhost", 11211);
for(i = 0; i < 6; i++) {
memcached_set(ptr, keys[i], key_length[i], "1", 1, (time_t) NULL, (uint32_t) NULL);
}
for(i = 0; i < 100000; i++) {
memcached_mget(ptr, keys, key_length, 6);
memcached_fetch_execute(ptr, callbacks, NULL, 1);
}
memcached_free(ptr);
}
k. now, for my needs, I need to enable this on both libmemcached and my sample test file. So this is what I do:
shell>
./configure --enable-pic CFLAGS=-pg LDFLAGS=-pg
shell> make
shell> make install
and then I compile/link with my foo.c
shell> gcc -pg -lmemcached foo.c
and now gprof:
shell> gprof -b
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls Ts/call Ts/call name
57.14 0.04 0.04 fetch
42.86 0.07 0.03 main
0.00 0.07 0.00 800009 0.00 0.00 data_start
Call graph
granularity: each sample hit covers 4 byte(s) for 14.29% of 0.07 seconds
index % time self children called name
<spontaneous>
[1] 57.1 0.04 0.00 fetch [1]
0.00 0.00 600000/800009 data_start [3]
-----------------------------------------------
<spontaneous>
[2] 42.9 0.03 0.00 main [2]
0.00 0.00 200009/800009 data_start [3]
-----------------------------------------------
0.00 0.00 200009/800009 main [2]
0.00 0.00 600000/800009 fetch [1]
[3] 0.0 0.00 0.00 800009 data_start [3]
-----------------------------------------------
Index by function name
[3] data_start [1] fetch [2] main
Uh, what? where are my calls to memcached_* functions? Naturally, I try googling for gprof with dynamic linking, etc. But this didn't turn up much.
I fought with this for a while (actually, for like 2 months, on and off), and this morning I just had the thought to just link the objects statically. god, why didn't I think of this before?
So I've now moved back to where my
shell> gcc -pg foo.c *.o
shell>./a.out
shell> gprof -b
And guess what, I finally got my gprof!!! aaaaahhh bliss. I can finally start digging. At first flance, it looks like there are too many reads for the 100K mget() with 6 keys I'm doing (the profiling output reads 8.9 million reads). I haven't done any work just yet. will see what I can find.
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls us/call us/call name
37.02 0.48 0.48 8900049 0.05 0.05 memcached_io_read
17.56 0.71 0.23 700006 0.33 1.24 memcached_response
10.69 0.85 0.14 1400019 0.10 0.12 memcached_io_write
8.40 0.96 0.11 600000 0.18 0.31 value_fetch
4.58 1.02 0.06 100000 0.60 2.50 memcached_mget_by_key
4.58 1.08 0.06 __i686.get_pc_thunk.bx
3.82 1.14 0.05 600000 0.08 0.08 fetch
3.05 1.18 0.04 100000 0.40 9.60 memcached_fetch_execute
2.29 1.21 0.03 600001 0.05 0.05 memcached_string_check
2.29 1.24 0.03 100007 0.30 0.30 io_flush
1.53 1.25 0.02 600006 0.03 0.03 memcached_generate_hash
1.53 1.27 0.02 hash_crc32
1.15 1.29 0.01 600000 0.03 0.03 memcached_result_reset
0.76 1.30 0.01 main
0.38 1.30 0.01 io_wait
0.38 1.31 0.01 memcached_result_set_value
0.00 1.31 0.00 600000 0.00 0.00 memcached_result_value
0.00 1.31 0.00 600000 0.00 0.00 memcached_string_reset
0.00 1.31 0.00 100007 0.00 0.00 memcached_connect
0.00 1.31 0.00 100007 0.00 0.00 network_connect
0.00 1.31 0.00 100000 0.00 2.50 memcached_mget
0.00 1.31 0.00 7 0.00 0.12 memcached_do
0.00 1.31 0.00 6 0.00 1.64 memcached_send
0.00 1.31 0.00 6 0.00 1.64 memcached_set
0.00 1.31 0.00 6 0.00 0.00 storage_op_string
0.00 1.31 0.00 1 0.00 0.00 host_reset
0.00 1.31 0.00 1 0.00 0.05 memcached_create
0.00 1.31 0.00 1 0.00 0.18 memcached_free
0.00 1.31 0.00 1 0.00 0.00 memcached_io_close
0.00 1.31 0.00 1 0.00 0.18 memcached_quit
0.00 1.31 0.00 1 0.00 0.18 memcached_quit_server
0.00 1.31 0.00 1 0.00 0.05 memcached_result_create
0.00 1.31 0.00 1 0.00 0.00 memcached_result_free
0.00 1.31 0.00 1 0.00 0.00 memcached_server_add
0.00 1.31 0.00 1 0.00 0.05 memcached_string_create
0.00 1.31 0.00 1 0.00 0.00 memcached_string_free
0.00 1.31 0.00 1 0.00 0.00 rebalance_wheel
0.00 1.31 0.00 1 0.00 0.00 server_add
0.00 1.31 0.00 1 0.00 0.00 server_list_free
0.00 1.31 0.00 1 0.00 0.00 set_hostinfo
0.00 1.31 0.00 1 0.00 0.00 set_socket_options
Call graph
granularity: each sample hit covers 4 byte(s) for 0.76% of 1.31 seconds
index % time self children called name
<spontaneous>
[1] 93.1 0.01 1.21 main [1]
0.04 0.92 100000/100000 memcached_fetch_execute [2]
0.00 0.25 100000/100000 memcached_mget [5]
0.00 0.00 6/6 memcached_set [19]
0.00 0.00 1/1 memcached_free [21]
0.00 0.00 1/1 memcached_create [24]
0.00 0.00 1/1 memcached_server_add [35]
-----------------------------------------------
0.04 0.92 100000/100000 main [1]
[2] 73.3 0.04 0.92 100000 memcached_fetch_execute [2]
0.23 0.64 700000/700006 memcached_response [3]
0.05 0.00 600000/600000 fetch [10]
-----------------------------------------------
0.00 0.00 6/700006 memcached_send [18]
0.23 0.64 700000/700006 memcached_fetch_execute [2]
[3] 66.4 0.23 0.64 700006 memcached_response [3]
0.45 0.00 8300048/8900049 memcached_io_read [4]
0.11 0.08 600000/600000 value_fetch [7]
-----------------------------------------------
0.00 0.00 1/8900049 memcached_quit_server [23]
0.03 0.00 600000/8900049 value_fetch [7]
0.45 0.00 8300048/8900049 memcached_response [3]
[4] 37.0 0.48 0.00 8900049 memcached_io_read [4]
-----------------------------------------------
0.00 0.25 100000/100000 main [1]
[5] 19.1 0.00 0.25 100000 memcached_mget [5]
0.06 0.19 100000/100000 memcached_mget_by_key [6]
-----------------------------------------------
0.06 0.19 100000/100000 memcached_mget [5]
[6] 19.1 0.06 0.19 100000 memcached_mget_by_key [6]
0.14 0.03 1400000/1400019 memcached_io_write [8]
0.02 0.00 600000/600006 memcached_generate_hash [13]
0.00 0.00 100000/100007 memcached_connect [29]
-----------------------------------------------
0.11 0.08 600000/600000 memcached_response [3]
[7] 14.3 0.11 0.08 600000 value_fetch [7]
0.03 0.00 600000/8900049 memcached_io_read [4]
0.03 0.00 600000/600001 memcached_string_check [11]
0.01 0.00 600000/600000 memcached_result_reset [15]
-----------------------------------------------
0.00 0.00 7/1400019 memcached_do [20]
0.00 0.00 12/1400019 memcached_send [18]
0.14 0.03 1400000/1400019 memcached_mget_by_key [6]
[8] 13.0 0.14 0.03 1400019 memcached_io_write [8]
0.03 0.00 100007/100007 io_flush [12]
-----------------------------------------------
<spontaneous>
[9] 4.6 0.06 0.00 __i686.get_pc_thunk.bx [9]
-----------------------------------------------
0.05 0.00 600000/600000 memcached_fetch_execute [2]
[10] 3.8 0.05 0.00 600000 fetch [10]
0.00 0.00 600000/600000 memcached_result_value [27]
-----------------------------------------------
0.00 0.00 1/600001 memcached_string_create [26]
0.03 0.00 600000/600001 value_fetch [7]
[11] 2.3 0.03 0.00 600001 memcached_string_check [11]
-----------------------------------------------
0.03 0.00 100007/100007 memcached_io_write [8]
[12] 2.3 0.03 0.00 100007 io_flush [12]
-----------------------------------------------
0.00 0.00 6/600006 memcached_send [18]
0.02 0.00 600000/600006 memcached_mget_by_key [6]
[13] 1.5 0.02 0.00 600006 memcached_generate_hash [13]
-----------------------------------------------
<spontaneous>
[14] 1.5 0.02 0.00 hash_crc32 [14]
-----------------------------------------------
0.01 0.00 600000/600000 value_fetch [7]
[15] 1.1 0.01 0.00 600000 memcached_result_reset [15]
0.00 0.00 600000/600000 memcached_string_reset [28]
-----------------------------------------------
<spontaneous>
[16] 0.4 0.01 0.00 io_wait [16]
-----------------------------------------------
<spontaneous>
[17] 0.4 0.01 0.00 memcached_result_set_value [17]
-----------------------------------------------
0.00 0.00 6/6 memcached_set [19]
[18] 0.0 0.00 0.00 6 memcached_send [18]
0.00 0.00 6/700006 memcached_response [3]
0.00 0.00 12/1400019 memcached_io_write [8]
0.00 0.00 6/7 memcached_do [20]
0.00 0.00 6/600006 memcached_generate_hash [13]
0.00 0.00 6/6 storage_op_string [31]
-----------------------------------------------
0.00 0.00 6/6 main [1]
[19] 0.0 0.00 0.00 6 memcached_set [19]
0.00 0.00 6/6 memcached_send [18]
-----------------------------------------------
0.00 0.00 1/7 memcached_quit_server [23]
0.00 0.00 6/7 memcached_send [18]
[20] 0.0 0.00 0.00 7 memcached_do [20]
0.00 0.00 7/1400019 memcached_io_write [8]
0.00 0.00 7/100007 memcached_connect [29]
-----------------------------------------------
0.00 0.00 1/1 main [1]
[21] 0.0 0.00 0.00 1 memcached_free [21]
0.00 0.00 1/1 memcached_quit [22]
0.00 0.00 1/1 server_list_free [39]
0.00 0.00 1/1 memcached_result_free [34]
-----------------------------------------------
0.00 0.00 1/1 memcached_free [21]
[22] 0.0 0.00 0.00 1 memcached_quit [22]
0.00 0.00 1/1 memcached_quit_server [23]
-----------------------------------------------
0.00 0.00 1/1 memcached_quit [22]
[23] 0.0 0.00 0.00 1 memcached_quit_server [23]
0.00 0.00 1/7 memcached_do [20]
0.00 0.00 1/8900049 memcached_io_read [4]
0.00 0.00 1/1 memcached_io_close [33]
-----------------------------------------------
0.00 0.00 1/1 main [1]
[24] 0.0 0.00 0.00 1 memcached_create [24]
0.00 0.00 1/1 memcached_result_create [25]
-----------------------------------------------
0.00 0.00 1/1 memcached_create [24]
[25] 0.0 0.00 0.00 1 memcached_result_create [25]
0.00 0.00 1/1 memcached_string_create [26]
-----------------------------------------------
0.00 0.00 1/1 memcached_result_create [25]
[26] 0.0 0.00 0.00 1 memcached_string_create [26]
0.00 0.00 1/600001 memcached_string_check [11]
-----------------------------------------------
0.00 0.00 600000/600000 fetch [10]
[27] 0.0 0.00 0.00 600000 memcached_result_value [27]
-----------------------------------------------
0.00 0.00 600000/600000 memcached_result_reset [15]
[28] 0.0 0.00 0.00 600000 memcached_string_reset [28]
-----------------------------------------------
0.00 0.00 7/100007 memcached_do [20]
0.00 0.00 100000/100007 memcached_mget_by_key [6]
[29] 0.0 0.00 0.00 100007 memcached_connect [29]
0.00 0.00 100007/100007 network_connect [30]
-----------------------------------------------
0.00 0.00 100007/100007 memcached_connect [29]
[30] 0.0 0.00 0.00 100007 network_connect [30]
0.00 0.00 1/1 set_hostinfo [40]
0.00 0.00 1/1 set_socket_options [41]
-----------------------------------------------
0.00 0.00 6/6 memcached_send [18]
[31] 0.0 0.00 0.00 6 storage_op_string [31]
-----------------------------------------------
0.00 0.00 1/1 server_add [38]
[32] 0.0 0.00 0.00 1 host_reset [32]
-----------------------------------------------
0.00 0.00 1/1 memcached_quit_server [23]
[33] 0.0 0.00 0.00 1 memcached_io_close [33]
-----------------------------------------------
0.00 0.00 1/1 memcached_free [21]
[34] 0.0 0.00 0.00 1 memcached_result_free [34]
0.00 0.00 1/1 memcached_string_free [36]
-----------------------------------------------
0.00 0.00 1/1 main [1]
[35] 0.0 0.00 0.00 1 memcached_server_add [35]
0.00 0.00 1/1 server_add [38]
-----------------------------------------------
0.00 0.00 1/1 memcached_result_free [34]
[36] 0.0 0.00 0.00 1 memcached_string_free [36]
-----------------------------------------------
0.00 0.00 1/1 server_add [38]
[37] 0.0 0.00 0.00 1 rebalance_wheel [37]
-----------------------------------------------
0.00 0.00 1/1 memcached_server_add [35]
[38] 0.0 0.00 0.00 1 server_add [38]
0.00 0.00 1/1 host_reset [32]
0.00 0.00 1/1 rebalance_wheel [37]
-----------------------------------------------
0.00 0.00 1/1 memcached_free [21]
[39] 0.0 0.00 0.00 1 server_list_free [39]
-----------------------------------------------
0.00 0.00 1/1 network_connect [30]
[40] 0.0 0.00 0.00 1 set_hostinfo [40]
-----------------------------------------------
0.00 0.00 1/1 network_connect [30]
[41] 0.0 0.00 0.00 1 set_socket_options [41]
-----------------------------------------------
Index by function name
[9] __i686.get_pc_thunk.bx [4] memcached_io_read [19] memcached_set
[10] fetch [8] memcached_io_write [11] memcached_string_check
[14] hash_crc32 [5] memcached_mget [26] memcached_string_create
[32] host_reset [6] memcached_mget_by_key [36] memcached_string_free
[12] io_flush [22] memcached_quit [28] memcached_string_reset
[16] io_wait [23] memcached_quit_server [30] network_connect
[1] main [3] memcached_response [37] rebalance_wheel
[29] memcached_connect [25] memcached_result_create [38] server_add
[24] memcached_create [34] memcached_result_free [39] server_list_free
[20] memcached_do [15] memcached_result_reset [40] set_hostinfo
[2] memcached_fetch_execute [17] memcached_result_set_value [41] set_socket_options
[21] memcached_free [27] memcached_result_value [31] storage_op_string
[13] memcached_generate_hash [18] memcached_send [7] value_fetch
[33] memcached_io_close [35] memcached_server_add
So I'm a bit late to the party, and only now have noticed that there's something called Spread (http://www.spread.org) from Ask Bjorn Hansen's presentation slides.
The documentation for spread seems to be er, lacking. I can think of ways to use it but first of all? How widely is it being used? What do YOU use it for?
The other thing: Which perl binding do you use? Spread.pm and Spread::Session looks ok, but my immediate response was, oh, why don't I write Spread::libspread (much like Memcached::libmemcached) and make it pure OO to start with? Bad idea?
In any case, I'm interested in knowning what/how Spread is being used.... anybody?
Last weekend, in seeing a blog post (note, it's in Japanese), I realized that while there is a version being developed by Tim Bunce, nobody had written a Cache::Memcached compatible interface to libmemcached.
Ah well there is one now. It's now available on CPAN
I'm really glad I wrote it, libmemcached is blazing fast!
Here's the output of running the benchmark included with the distro on my laptop:
daisuke@beefcake Cache-Memcached-LibMemcached$ perl -Mblib tools/benchmark.pl
==== Benchmark "Simple get() (scalar)" ====
Rate perl_memcahed memcached_fast libmemcached
perl_memcahed 3837/s -- -80% -88%
memcached_fast 18939/s 394% -- -42%
libmemcached 32895/s 757% 74% --
==== Benchmark "Simple get() (w/serialize)" ====
Rate perl_memcahed memcached_fast libmemcached
perl_memcahed 3658/s -- -77% -84%
memcached_fast 16077/s 340% -- -29%
libmemcached 22727/s 521% 41% --
==== Benchmark "Simple get() (w/compression)" ====
Rate perl_memcahed memcached_fast libmemcached
perl_memcahed 3720/s -- -74% -81%
memcached_fast 14535/s 291% -- -27%
libmemcached 19841/s 433% 37% --
==== Benchmark "Simple set() (scalar)" ====
Rate perl_memcahed memcached_fast libmemcached
perl_memcahed 7163/s -- -61% -77%
memcached_fast 18519/s 159% -- -41%
libmemcached 31447/s 339% 70% --
==== Benchmark "Simple set() (w/serialize)" ====
Rate perl_memcahed memcached_fast libmemcached
perl_memcahed 3529/s -- -31% -41%
memcached_fast 5086/s 44% -- -15%
libmemcached 6002/s 70% 18% --
==== Benchmark "Simple set() (w/compress)" ====
Rate perl_memcahed libmemcached memcached_fast
perl_memcahed 6684/s -- -39% -63%
libmemcached 10989/s 64% -- -39%
memcached_fast 17986/s 169% 64% --
Of course, there are some loose ends. First, there's that last benchmark entry (set w/compression) where Cache::Memcached::Fast is faster than libmemcached. I haven't really looked into it. Also, I haven't implemented stat() and the various libmemcached-specific stuff like consistent hashing, mostly because I've run out of tuits for now. Patches and/or comments are welcome. the svn repository is at
http://svn.coderepos.org/share/lang/perl/Cache-Memcached-LibMemcached/trunk
Update: Apparently the author is still around. My emails keep being bounced, though... hmm.
Okay, so I'm heavily heavily using STOMP lately. I'm building systems that rely on POE::Component::MessageQueue, and now I need to implement the client portion in POE.
So I grab POE::Filter::Stomp, and I notice it's broken. I could send in patches, but that module has no tests. That to me calls for a rewrite, really, but it's not my module.
Anyway, I was in a bit of a hurry, so I just re-implemented the module with tests and all (well, they aren't the greatest of tests, but they at least excercise *something*). So I conjured up a tarball and email the author...
and he turns out to be MIA. yikes.
So now I'm going to have to send emails to modules@perl.org, wait for god knows how long until the author is official accepted as MIA...
Argh, no, I can't wait. Fine, I won't upload it to CPAN. But here's my repository:
http://svn.endeworks.jp/svn/perl/POE-Filter-Stomp/trunk
Feel free to bang on it, I'm all up for fixing things -- I really need a stable STOMP infrastructure. Let me know if you use it.
P.S. If the author of POE-Filter-Stomp reads this, please contact me!