Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • At least not against a good database. (eg Oracle, PostgreSQL, etc - but MySQL would be fine.)

    The problem is that there is a trade-off between time spend preparing a query and query performance. It would not be inappropriate to think of preparing a query as a "compile and optimize this SQL" step. MySQL spends very little energy preparing, and therefore its ability to handle complex queries suffers. Most other databases put a lot of energy into preparing, and so it is very important to try to avoid recomp

    • Actually I am going to use it on a very very busy site ;) SQL preparing does not make much sense for me since our platform will usually work in the context of PostgreSQL PL/Proxy cluster *AND* the query could be dynamic enough to defeat ordinary DBI param binding.

      Actually runtime performance is the reason to choose source-filter solutions in the first place ;)

      Also, the SQL example is, well, merely an example...Filter::QuasiQuote's power reveals in the context of true DSLs ;)

      • My experience says that with a sane design you can run one of the top couple thousand busiest websites on the internet on a handful of webservers, paying only a modest amount of attention to performance of the code on your webservers.

        That same experience says that tiny mistakes in how you handle your database can cause that same site to melt unexpectedly.

        The lesson is to not worry about webserver performance, but be paranoid about database performance. Which means use placeholders properly. If you do it d

        • I must say that all you say is indeed true for an ordinary web application :)

          But unfortunately I can't use prepare+execute in my OpenResty platform in particular. Why? Because it must scale by design to serve lots of apps here in Yahoo! China and Alibaba. So it must be a cluster or something like that.

          The PL/Proxy database server requires frontend queries to be of the following form:

                select xquery('account', 'select * from posts...', 1);

          That is, the user sql query itself must be a dynamic string (by definition), and passed to a PL/Proxy function ("xquery" here).

          Silly me, I don't see how preparing such things in advance would help in terms of performance ;)

          Also, we use PgBouncer extensively. According to the DBD::Pg documentation, "programs such as PgBouncer which cache connections at a low level should not use prepared statements via DBD::Pg".

          Furthermore, in other contexts like importing lots of data into Pg, I've found directly sending multiple insert statements at a time could be *much* faster than first preparing an insert statement, then executing the statement handle to do the actual insertion one by one. And yeah, it's a "special" case :)

          I believe, however, preparing a common-use not-so-tricky SQL query first for many subsequent reuses will indeed be a big win ;) I won't argue against this ;)

          But my project sadly falls out of that category, it seems. Or am I taking things terribly wrong here?

          Thanks!

          • Responding out of order.

            On sending multiple insert statements at once. Yes, that can be a big win because you're cutting down on round trips to the database. Each round trip takes unavoidable resources on the client, server, and network. With network latency typically being the biggest deal. However there is an upper limit to the win from that. A compromise that works fairly well is to prepare a bulk insert that inserts multiple records, thereby bypassing the prepare and reducing round trips. YMMV. B

            • Right, preparing a bunch of insert statements first would be faster :) Merely have to deal with the last few specially :) Thanks for the tip.

              We use PgBouncer at the PL/Proxy level to cache connections to the data nodes. On the FastCGI level, a pre-forked lighttpd is used. Database connection to the PL/Proxy nodes are reused across fastcgi loops in a similar fashion as you described :)

              Well, I don't think the use of PL/Proxy necessarily means loss of relationality. Relational constraints still hold for data r

              • Tell me if a donation could help the open-sourcing decision. I'm btilly, at gmail dot com.