use Perl

I don't particularly like the way vim's grep works. It's slow, jumps to the first match (you can suppress this) and generally just doesn't behave the way I want it to behave. I want to search, automatically jump to the first match if only one file is found or get a list of files and choose one to edit. I tried egrep, but was getting strange "too many files" errors, so I switched to ack and everything magically worked (I've no idea why).

noremap <silent> <leader>g :call MyGrep("lib/")<cr>
noremap <silent> <leader>G :call MyGrep("lib/ t/ aggtests/ deps_patched/")<cr>
noremap <silent> <leader>f :call MyGrep("lib/", expand('<cword>'))<cr>

function! MyGrep(paths, ...)
    let pattern = a:0 ? a:1 : input("Enter pattern to search for: ")

    if !strlen(pattern)
        return
    endif

    let command = 'ack "' . pattern . '" ' . a:paths .' -l'
    let bufname = bufname("%")
    let result  = filter(split( system(command), "\n" ), 'v:val != "'.bufname.'"')
    let lines   = []
    if !empty(result)
        if 1 == len(result)
            let file  = 1
        else

            " grab all the filenames, skipping the current file
            let lines = [ 'Choose a file to edit:' ]
                \ + map(range(1, len(result)), 'v:val .": ". result[v:val - 1]')
            let file  = inputlist(lines)
        end
        if
            \ ( file > 0 && len(result) > 1 && file < len(lines) )
            \ ||
            \ ( 1 == len(result) && 1 == file )
            execute "edit +1 " . result[ file - 1 ]
            execute "/\\v"  . pattern
        endif
    else
        echomsg("No files found matching pattern:  " . pattern)
    endif
endfunction

The main drawback is that vim and perl regular expressions aren't compatible (I don't have perl integration in this vim). The '\v' switch mitigates some of the pain, but I think a primitive regex transformation tool might alleviate some of the pain.

Update: Now attempts to skip the current file in the buffer. Not really needed, but when you have two files with the thing you're searching for and you're in one, it will automatically jump to the other. I've found this common enough that it seems an obvious use case.

Constantly I'll find myself working with DBIx::Class and I want to see the underlying table structure. So I exit my editor, fire up mysql, sob quietly, and type "show create table $some_table". Now I don't have to -- except for the sobbing part.

Make sure you have filetype plugin on in your .vimrc and in your .vim/ftplugin/perl.vim file add the following code (replacing the variables in the beginning of the function, of course):

noremap T :call ShowCreateTable(expand("<cword>"))<cr>

function! ShowCreateTable(class_segment)

    " replace these values with whatever your system needs
    let dbic_base = "My::Schema::"
    let host      = "localhost"
    let port      = 3306
    let user      = "someuser"
    let pass      = "somepass"
    let db        = "somedatabase"

    let class     = dbic_base . a:class_segment
    let table     = system("perl -M". class ." -e 'print ". class ."->table'")
    let create    =  system(
        \ "mysql -h".host.
        \ " -P"     .port.
        \ " -u"     .user.
        \ " -p"     .pass.
        \ " "       .db.
        \ " -e 'show create table ".table."'"
    \ )
    echo substitute(create, "\\\\n", "\n", "g")
endfunction

Then, when I see stuff like this:

$schema->resultset('MasterBrand')->find_or_create({ ...

I just position my cursor on the MasterBrand word, type 'T' and it automatically shows the "create table" statement.

Of course, if you don't want to create a .vim/ftplugin/perl.vim file (you should, but you don't have to :), then you could drop the function in your .vimrc and add the following:

filetype plugin on
au! FileType perl :noremap T :call ShowCreateTable(expand("<cword>"))<cr>

After a lot of aggressive work improving our test suite performance, adding 50% more tests has tremendously slowed us down.

All tests successful.
Files=48, Tests=14025, 1386 wallclock secs ( 3.06 usr  0.46 sys + 1035.71 cusr 42.77 csys = 1082.00 CPU)
Result: PASS
Number of test programs: 48
Total runtime approximately 23 minutes 5 seconds

Ten slowest tests:
+---------+-----------------------------------+
| Time    | Test                              |
+---------+-----------------------------------+
| 13m 25s | t/acceptance.t                    |
| 5m 36s  | t/aggregate.t                     |
| 0m 36s  | t/standards/use.t                 |
| 0m 24s  | t/system/both/import/log/pager.t  |
| 0m 20s  | t/system/both/import/log/search.t |
| 0m 11s  | t/system/both/import/log/log.t    |
| 0m 11s  | t/unit/db/migrations.t            |
| 0m 11s  | t/unit/fip/record-changes.t       |
| 0m 10s  | t/unit/piptest/pprove/testdb.t    |
| 0m 9s   | t/test_class_tests.t              |
+---------+-----------------------------------+

(Note that the acceptance and aggregate tests both emcompass the vast majority of those 14000 tests and simple aggregation is no longer a win.)

Since we're running at roughly 10 tests a second, I started paying attention to where we can gain wins and it looks like XML::XPath is a likely candidate for performance improvements. In fact, the code has a number of areas where it could be improved substantially. It's not been updated in years and I've contacted Matt Sergeant about having a colleague take it over. This could also be an excellent opportunity to pay attention to its RT queue.

$ dprofpp
Total Elapsed Time = 35.55858 Seconds
  User+System Time = 13.44858 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c  Name
3.32   0.446  0.655  39692   0.0000 0.0000  <anon>:/home/poec01/trunk/deps/lib
                                             /perl5//XML/XPath/XMLParser.pm:63
2.94   0.395  1.373   5544   0.0001 0.0002  XML::XPath::XMLParser::parse_start
2.35   0.316  0.385  20180   0.0000 0.0000  <anon>:/home/poec01/trunk/deps/lib
                                             /perl5//XML/XPath/Node.pm:236
2.04   0.275  3.025  30768   0.0000 0.0001  <anon>:/opt/csw/lib/perl/site_perl
                                             /XML/Parser.pm:187
1.84   0.248  1.229  19680   0.0000 0.0001  XML::XPath::XMLParser::parse_char
1.84   0.247  0.247  48278   0.0000 0.0000  Class::Accessor::Grouped::get_simp
                                             le
1.83   0.246  0.598  20196   0.0000 0.0000  <anon>:/home/poec01/trunk/deps_pat
                                             ched/lib/Test/XML/XPath.pm:67
1.80   0.242  0.337  19252   0.0000 0.0000  Class::Accessor::Grouped::get_inhe
                                             rited
1.50   0.202  0.202  11760   0.0000 0.0000  XML::XPath::XMLParser::_namespace
1.42   0.191  1.113  11136   0.0000 0.0001  XML::XPath::Node::ElementImpl::DES
                                             TROY
1.32   0.178  0.178  11200   0.0000 0.0000  XML::XPath::Node::ElementImpl::get
                                             Attributes
1.28   0.172  0.418   1361   0.0001 0.0003  DBIx::Class::ResultSet::new
1.23   0.166  0.869    549   0.0003 0.0016  base::import
1.22   0.164  0.362  13872   0.0000 0.0000  XML::XPath::Node::ElementImpl::app
                                             endChild
1.20   0.162  3.187     24   0.0068 0.1328  XML::Parser::Expat::ParseString

Just that list above shows that almost 16% of our time is spent in XML::XPath.

Steve Yegge transcribed an excellent talk he gave on dynamic languages. It's incredibly thought-provoking and worth reading all the way through. And for the pseudo-obligatory Perl content, here is one of his comments on marketing:

I mean, like, Perl was a marketing success, right? But it didn't have Sun or Microsoft or somebody hyping it. It had, you know, the guy in the cube next to you saying "Hey, check out this Perl. I know you're using Awk, but Perl's, like, weirder!"

And for the Big Bucket of Fail, a someone named "dibblego" had this to say about Yegge's pre-emptive apology regarding some of his colorful comments:

Steve,
Some of these people are more likely to be offended by your compulsion to pass severely under-qualified comment on the topic; something you have done more than once before. The offense comes about because it is almost deliberately misleading to others who might have the desire to learn and are not in a position to know any better and may mistake your pseudo-scientific nonsense with warranted factual claims.

I say "almost deliberate" because I am more inclined to believe your desire to continue doing this is a result of your ignorance rather than malice.

Here's a tip for you kids: "dibblego" might be right, but we'll never know since he never bothers to back up his assertions. Hell, if you're going to be rude, at least be rude with some meat on, will ya?

On another note, Yegge commented that many issues with Java generics require relatively newer programmers to have at least an implicit understanding of covariance and contravariance. These deal with the substitutability of return types and parameters in OO systems and, and, and ...

Boom!

Here's a tip for language designers: if an oft-used feature of your language requires a deep understanding of comp-sci/math, programmers are going to get it wrong. Here be dragons. It's OK to allow things to require advanced concepts, but not for common code that you hope to spread to the masses.

For extra credit, try to find a clear, concise explanation of covariance and contravariance (and invariance, while you're at it) that your typical Jack in the Box programmer is going to be willing to read, much less understand.

On a side note, I still think it would be good to have a "Just Enough Theory" book for programmers that explains the basics they need to know to prevent the "beat them to death with Knuth" fantasies. I don't expect a new programmer to understand why restricting first order predicate logic to horn clauses is considered (by some) to be a weakness, but they had better know about loose coupling and cohesive functions.

I hate Solaris. I really do.

cache_builder_xml $ ls ../../trunk/conf
../../trunk/conf: No such file or directory
cache_builder_xml $ cd ../../
~ $ ls trunk/conf
cat               log               pips3.yml
database.yml      mad.yml           related_apps.yml

For a party game, needed to repeatedly pick random letters from the (ASCII) alphabet. I had my laptop, so I just did it. A friend, a Java programmer, was amazed. My favorite part was when he said "you're going to type out all 26 letters, aren't you?" Um, no I'm not.

perl -le "print ['a' .. 'z']->[rand(26)] while <STDIN>"

For those not quite getting it, here's the shortest I could think of in Java (without getting to the point of obfuscation and with the caveat that they're functionally equivalent, though internally they handle things in a moderately different way):

import java.io.*;

class Pick {
    public static void main(String[] args) {
        String letters    = "abcdefghijklmnopqrstuvwxyz";
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

        while (true) {
            try {
                br.readLine();
            }
            catch (IOException e) {
                System.out.println("Caught exception trying to readline: " + e);
                System.exit(1);
            }
            int rand = (int)(Math.random() * 26);
            System.out.println(letters.charAt(rand));
        }
    }
}

Of course, then you have to compile it and run it as separate steps.

Can anyone post any common code snippets which express themselves more naturally in Java than Perl? (Libraries which can be readily duplicated in Perl don't count)

Er, no it's not. This brief (and snarky) blog post pretty much sums up my feelings.

I've already been pretty critical of Arc, but pretty much ignored it after that. I've taken another peek and, once again, I'm still terribly impressed. Paul Graham has made arc code smaller for those projects he wants to focus on. He does this with decent libraries (well, decent if you don't mind 'ASCII-only' and layout controlled via HTML tables).

Welcome to the Intarweb, circa 1999.

I probably wouldn't be this harsh, but Graham has spent so long talking about "superior" programming languages that he pretty much had to knock this out of the park or tone his rhetoric down.

It seems tough to find a well-written SQL formatter out there which is both accurate and open-source. I looked at SQL::Tidy as an inspiration, but it's tokenizer is bad and there are plenty of issues with the code. A quick bit of hacking has gotten it to the point where it will take this:

SELECT me.import_id, me.timestamp, DATE_FORMAT ( timestamp, '%Y-%m-%d' ) as day
FROM import me LEFT JOIN import_error import_errors ON ( me.import_id =
import_errors.import_id AND import_errors.type IN (
'X::IMPORTER::VALIDATION::BRANDTITLEMISSING')) WHERE ( ( ( timestamp >=
'2008-05-01T00:00:00' ) AND ( timestamp <= '2008-05-06T13:12:30' ))
) GROUP BY me.import_id

And turn it into this:

SELECT me.import_id, me.timestamp, DATE_FORMAT ( timestamp, '%Y-%m-%d' )
           as day
      FROM import me LEFT
      JOIN import_error import_errors
      ON ( me.import_id = import_errors.import_id
               AND import_errors.type
               IN (
                        'X::IMPORTER::VALIDATION::BRANDTITLEMISSING'
               )
      )
WHERE ( ( ( timestamp >= '2008-05-01T00:00:00' )
               AND ( timestamp <= '2008-05-06T13:12:30' )
             )
           ) GROUP BY me.import_id

That's pretty ugly, but at least it makes it a lot easier to read auto-generated SQL that's all on one line. Still, it's only for quick cleanup. The tokenizer needs a lot of work before we can even think of a reasonable reformatter.

Amusingly, though, I accidentally ran Perl::Tidy over it.

SELECT me . import_id, me . timestamp,
    DATE_FORMAT(timestamp, '%Y-%m-%d')
    as day FROM import me LEFT JOIN import_error import_errors ON(me
        . import_id
        = import_errors
        . import_id AND import_errors
        . type IN('X::IMPORTER::VALIDATION::BRANDTITLEMISSING')) WHERE((
    (timestamp >= '2008-05-01T00:00:00')
    AND(timestamp <= '2008-05-06T13:12:30'))
        ) GROUP BY me
    . import_id

OK, not great, but not significantly worse than the SQL::Tidy.

You see, the great thing about posting vim stuff here is that people (Smylers, Aristotle, etc.) immediately post corrections. I appreciate that :)

Problem: when in the MySQL client, you can type 'edit' and edit your last SQL statement (can I go back further? I thought so, but I can't recall how). However, I want my syntax highlighting, but the filetype isn't set. After a bit of testing, I found out that the filename is always /var/tmp/sql* (on my system). So the following addition to my .vimrc fixes this:

" this is for MySQL's 'edit' command while in the client
au! BufRead,BufNewFile /var/tmp/sql* :call SetMySQL()
function! SetMySQL()
    set ft=sql
    SQLSetType mysql
endfunction