Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Wednesday July 16, 2008
04:48 AM

TestNG

[ #36925 ]

If you read up on TestNG, you can see an example of their XML output:

<testng-results>
    <suite name="Suite1">
        <groups>
            <group name="group1">
                <method signature="com.test.TestOne.test2()" name="test2" class="com.test.TestOne"/>
                <method signature="com.test.TestOne.test1()" name="test1" class="com.test.TestOne"/>
            </group>
            <group name="group2">
                <method signature="com.test.TestOne.test2()" name="test2" class="com.test.TestOne"/>
            </group>
        </groups>
        <test name="test1">
            <class name="com.test.TestOne">
                <test-method status="FAIL" signature="test1()" name="test1" duration-ms="0"
                             started-at="2007-05-28T12:14:37Z" description="someDescription2"
                             finished-at="2007-05-28T12:14:37Z">
                    <exception class="java.lang.AssertionError">
                        <short-stacktrace>java.lang.AssertionError
                            ... Removed 22 stack frames
                        </short-stacktrace>
                    </exception>
                </test-method>
                <test-method status="PASS" signature="test2()" name="test2" duration-ms="0"
                             started-at="2007-05-28T12:14:37Z" description="someDescription1"
                             finished-at="2007-05-28T12:14:37Z">
                </test-method>
                <test-method status="PASS" signature="setUp()" name="setUp" is-config="true" duration-ms="15"
                             started-at="2007-05-28T12:14:37Z" finished-at="2007-05-28T12:14:37Z">
                </test-method>
            </class>
        </test>
    </suite>
</testng-results>

Wow. That's breath-takingly ugly. I compare that to the equivalent in TAP with our optional YAML diagnostic syntax (this isn't how the YAML will really look, but it's just to give you an idea):

1..2
not ok 1 - someDescription2
    ---
    signature: test1()
    name: test1
    start: 2007-05-28T12:14:37Z
    end: 2007-05-28T12:14:37Z
    exception:
        class: java.lang.AssertionError
        stacktrace: |
            java.lang.AssertionError
            ... Removed 22 stack frames
ok 2 - someDescription2
    signature: test2()
    name: test2
    start: 2007-05-28T12:14:37Z
    end: 2007-05-28T12:14:37Z
    xtra:
        is-config: true

The beauty of that is a TAP parser only needs to see this:

1..2
not ok 1 - someDescription2
ok 2 - someDescription2

TestNG has one big advantage, though: it's been written. We're still waiting on Test::Builder 2.0 before we can get the YAML. Fortunately Schwern has a grant for that work and hopefully he's making it happen.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Wow. That's breath-takingly ugly.

    Yes it is but I doubt that any Java programmer looks at it in that format. They are more likely to look at it (or JUnit results) as a green/red bar in a test runner applet or via some plugin to Eclipse or Netbeans.

    In fact you will probably be hard-pressed to find a Java programmer who can tell you anything about the JUnit/TestNG formats apart from the fact that they are in XML.

    John.
    --