Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

jdavidb (1361)

jdavidb
  (email not shown publicly)
http://voiceofjohn.blogspot.com/
J. David Blackstone has a Bachelor of Science in Computer Science and Engineering and nine years of experience at a wireless telecommunications company, where he learned Perl and never looked back. J. David has an advantage in that he works really hard, he has a passion for writing good software, and he knows many of the world's best Perl programmers.

Journal of jdavidb (1361)

Tuesday December 09, 2003
12:28 PM

Testing the untestable

[ #16243 ]

I want to move to the next level in testing all my code, and writing tests before I code. One thing that's stopping me is I see some very difficult situations to test. How do I test something graphical? For example, I have routines that are generating charts and graphs in PNG format. How do I test these? How would I test a GUI interface for something? Also, how do I test routines with side effects? (i.e., routine prints to the screen rather than or in addition to returning a value).

I understand that writing my tests first and designing with testing in mind will go a long way toward making my code testable. I also understand that experience will go a long way toward teaching me to test anything. But so far I still just can't foresee how to do some of these things (or the ways I can imagine to do it are impractical).

Any sources people could point me to about more complicated testing like this, books, urls, periodicals, or forums, would be immensely appreciated. I need to see what path people took when they came this way before.


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Testing the untestable 6 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.

  • You have several options:

    • Leave those bits untested.
    • Break the problem up into smaller steps. Instead of building a string and printing it from one routine, break it into two routines, one that builds the string and one that prints it. You can test the building one automatically and the printing one by hand if you need to.
    • Use mock objects or packages or subs to test the printing or display routines.
    • Add more accessors to your graphics objects -- test that a certain set of pixels are correct or that th

    • You can test the building one automatically and the printing one by hand if you need to.

      Ugh, no testing by hand, thanks. :) That's why the question "how do I test a routine that prints something" still exists after I design for testing and break the routine up. The other ideas were good, though; thanks.

      --
      J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers

  • Question: How do I test something graphical? For example, I have routines that are generating charts and graphs in PNG format.

    Answer: One way would be to pre-generate "known good" charts and graphs, save them in a test_data directory and, run your routines and see if the output files are the same.

    Question: How would I test a GUI interface for something?

    Answer: I've struggled with this and for the most part, I don't think there are good, generic answers. One suggestion that I've heard is to have t

    • What we did at an old job was get a known good image (chart, graph, or even a snapshot of the desktop).

      Then create a checksum (md5 is kind of small but I liked to use sha1). Then you just need to create a checksum on the test image and compare checksums, and then you could flag an image for human review.

      There was some other super simple things I came up with as well for images and rough UI testing.
    • I've occasionally used another technique for testing the text output of legacy scripts to prevent regression while refactoring or fixing bugs. Redirect all output to a file, for both the new and old versions of the script, and then use "diff" to detect differences. This is certainly worse than fully automated testing, but much better than fully manual testing.

      Redirecting "all output" is slightly more challenging if the legacy code doesn't have the hooks Ovid recommends, but not impossible. Although "pri
  • How would I test a GUI interface for something?

    Do you mean test that the interface does what you think it will on the back end, or test that the interface is usable?

Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.