1=head1 NAME 2 3Test::Tutorial - A tutorial about writing really basic tests 4 5=head1 DESCRIPTION 6 7 8I<AHHHHHHH!!!! NOT TESTING! Anything but testing! 9Beat me, whip me, send me to Detroit, but don't make 10me write tests!> 11 12I<*sob*> 13 14I<Besides, I don't know how to write the damned things.> 15 16 17Is this you? Is writing tests right up there with writing 18documentation and having your fingernails pulled out? Did you open up 19a test and read 20 21 ######## We start with some black magic 22 23and decide that's quite enough for you? 24 25It's ok. That's all gone now. We've done all the black magic for 26you. And here are the tricks... 27 28 29=head2 Nuts and bolts of testing. 30 31Here's the most basic test program. 32 33 #!/usr/bin/perl -w 34 35 print "1..1\n"; 36 37 print 1 + 1 == 2 ? "ok 1\n" : "not ok 1\n"; 38 39since 1 + 1 is 2, it prints: 40 41 1..1 42 ok 1 43 44What this says is: C<1..1> "I'm going to run one test." [1] C<ok 1> 45"The first test passed". And that's about all magic there is to 46testing. Your basic unit of testing is the I<ok>. For each thing you 47test, an C<ok> is printed. Simple. B<Test::Harness> interprets your test 48results to determine if you succeeded or failed (more on that later). 49 50Writing all these print statements rapidly gets tedious. Fortunately, 51there's B<Test::Simple>. It has one function, C<ok()>. 52 53 #!/usr/bin/perl -w 54 55 use Test::Simple tests => 1; 56 57 ok( 1 + 1 == 2 ); 58 59and that does the same thing as the code above. C<ok()> is the backbone 60of Perl testing, and we'll be using it instead of roll-your-own from 61here on. If C<ok()> gets a true value, the test passes. False, it 62fails. 63 64 #!/usr/bin/perl -w 65 66 use Test::Simple tests => 2; 67 ok( 1 + 1 == 2 ); 68 ok( 2 + 2 == 5 ); 69 70from that comes 71 72 1..2 73 ok 1 74 not ok 2 75 # Failed test (test.pl at line 5) 76 # Looks like you failed 1 tests of 2. 77 78C<1..2> "I'm going to run two tests." This number is used to ensure 79your test program ran all the way through and didn't die or skip some 80tests. C<ok 1> "The first test passed." C<not ok 2> "The second test 81failed". Test::Simple helpfully prints out some extra commentary about 82your tests. 83 84It's not scary. Come, hold my hand. We're going to give an example 85of testing a module. For our example, we'll be testing a date 86library, B<Date::ICal>. It's on CPAN, so download a copy and follow 87along. [2] 88 89 90=head2 Where to start? 91 92This is the hardest part of testing, where do you start? People often 93get overwhelmed at the apparent enormity of the task of testing a 94whole module. Best place to start is at the beginning. Date::ICal is 95an object-oriented module, and that means you start by making an 96object. So we test C<new()>. 97 98 #!/usr/bin/perl -w 99 100 use Test::Simple tests => 2; 101 102 use Date::ICal; 103 104 my $ical = Date::ICal->new; # create an object 105 ok( defined $ical ); # check that we got something 106 ok( $ical->isa('Date::ICal') ); # and it's the right class 107 108run that and you should get: 109 110 1..2 111 ok 1 112 ok 2 113 114congratulations, you've written your first useful test. 115 116 117=head2 Names 118 119That output isn't terribly descriptive, is it? When you have two 120tests you can figure out which one is #2, but what if you have 102? 121 122Each test can be given a little descriptive name as the second 123argument to C<ok()>. 124 125 use Test::Simple tests => 2; 126 127 ok( defined $ical, 'new() returned something' ); 128 ok( $ical->isa('Date::ICal'), " and it's the right class" ); 129 130So now you'd see... 131 132 1..2 133 ok 1 - new() returned something 134 ok 2 - and it's the right class 135 136 137=head2 Test the manual 138 139Simplest way to build up a decent testing suite is to just test what 140the manual says it does. [3] Let's pull something out of the 141L<Date::ICal/SYNOPSIS> and test that all its bits work. 142 143 #!/usr/bin/perl -w 144 145 use Test::Simple tests => 8; 146 147 use Date::ICal; 148 149 $ical = Date::ICal->new( year => 1964, month => 10, day => 16, 150 hour => 16, min => 12, sec => 47, 151 tz => '0530' ); 152 153 ok( defined $ical, 'new() returned something' ); 154 ok( $ical->isa('Date::ICal'), " and it's the right class" ); 155 ok( $ical->sec == 47, ' sec()' ); 156 ok( $ical->min == 12, ' min()' ); 157 ok( $ical->hour == 16, ' hour()' ); 158 ok( $ical->day == 17, ' day()' ); 159 ok( $ical->month == 10, ' month()' ); 160 ok( $ical->year == 1964, ' year()' ); 161 162run that and you get: 163 164 1..8 165 ok 1 - new() returned something 166 ok 2 - and it's the right class 167 ok 3 - sec() 168 ok 4 - min() 169 ok 5 - hour() 170 not ok 6 - day() 171 # Failed test (- at line 16) 172 ok 7 - month() 173 ok 8 - year() 174 # Looks like you failed 1 tests of 8. 175 176Whoops, a failure! [4] Test::Simple helpfully lets us know on what line 177the failure occurred, but not much else. We were supposed to get 17, 178but we didn't. What did we get?? Dunno. We'll have to re-run the 179test in the debugger or throw in some print statements to find out. 180 181Instead, we'll switch from B<Test::Simple> to B<Test::More>. B<Test::More> 182does everything B<Test::Simple> does, and more! In fact, Test::More does 183things I<exactly> the way Test::Simple does. You can literally swap 184Test::Simple out and put Test::More in its place. That's just what 185we're going to do. 186 187Test::More does more than Test::Simple. The most important difference 188at this point is it provides more informative ways to say "ok". 189Although you can write almost any test with a generic C<ok()>, it 190can't tell you what went wrong. Instead, we'll use the C<is()> 191function, which lets us declare that something is supposed to be the 192same as something else: 193 194 #!/usr/bin/perl -w 195 196 use Test::More tests => 8; 197 198 use Date::ICal; 199 200 $ical = Date::ICal->new( year => 1964, month => 10, day => 16, 201 hour => 16, min => 12, sec => 47, 202 tz => '0530' ); 203 204 ok( defined $ical, 'new() returned something' ); 205 ok( $ical->isa('Date::ICal'), " and it's the right class" ); 206 is( $ical->sec, 47, ' sec()' ); 207 is( $ical->min, 12, ' min()' ); 208 is( $ical->hour, 16, ' hour()' ); 209 is( $ical->day, 17, ' day()' ); 210 is( $ical->month, 10, ' month()' ); 211 is( $ical->year, 1964, ' year()' ); 212 213"Is C<$ical-E<gt>sec> 47?" "Is C<$ical-E<gt>min> 12?" With C<is()> in place, 214you get some more information 215 216 1..8 217 ok 1 - new() returned something 218 ok 2 - and it's the right class 219 ok 3 - sec() 220 ok 4 - min() 221 ok 5 - hour() 222 not ok 6 - day() 223 # Failed test (- at line 16) 224 # got: '16' 225 # expected: '17' 226 ok 7 - month() 227 ok 8 - year() 228 # Looks like you failed 1 tests of 8. 229 230letting us know that C<$ical-E<gt>day> returned 16, but we expected 17. A 231quick check shows that the code is working fine, we made a mistake 232when writing up the tests. Just change it to: 233 234 is( $ical->day, 16, ' day()' ); 235 236and everything works. 237 238So any time you're doing a "this equals that" sort of test, use C<is()>. 239It even works on arrays. The test is always in scalar context, so you 240can test how many elements are in a list this way. [5] 241 242 is( @foo, 5, 'foo has 5 elements' ); 243 244 245=head2 Sometimes the tests are wrong 246 247Which brings us to a very important lesson. Code has bugs. Tests are 248code. Ergo, tests have bugs. A failing test could mean a bug in the 249code, but don't discount the possibility that the test is wrong. 250 251On the flip side, don't be tempted to prematurely declare a test 252incorrect just because you're having trouble finding the bug. 253Invalidating a test isn't something to be taken lightly, and don't use 254it as a cop out to avoid work. 255 256 257=head2 Testing lots of values 258 259We're going to be wanting to test a lot of dates here, trying to trick 260the code with lots of different edge cases. Does it work before 1970? 261After 2038? Before 1904? Do years after 10,000 give it trouble? 262Does it get leap years right? We could keep repeating the code above, 263or we could set up a little try/expect loop. 264 265 use Test::More tests => 32; 266 use Date::ICal; 267 268 my %ICal_Dates = ( 269 # An ICal string And the year, month, date 270 # hour, minute and second we expect. 271 '19971024T120000' => # from the docs. 272 [ 1997, 10, 24, 12, 0, 0 ], 273 '20390123T232832' => # after the Unix epoch 274 [ 2039, 1, 23, 23, 28, 32 ], 275 '19671225T000000' => # before the Unix epoch 276 [ 1967, 12, 25, 0, 0, 0 ], 277 '18990505T232323' => # before the MacOS epoch 278 [ 1899, 5, 5, 23, 23, 23 ], 279 ); 280 281 282 while( my($ical_str, $expect) = each %ICal_Dates ) { 283 my $ical = Date::ICal->new( ical => $ical_str ); 284 285 ok( defined $ical, "new(ical => '$ical_str')" ); 286 ok( $ical->isa('Date::ICal'), " and it's the right class" ); 287 288 is( $ical->year, $expect->[0], ' year()' ); 289 is( $ical->month, $expect->[1], ' month()' ); 290 is( $ical->day, $expect->[2], ' day()' ); 291 is( $ical->hour, $expect->[3], ' hour()' ); 292 is( $ical->min, $expect->[4], ' min()' ); 293 is( $ical->sec, $expect->[5], ' sec()' ); 294 } 295 296So now we can test bunches of dates by just adding them to 297C<%ICal_Dates>. Now that it's less work to test with more dates, you'll 298be inclined to just throw more in as you think of them. 299Only problem is, every time we add to that we have to keep adjusting 300the C<use Test::More tests =E<gt> ##> line. That can rapidly get 301annoying. There's two ways to make this work better. 302 303First, we can calculate the plan dynamically using the C<plan()> 304function. 305 306 use Test::More; 307 use Date::ICal; 308 309 my %ICal_Dates = ( 310 ...same as before... 311 ); 312 313 # For each key in the hash we're running 8 tests. 314 plan tests => keys %ICal_Dates * 8; 315 316Or to be even more flexible, we use C<no_plan>. This means we're just 317running some tests, don't know how many. [6] 318 319 use Test::More 'no_plan'; # instead of tests => 32 320 321now we can just add tests and not have to do all sorts of math to 322figure out how many we're running. 323 324 325=head2 Informative names 326 327Take a look at this line here 328 329 ok( defined $ical, "new(ical => '$ical_str')" ); 330 331we've added more detail about what we're testing and the ICal string 332itself we're trying out to the name. So you get results like: 333 334 ok 25 - new(ical => '19971024T120000') 335 ok 26 - and it's the right class 336 ok 27 - year() 337 ok 28 - month() 338 ok 29 - day() 339 ok 30 - hour() 340 ok 31 - min() 341 ok 32 - sec() 342 343if something in there fails, you'll know which one it was and that 344will make tracking down the problem easier. So try to put a bit of 345debugging information into the test names. 346 347Describe what the tests test, to make debugging a failed test easier 348for you or for the next person who runs your test. 349 350 351=head2 Skipping tests 352 353Poking around in the existing Date::ICal tests, I found this in 354F<t/01sanity.t> [7] 355 356 #!/usr/bin/perl -w 357 358 use Test::More tests => 7; 359 use Date::ICal; 360 361 # Make sure epoch time is being handled sanely. 362 my $t1 = Date::ICal->new( epoch => 0 ); 363 is( $t1->epoch, 0, "Epoch time of 0" ); 364 365 # XXX This will only work on unix systems. 366 is( $t1->ical, '19700101Z', " epoch to ical" ); 367 368 is( $t1->year, 1970, " year()" ); 369 is( $t1->month, 1, " month()" ); 370 is( $t1->day, 1, " day()" ); 371 372 # like the tests above, but starting with ical instead of epoch 373 my $t2 = Date::ICal->new( ical => '19700101Z' ); 374 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" ); 375 376 is( $t2->epoch, 0, " and back to ICal" ); 377 378The beginning of the epoch is different on most non-Unix operating 379systems [8]. Even though Perl smooths out the differences for the most 380part, certain ports do it differently. MacPerl is one off the top of 381my head. [9] We I<know> this will never work on MacOS. So rather than 382just putting a comment in the test, we can explicitly say it's never 383going to work and skip the test. 384 385 use Test::More tests => 7; 386 use Date::ICal; 387 388 # Make sure epoch time is being handled sanely. 389 my $t1 = Date::ICal->new( epoch => 0 ); 390 is( $t1->epoch, 0, "Epoch time of 0" ); 391 392 SKIP: { 393 skip('epoch to ICal not working on MacOS', 6) 394 if $^O eq 'MacOS'; 395 396 is( $t1->ical, '19700101Z', " epoch to ical" ); 397 398 is( $t1->year, 1970, " year()" ); 399 is( $t1->month, 1, " month()" ); 400 is( $t1->day, 1, " day()" ); 401 402 # like the tests above, but starting with ical instead of epoch 403 my $t2 = Date::ICal->new( ical => '19700101Z' ); 404 is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" ); 405 406 is( $t2->epoch, 0, " and back to ICal" ); 407 } 408 409A little bit of magic happens here. When running on anything but 410MacOS, all the tests run normally. But when on MacOS, C<skip()> causes 411the entire contents of the SKIP block to be jumped over. It's never 412run. Instead, it prints special output that tells Test::Harness that 413the tests have been skipped. 414 415 1..7 416 ok 1 - Epoch time of 0 417 ok 2 # skip epoch to ICal not working on MacOS 418 ok 3 # skip epoch to ICal not working on MacOS 419 ok 4 # skip epoch to ICal not working on MacOS 420 ok 5 # skip epoch to ICal not working on MacOS 421 ok 6 # skip epoch to ICal not working on MacOS 422 ok 7 # skip epoch to ICal not working on MacOS 423 424This means your tests won't fail on MacOS. This means less emails 425from MacPerl users telling you about failing tests that you know will 426never work. You've got to be careful with skip tests. These are for 427tests which don't work and I<never will>. It is not for skipping 428genuine bugs (we'll get to that in a moment). 429 430The tests are wholly and completely skipped. [10] This will work. 431 432 SKIP: { 433 skip("I don't wanna die!"); 434 435 die, die, die, die, die; 436 } 437 438 439=head2 Todo tests 440 441Thumbing through the Date::ICal man page, I came across this: 442 443 ical 444 445 $ical_string = $ical->ical; 446 447 Retrieves, or sets, the date on the object, using any 448 valid ICal date/time string. 449 450"Retrieves or sets". Hmmm, didn't see a test for using C<ical()> to set 451the date in the Date::ICal test suite. So I'll write one. 452 453 use Test::More tests => 1; 454 use Date::ICal; 455 456 my $ical = Date::ICal->new; 457 $ical->ical('20201231Z'); 458 is( $ical->ical, '20201231Z', 'Setting via ical()' ); 459 460run that and I get 461 462 1..1 463 not ok 1 - Setting via ical() 464 # Failed test (- at line 6) 465 # got: '20010814T233649Z' 466 # expected: '20201231Z' 467 # Looks like you failed 1 tests of 1. 468 469Whoops! Looks like it's unimplemented. Let's assume we don't have 470the time to fix this. [11] Normally, you'd just comment out the test 471and put a note in a todo list somewhere. Instead, we're going to 472explicitly state "this test will fail" by wrapping it in a C<TODO> block. 473 474 use Test::More tests => 1; 475 476 TODO: { 477 local $TODO = 'ical($ical) not yet implemented'; 478 479 my $ical = Date::ICal->new; 480 $ical->ical('20201231Z'); 481 482 is( $ical->ical, '20201231Z', 'Setting via ical()' ); 483 } 484 485Now when you run, it's a little different: 486 487 1..1 488 not ok 1 - Setting via ical() # TODO ical($ical) not yet implemented 489 # got: '20010822T201551Z' 490 # expected: '20201231Z' 491 492Test::More doesn't say "Looks like you failed 1 tests of 1". That '# 493TODO' tells Test::Harness "this is supposed to fail" and it treats a 494failure as a successful test. So you can write tests even before 495you've fixed the underlying code. 496 497If a TODO test passes, Test::Harness will report it "UNEXPECTEDLY 498SUCCEEDED". When that happens, you simply remove the TODO block with 499C<local $TODO> and turn it into a real test. 500 501 502=head2 Testing with taint mode. 503 504Taint mode is a funny thing. It's the globalest of all global 505features. Once you turn it on, it affects I<all> code in your program 506and I<all> modules used (and all the modules they use). If a single 507piece of code isn't taint clean, the whole thing explodes. With that 508in mind, it's very important to ensure your module works under taint 509mode. 510 511It's very simple to have your tests run under taint mode. Just throw 512a C<-T> into the C<#!> line. Test::Harness will read the switches 513in C<#!> and use them to run your tests. 514 515 #!/usr/bin/perl -Tw 516 517 ...test normally here... 518 519So when you say C<make test> it will be run with taint mode and 520warnings on. 521 522 523=head1 FOOTNOTES 524 525=over 4 526 527=item 1 528 529The first number doesn't really mean anything, but it has to be 1. 530It's the second number that's important. 531 532=item 2 533 534For those following along at home, I'm using version 1.31. It has 535some bugs, which is good -- we'll uncover them with our tests. 536 537=item 3 538 539You can actually take this one step further and test the manual 540itself. Have a look at B<Test::Inline> (formerly B<Pod::Tests>). 541 542=item 4 543 544Yes, there's a mistake in the test suite. What! Me, contrived? 545 546=item 5 547 548We'll get to testing the contents of lists later. 549 550=item 6 551 552But what happens if your test program dies halfway through?! Since we 553didn't say how many tests we're going to run, how can we know it 554failed? No problem, Test::More employs some magic to catch that death 555and turn the test into a failure, even if every test passed up to that 556point. 557 558=item 7 559 560I cleaned it up a little. 561 562=item 8 563 564Most Operating Systems record time as the number of seconds since a 565certain date. This date is the beginning of the epoch. Unix's starts 566at midnight January 1st, 1970 GMT. 567 568=item 9 569 570MacOS's epoch is midnight January 1st, 1904. VMS's is midnight, 571November 17th, 1858, but vmsperl emulates the Unix epoch so it's not a 572problem. 573 574=item 10 575 576As long as the code inside the SKIP block at least compiles. Please 577don't ask how. No, it's not a filter. 578 579=item 11 580 581Do NOT be tempted to use TODO tests as a way to avoid fixing simple 582bugs! 583 584=back 585 586=head1 AUTHORS 587 588Michael G Schwern E<lt>schwern@pobox.comE<gt> and the perl-qa dancers! 589 590=head1 COPYRIGHT 591 592Copyright 2001 by Michael G Schwern E<lt>schwern@pobox.comE<gt>. 593 594This documentation is free; you can redistribute it and/or modify it 595under the same terms as Perl itself. 596 597Irrespective of its distribution, all code examples in these files 598are hereby placed into the public domain. You are permitted and 599encouraged to use this code in your own programs for fun 600or for profit as you see fit. A simple comment in the code giving 601credit would be courteous but is not required. 602 603=cut 604