1PCRETEST(1) PCRETEST(1) 2 3 4NAME 5 pcretest - a program for testing Perl-compatible regular expressions. 6 7 8SYNOPSIS 9 10 pcretest [options] [input file [output file]] 11 12 pcretest was written as a test program for the PCRE regular expression 13 library itself, but it can also be used for experimenting with regular 14 expressions. This document describes the features of the test program; 15 for details of the regular expressions themselves, see the pcrepattern 16 documentation. For details of the PCRE library function calls and their 17 options, see the pcreapi and pcre16 documentation. The input for 18 pcretest is a sequence of regular expression patterns and strings to be 19 matched, as described below. The output shows the result of each match. 20 Options on the command line and the patterns control PCRE options and 21 exactly what is output. 22 23 24PCRE's 8-BIT and 16-BIT LIBRARIES 25 26 From release 8.30, two separate PCRE libraries can be built. The origi- 27 nal one supports 8-bit character strings, whereas the newer 16-bit 28 library supports character strings encoded in 16-bit units. The 29 pcretest program can be used to test both libraries. However, it is 30 itself still an 8-bit program, reading 8-bit input and writing 8-bit 31 output. When testing the 16-bit library, the patterns and data strings 32 are converted to 16-bit format before being passed to the PCRE library 33 functions. Results are converted to 8-bit for output. 34 35 References to functions and structures of the form pcre[16]_xx below 36 mean "pcre_xx when using the 8-bit library or pcre16_xx when using the 37 16-bit library". 38 39 40COMMAND LINE OPTIONS 41 42 -16 If both the 8-bit and the 16-bit libraries have been built, 43 this option causes the 16-bit library to be used. If only the 44 16-bit library has been built, this is the default (so has no 45 effect). If only the 8-bit library has been built, this 46 option causes an error. 47 48 -b Behave as if each pattern has the /B (show byte code) modi- 49 fier; the internal form is output after compilation. 50 51 -C Output the version number of the PCRE library, and all avail- 52 able information about the optional features that are 53 included, and then exit. All other options are ignored. 54 55 -C option Output information about a specific build-time option, then 56 exit. This functionality is intended for use in scripts such 57 as RunTest. The following options output the value indicated: 58 59 linksize the internal link size (2, 3, or 4) 60 newline the default newline setting: 61 CR, LF, CRLF, ANYCRLF, or ANY 62 63 The following options output 1 for true or zero for false: 64 65 jit just-in-time support is available 66 pcre16 the 16-bit library was built 67 pcre8 the 8-bit library was built 68 ucp Unicode property support is available 69 utf UTF-8 and/or UTF-16 support is available 70 71 -d Behave as if each pattern has the /D (debug) modifier; the 72 internal form and information about the compiled pattern is 73 output after compilation; -d is equivalent to -b -i. 74 75 -dfa Behave as if each data line contains the \D escape sequence; 76 this causes the alternative matching function, 77 pcre[16]_dfa_exec(), to be used instead of the standard 78 pcre[16]_exec() function (more detail is given below). 79 80 -help Output a brief summary these options and then exit. 81 82 -i Behave as if each pattern has the /I modifier; information 83 about the compiled pattern is given after compilation. 84 85 -M Behave as if each data line contains the \M escape sequence; 86 this causes PCRE to discover the minimum MATCH_LIMIT and 87 MATCH_LIMIT_RECURSION settings by calling pcre[16]_exec() 88 repeatedly with different limits. 89 90 -m Output the size of each compiled pattern after it has been 91 compiled. This is equivalent to adding /M to each regular 92 expression. The size is given in bytes for both libraries. 93 94 -o osize Set the number of elements in the output vector that is used 95 when calling pcre[16]_exec() or pcre[16]_dfa_exec() to be 96 osize. The default value is 45, which is enough for 14 cap- 97 turing subexpressions for pcre[16]_exec() or 22 different 98 matches for pcre[16]_dfa_exec(). The vector size can be 99 changed for individual matching calls by including \O in the 100 data line (see below). 101 102 -p Behave as if each pattern has the /P modifier; the POSIX 103 wrapper API is used to call PCRE. None of the other options 104 has any effect when -p is set. This option can be used only 105 with the 8-bit library. 106 107 -q Do not output the version number of pcretest at the start of 108 execution. 109 110 -S size On Unix-like systems, set the size of the run-time stack to 111 size megabytes. 112 113 -s or -s+ Behave as if each pattern has the /S modifier; in other 114 words, force each pattern to be studied. If -s+ is used, all 115 the JIT compile options are passed to pcre[16]_study(), caus- 116 ing just-in-time optimization to be set up if it is avail- 117 able, for both full and partial matching. Specific JIT com- 118 pile options can be selected by following -s+ with a digit in 119 the range 1 to 7, which selects the JIT compile modes as fol- 120 lows: 121 122 1 normal match only 123 2 soft partial match only 124 3 normal match and soft partial match 125 4 hard partial match only 126 6 soft and hard partial match 127 7 all three modes (default) 128 129 If -s++ is used instead of -s+ (with or without a following 130 digit), the text "(JIT)" is added to the first output line 131 after a match or no match when JIT-compiled code was actually 132 used. 133 134 If the /I or /D option is present on a pattern (requesting output about 135 the compiled pattern), information about the result of studying is not 136 included when studying is caused only by -s and neither -i nor -d is 137 present on the command line. This behaviour means that the output from 138 tests that are run with and without -s should be identical, except when 139 options that output information about the actual running of a match are 140 set. 141 142 The -M, -t, and -tm options, which give information about resources 143 used, are likely to produce different output with and without -s. Out- 144 put may also differ if the /C option is present on an individual pat- 145 tern. This uses callouts to trace the the matching process, and this 146 may be different between studied and non-studied patterns. If the pat- 147 tern contains (*MARK) items there may also be differences, for the same 148 reason. The -s command line option can be overridden for specific pat- 149 terns that should never be studied (see the /S pattern modifier below). 150 151 -t Run each compile, study, and match many times with a timer, 152 and output resulting time per compile or match (in millisec- 153 onds). Do not set -m with -t, because you will then get the 154 size output a zillion times, and the timing will be dis- 155 torted. You can control the number of iterations that are 156 used for timing by following -t with a number (as a separate 157 item on the command line). For example, "-t 1000" would iter- 158 ate 1000 times. The default is to iterate 500000 times. 159 160 -tm This is like -t except that it times only the matching phase, 161 not the compile or study phases. 162 163 164DESCRIPTION 165 166 If pcretest is given two filename arguments, it reads from the first 167 and writes to the second. If it is given only one filename argument, it 168 reads from that file and writes to stdout. Otherwise, it reads from 169 stdin and writes to stdout, and prompts for each line of input, using 170 "re>" to prompt for regular expressions, and "data>" to prompt for data 171 lines. 172 173 When pcretest is built, a configuration option can specify that it 174 should be linked with the libreadline library. When this is done, if 175 the input is from a terminal, it is read using the readline() function. 176 This provides line-editing and history facilities. The output from the 177 -help option states whether or not readline() will be used. 178 179 The program handles any number of sets of input on a single input file. 180 Each set starts with a regular expression, and continues with any num- 181 ber of data lines to be matched against the pattern. 182 183 Each data line is matched separately and independently. If you want to 184 do multi-line matches, you have to use the \n escape sequence (or \r or 185 \r\n, etc., depending on the newline setting) in a single line of input 186 to encode the newline sequences. There is no limit on the length of 187 data lines; the input buffer is automatically extended if it is too 188 small. 189 190 An empty line signals the end of the data lines, at which point a new 191 regular expression is read. The regular expressions are given enclosed 192 in any non-alphanumeric delimiters other than backslash, for example: 193 194 /(a|bc)x+yz/ 195 196 White space before the initial delimiter is ignored. A regular expres- 197 sion may be continued over several input lines, in which case the new- 198 line characters are included within it. It is possible to include the 199 delimiter within the pattern by escaping it, for example 200 201 /abc\/def/ 202 203 If you do so, the escape and the delimiter form part of the pattern, 204 but since delimiters are always non-alphanumeric, this does not affect 205 its interpretation. If the terminating delimiter is immediately fol- 206 lowed by a backslash, for example, 207 208 /abc/\ 209 210 then a backslash is added to the end of the pattern. This is done to 211 provide a way of testing the error condition that arises if a pattern 212 finishes with a backslash, because 213 214 /abc\/ 215 216 is interpreted as the first line of a pattern that starts with "abc/", 217 causing pcretest to read the next line as a continuation of the regular 218 expression. 219 220 221PATTERN MODIFIERS 222 223 A pattern may be followed by any number of modifiers, which are mostly 224 single characters. Following Perl usage, these are referred to below 225 as, for example, "the /i modifier", even though the delimiter of the 226 pattern need not always be a slash, and no slash is used when writing 227 modifiers. White space may appear between the final pattern delimiter 228 and the first modifier, and between the modifiers themselves. 229 230 The /i, /m, /s, and /x modifiers set the PCRE_CASELESS, PCRE_MULTILINE, 231 PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when pcre[16]_com- 232 pile() is called. These four modifier letters have the same effect as 233 they do in Perl. For example: 234 235 /caseless/i 236 237 The following table shows additional modifiers for setting PCRE com- 238 pile-time options that do not correspond to anything in Perl: 239 240 /8 PCRE_UTF8 ) when using the 8-bit 241 /? PCRE_NO_UTF8_CHECK ) library 242 243 /8 PCRE_UTF16 ) when using the 16-bit 244 /? PCRE_NO_UTF16_CHECK ) library 245 246 /A PCRE_ANCHORED 247 /C PCRE_AUTO_CALLOUT 248 /E PCRE_DOLLAR_ENDONLY 249 /f PCRE_FIRSTLINE 250 /J PCRE_DUPNAMES 251 /N PCRE_NO_AUTO_CAPTURE 252 /U PCRE_UNGREEDY 253 /W PCRE_UCP 254 /X PCRE_EXTRA 255 /Y PCRE_NO_START_OPTIMIZE 256 /<JS> PCRE_JAVASCRIPT_COMPAT 257 /<cr> PCRE_NEWLINE_CR 258 /<lf> PCRE_NEWLINE_LF 259 /<crlf> PCRE_NEWLINE_CRLF 260 /<anycrlf> PCRE_NEWLINE_ANYCRLF 261 /<any> PCRE_NEWLINE_ANY 262 /<bsr_anycrlf> PCRE_BSR_ANYCRLF 263 /<bsr_unicode> PCRE_BSR_UNICODE 264 265 The modifiers that are enclosed in angle brackets are literal strings 266 as shown, including the angle brackets, but the letters within can be 267 in either case. This example sets multiline matching with CRLF as the 268 line ending sequence: 269 270 /^abc/m<CRLF> 271 272 As well as turning on the PCRE_UTF8/16 option, the /8 modifier causes 273 all non-printing characters in output strings to be printed using the 274 \x{hh...} notation. Otherwise, those less than 0x100 are output in hex 275 without the curly brackets. 276 277 Full details of the PCRE options are given in the pcreapi documenta- 278 tion. 279 280 Finding all matches in a string 281 282 Searching for all possible matches within each subject string can be 283 requested by the /g or /G modifier. After finding a match, PCRE is 284 called again to search the remainder of the subject string. The differ- 285 ence between /g and /G is that the former uses the startoffset argument 286 to pcre[16]_exec() to start searching at a new point within the entire 287 string (which is in effect what Perl does), whereas the latter passes 288 over a shortened substring. This makes a difference to the matching 289 process if the pattern begins with a lookbehind assertion (including \b 290 or \B). 291 292 If any call to pcre[16]_exec() in a /g or /G sequence matches an empty 293 string, the next call is done with the PCRE_NOTEMPTY_ATSTART and 294 PCRE_ANCHORED flags set in order to search for another, non-empty, 295 match at the same point. If this second match fails, the start offset 296 is advanced, and the normal match is retried. This imitates the way 297 Perl handles such cases when using the /g modifier or the split() func- 298 tion. Normally, the start offset is advanced by one character, but if 299 the newline convention recognizes CRLF as a newline, and the current 300 character is CR followed by LF, an advance of two is used. 301 302 Other modifiers 303 304 There are yet more modifiers for controlling the way pcretest operates. 305 306 The /+ modifier requests that as well as outputting the substring that 307 matched the entire pattern, pcretest should in addition output the 308 remainder of the subject string. This is useful for tests where the 309 subject contains multiple copies of the same substring. If the + modi- 310 fier appears twice, the same action is taken for captured substrings. 311 In each case the remainder is output on the following line with a plus 312 character following the capture number. Note that this modifier must 313 not immediately follow the /S modifier because /S+ and /S++ have other 314 meanings. 315 316 The /= modifier requests that the values of all potential captured 317 parentheses be output after a match. By default, only those up to the 318 highest one actually used in the match are output (corresponding to the 319 return code from pcre[16]_exec()). Values in the offsets vector corre- 320 sponding to higher numbers should be set to -1, and these are output as 321 "<unset>". This modifier gives a way of checking that this is happen- 322 ing. 323 324 The /B modifier is a debugging feature. It requests that pcretest out- 325 put a representation of the compiled code after compilation. Normally 326 this information contains length and offset values; however, if /Z is 327 also present, this data is replaced by spaces. This is a special fea- 328 ture for use in the automatic test scripts; it ensures that the same 329 output is generated for different internal link sizes. 330 331 The /D modifier is a PCRE debugging feature, and is equivalent to /BI, 332 that is, both the /B and the /I modifiers. 333 334 The /F modifier causes pcretest to flip the byte order of the 2-byte 335 and 4-byte fields in the compiled pattern. This facility is for testing 336 the feature in PCRE that allows it to execute patterns that were com- 337 piled on a host with a different endianness. This feature is not avail- 338 able when the POSIX interface to PCRE is being used, that is, when the 339 /P pattern modifier is specified. See also the section about saving and 340 reloading compiled patterns below. 341 342 The /I modifier requests that pcretest output information about the 343 compiled pattern (whether it is anchored, has a fixed first character, 344 and so on). It does this by calling pcre[16]_fullinfo() after compiling 345 a pattern. If the pattern is studied, the results of that are also out- 346 put. 347 348 The /K modifier requests pcretest to show names from backtracking con- 349 trol verbs that are returned from calls to pcre[16]_exec(). It causes 350 pcretest to create a pcre[16]_extra block if one has not already been 351 created by a call to pcre[16]_study(), and to set the PCRE_EXTRA_MARK 352 flag and the mark field within it, every time that pcre[16]_exec() is 353 called. If the variable that the mark field points to is non-NULL for a 354 match, non-match, or partial match, pcretest prints the string to which 355 it points. For a match, this is shown on a line by itself, tagged with 356 "MK:". For a non-match it is added to the message. 357 358 The /L modifier must be followed directly by the name of a locale, for 359 example, 360 361 /pattern/Lfr_FR 362 363 For this reason, it must be the last modifier. The given locale is set, 364 pcre[16]_maketables() is called to build a set of character tables for 365 the locale, and this is then passed to pcre[16]_compile() when compil- 366 ing the regular expression. Without an /L (or /T) modifier, NULL is 367 passed as the tables pointer; that is, /L applies only to the expres- 368 sion on which it appears. 369 370 The /M modifier causes the size in bytes of the memory block used to 371 hold the compiled pattern to be output. This does not include the size 372 of the pcre[16] block; it is just the actual compiled data. If the pat- 373 tern is successfully studied with the PCRE_STUDY_JIT_COMPILE option, 374 the size of the JIT compiled code is also output. 375 376 If the /S modifier appears once, it causes pcre[16]_study() to be 377 called after the expression has been compiled, and the results used 378 when the expression is matched. If /S appears twice, it suppresses 379 studying, even if it was requested externally by the -s command line 380 option. This makes it possible to specify that certain patterns are 381 always studied, and others are never studied, independently of -s. This 382 feature is used in the test files in a few cases where the output is 383 different when the pattern is studied. 384 385 If the /S modifier is immediately followed by a + character, the call 386 to pcre[16]_study() is made with all the JIT study options, requesting 387 just-in-time optimization support if it is available, for both normal 388 and partial matching. If you want to restrict the JIT compiling modes, 389 you can follow /S+ with a digit in the range 1 to 7: 390 391 1 normal match only 392 2 soft partial match only 393 3 normal match and soft partial match 394 4 hard partial match only 395 6 soft and hard partial match 396 7 all three modes (default) 397 398 If /S++ is used instead of /S+ (with or without a following digit), the 399 text "(JIT)" is added to the first output line after a match or no 400 match when JIT-compiled code was actually used. 401 402 Note that there is also an independent /+ modifier; it must not be 403 given immediately after /S or /S+ because this will be misinterpreted. 404 405 If JIT studying is successful, the compiled JIT code will automatically 406 be used when pcre[16]_exec() is run, except when incompatible run-time 407 options are specified. For more details, see the pcrejit documentation. 408 See also the \J escape sequence below for a way of setting the size of 409 the JIT stack. 410 411 The /T modifier must be followed by a single digit. It causes a spe- 412 cific set of built-in character tables to be passed to pcre[16]_com- 413 pile(). It is used in the standard PCRE tests to check behaviour with 414 different character tables. The digit specifies the tables as follows: 415 416 0 the default ASCII tables, as distributed in 417 pcre_chartables.c.dist 418 1 a set of tables defining ISO 8859 characters 419 420 In table 1, some characters whose codes are greater than 128 are iden- 421 tified as letters, digits, spaces, etc. 422 423 Using the POSIX wrapper API 424 425 The /P modifier causes pcretest to call PCRE via the POSIX wrapper API 426 rather than its native API. This supports only the 8-bit library. When 427 /P is set, the following modifiers set options for the regcomp() func- 428 tion: 429 430 /i REG_ICASE 431 /m REG_NEWLINE 432 /N REG_NOSUB 433 /s REG_DOTALL ) 434 /U REG_UNGREEDY ) These options are not part of 435 /W REG_UCP ) the POSIX standard 436 /8 REG_UTF8 ) 437 438 The /+ modifier works as described above. All other modifiers are 439 ignored. 440 441 442DATA LINES 443 444 Before each data line is passed to pcre[16]_exec(), leading and trail- 445 ing white space is removed, and it is then scanned for \ escapes. Some 446 of these are pretty esoteric features, intended for checking out some 447 of the more complicated features of PCRE. If you are just testing 448 "ordinary" regular expressions, you probably don't need any of these. 449 The following escapes are recognized: 450 451 \a alarm (BEL, \x07) 452 \b backspace (\x08) 453 \e escape (\x27) 454 \f form feed (\x0c) 455 \n newline (\x0a) 456 \qdd set the PCRE_MATCH_LIMIT limit to dd 457 (any number of digits) 458 \r carriage return (\x0d) 459 \t tab (\x09) 460 \v vertical tab (\x0b) 461 \nnn octal character (up to 3 octal digits); always 462 a byte unless > 255 in UTF-8 or 16-bit mode 463 \xhh hexadecimal byte (up to 2 hex digits) 464 \x{hh...} hexadecimal character (any number of hex digits) 465 \A pass the PCRE_ANCHORED option to pcre[16]_exec() 466 or pcre[16]_dfa_exec() 467 \B pass the PCRE_NOTBOL option to pcre[16]_exec() 468 or pcre[16]_dfa_exec() 469 \Cdd call pcre[16]_copy_substring() for substring dd 470 after a successful match (number less than 32) 471 \Cname call pcre[16]_copy_named_substring() for substring 472 "name" after a successful match (name termin- 473 ated by next non alphanumeric character) 474 \C+ show the current captured substrings at callout 475 time 476 \C- do not supply a callout function 477 \C!n return 1 instead of 0 when callout number n is 478 reached 479 \C!n!m return 1 instead of 0 when callout number n is 480 reached for the nth time 481 \C*n pass the number n (may be negative) as callout 482 data; this is used as the callout return value 483 \D use the pcre[16]_dfa_exec() match function 484 \F only shortest match for pcre[16]_dfa_exec() 485 \Gdd call pcre[16]_get_substring() for substring dd 486 after a successful match (number less than 32) 487 \Gname call pcre[16]_get_named_substring() for substring 488 "name" after a successful match (name termin- 489 ated by next non-alphanumeric character) 490 \Jdd set up a JIT stack of dd kilobytes maximum (any 491 number of digits) 492 \L call pcre[16]_get_substringlist() after a 493 successful match 494 \M discover the minimum MATCH_LIMIT and 495 MATCH_LIMIT_RECURSION settings 496 \N pass the PCRE_NOTEMPTY option to pcre[16]_exec() 497 or pcre[16]_dfa_exec(); if used twice, pass the 498 PCRE_NOTEMPTY_ATSTART option 499 \Odd set the size of the output vector passed to 500 pcre[16]_exec() to dd (any number of digits) 501 \P pass the PCRE_PARTIAL_SOFT option to pcre[16]_exec() 502 or pcre[16]_dfa_exec(); if used twice, pass the 503 PCRE_PARTIAL_HARD option 504 \Qdd set the PCRE_MATCH_LIMIT_RECURSION limit to dd 505 (any number of digits) 506 \R pass the PCRE_DFA_RESTART option to pcre[16]_dfa_exec() 507 \S output details of memory get/free calls during matching 508 \Y pass the PCRE_NO_START_OPTIMIZE option to pcre[16]_exec() 509 or pcre[16]_dfa_exec() 510 \Z pass the PCRE_NOTEOL option to pcre[16]_exec() 511 or pcre[16]_dfa_exec() 512 \? pass the PCRE_NO_UTF[8|16]_CHECK option to 513 pcre[16]_exec() or pcre[16]_dfa_exec() 514 \>dd start the match at offset dd (optional "-"; then 515 any number of digits); this sets the startoffset 516 argument for pcre[16]_exec() or pcre[16]_dfa_exec() 517 \<cr> pass the PCRE_NEWLINE_CR option to pcre[16]_exec() 518 or pcre[16]_dfa_exec() 519 \<lf> pass the PCRE_NEWLINE_LF option to pcre[16]_exec() 520 or pcre[16]_dfa_exec() 521 \<crlf> pass the PCRE_NEWLINE_CRLF option to pcre[16]_exec() 522 or pcre[16]_dfa_exec() 523 \<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to pcre[16]_exec() 524 or pcre[16]_dfa_exec() 525 \<any> pass the PCRE_NEWLINE_ANY option to pcre[16]_exec() 526 or pcre[16]_dfa_exec() 527 528 The use of \x{hh...} is not dependent on the use of the /8 modifier on 529 the pattern. It is recognized always. There may be any number of hexa- 530 decimal digits inside the braces; invalid values provoke error mes- 531 sages. 532 533 Note that \xhh specifies one byte rather than one character in UTF-8 534 mode; this makes it possible to construct invalid UTF-8 sequences for 535 testing purposes. On the other hand, \x{hh} is interpreted as a UTF-8 536 character in UTF-8 mode, generating more than one byte if the value is 537 greater than 127. When testing the 8-bit library not in UTF-8 mode, 538 \x{hh} generates one byte for values less than 256, and causes an error 539 for greater values. 540 541 In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it 542 possible to construct invalid UTF-16 sequences for testing purposes. 543 544 The escapes that specify line ending sequences are literal strings, 545 exactly as shown. No more than one newline setting should be present in 546 any data line. 547 548 A backslash followed by anything else just escapes the anything else. 549 If the very last character is a backslash, it is ignored. This gives a 550 way of passing an empty line as data, since a real empty line termi- 551 nates the data input. 552 553 The \J escape provides a way of setting the maximum stack size that is 554 used by the just-in-time optimization code. It is ignored if JIT opti- 555 mization is not being used. Providing a stack that is larger than the 556 default 32K is necessary only for very complicated patterns. 557 558 If \M is present, pcretest calls pcre[16]_exec() several times, with 559 different values in the match_limit and match_limit_recursion fields of 560 the pcre[16]_extra data structure, until it finds the minimum numbers 561 for each parameter that allow pcre[16]_exec() to complete without 562 error. Because this is testing a specific feature of the normal inter- 563 pretive pcre[16]_exec() execution, the use of any JIT optimization that 564 might have been set up by the /S+ qualifier of -s+ option is disabled. 565 566 The match_limit number is a measure of the amount of backtracking that 567 takes place, and checking it out can be instructive. For most simple 568 matches, the number is quite small, but for patterns with very large 569 numbers of matching possibilities, it can become large very quickly 570 with increasing length of subject string. The match_limit_recursion 571 number is a measure of how much stack (or, if PCRE is compiled with 572 NO_RECURSE, how much heap) memory is needed to complete the match 573 attempt. 574 575 When \O is used, the value specified may be higher or lower than the 576 size set by the -O command line option (or defaulted to 45); \O applies 577 only to the call of pcre[16]_exec() for the line in which it appears. 578 579 If the /P modifier was present on the pattern, causing the POSIX wrap- 580 per API to be used, the only option-setting sequences that have any 581 effect are \B, \N, and \Z, causing REG_NOTBOL, REG_NOTEMPTY, and 582 REG_NOTEOL, respectively, to be passed to regexec(). 583 584 585THE ALTERNATIVE MATCHING FUNCTION 586 587 By default, pcretest uses the standard PCRE matching function, 588 pcre[16]_exec() to match each data line. PCRE also supports an alterna- 589 tive matching function, pcre[16]_dfa_test(), which operates in a dif- 590 ferent way, and has some restrictions. The differences between the two 591 functions are described in the pcrematching documentation. 592 593 If a data line contains the \D escape sequence, or if the command line 594 contains the -dfa option, the alternative matching function is used. 595 This function finds all possible matches at a given point. If, however, 596 the \F escape sequence is present in the data line, it stops after the 597 first match is found. This is always the shortest possible match. 598 599 600DEFAULT OUTPUT FROM PCRETEST 601 602 This section describes the output when the normal matching function, 603 pcre[16]_exec(), is being used. 604 605 When a match succeeds, pcretest outputs the list of captured substrings 606 that pcre[16]_exec() returns, starting with number 0 for the string 607 that matched the whole pattern. Otherwise, it outputs "No match" when 608 the return is PCRE_ERROR_NOMATCH, and "Partial match:" followed by the 609 partially matching substring when pcre[16]_exec() returns 610 PCRE_ERROR_PARTIAL. (Note that this is the entire substring that was 611 inspected during the partial match; it may include characters before 612 the actual match start if a lookbehind assertion, \K, \b, or \B was 613 involved.) For any other return, pcretest outputs the PCRE negative 614 error number and a short descriptive phrase. If the error is a failed 615 UTF string check, the offset of the start of the failing character and 616 the reason code are also output, provided that the size of the output 617 vector is at least two. Here is an example of an interactive pcretest 618 run. 619 620 $ pcretest 621 PCRE version 8.13 2011-04-30 622 623 re> /^abc(\d+)/ 624 data> abc123 625 0: abc123 626 1: 123 627 data> xyz 628 No match 629 630 Unset capturing substrings that are not followed by one that is set are 631 not returned by pcre[16]_exec(), and are not shown by pcretest. In the 632 following example, there are two capturing substrings, but when the 633 first data line is matched, the second, unset substring is not shown. 634 An "internal" unset substring is shown as "<unset>", as for the second 635 data line. 636 637 re> /(a)|(b)/ 638 data> a 639 0: a 640 1: a 641 data> b 642 0: b 643 1: <unset> 644 2: b 645 646 If the strings contain any non-printing characters, they are output as 647 \xhh escapes if the value is less than 256 and UTF mode is not set. 648 Otherwise they are output as \x{hh...} escapes. See below for the defi- 649 nition of non-printing characters. If the pattern has the /+ modifier, 650 the output for substring 0 is followed by the the rest of the subject 651 string, identified by "0+" like this: 652 653 re> /cat/+ 654 data> cataract 655 0: cat 656 0+ aract 657 658 If the pattern has the /g or /G modifier, the results of successive 659 matching attempts are output in sequence, like this: 660 661 re> /\Bi(\w\w)/g 662 data> Mississippi 663 0: iss 664 1: ss 665 0: iss 666 1: ss 667 0: ipp 668 1: pp 669 670 "No match" is output only if the first match attempt fails. Here is an 671 example of a failure message (the offset 4 that is specified by \>4 is 672 past the end of the subject string): 673 674 re> /xyz/ 675 data> xyz\>4 676 Error -24 (bad offset value) 677 678 If any of the sequences \C, \G, or \L are present in a data line that 679 is successfully matched, the substrings extracted by the convenience 680 functions are output with C, G, or L after the string number instead of 681 a colon. This is in addition to the normal full list. The string length 682 (that is, the return from the extraction function) is given in paren- 683 theses after each string for \C and \G. 684 685 Note that whereas patterns can be continued over several lines (a plain 686 ">" prompt is used for continuations), data lines may not. However new- 687 lines can be included in data by means of the \n escape (or \r, \r\n, 688 etc., depending on the newline sequence setting). 689 690 691OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION 692 693 When the alternative matching function, pcre[16]_dfa_exec(), is used 694 (by means of the \D escape sequence or the -dfa command line option), 695 the output consists of a list of all the matches that start at the 696 first point in the subject where there is at least one match. For exam- 697 ple: 698 699 re> /(tang|tangerine|tan)/ 700 data> yellow tangerine\D 701 0: tangerine 702 1: tang 703 2: tan 704 705 (Using the normal matching function on this data finds only "tang".) 706 The longest matching string is always given first (and numbered zero). 707 After a PCRE_ERROR_PARTIAL return, the output is "Partial match:", fol- 708 lowed by the partially matching substring. (Note that this is the 709 entire substring that was inspected during the partial match; it may 710 include characters before the actual match start if a lookbehind asser- 711 tion, \K, \b, or \B was involved.) 712 713 If /g is present on the pattern, the search for further matches resumes 714 at the end of the longest match. For example: 715 716 re> /(tang|tangerine|tan)/g 717 data> yellow tangerine and tangy sultana\D 718 0: tangerine 719 1: tang 720 2: tan 721 0: tang 722 1: tan 723 0: tan 724 725 Since the matching function does not support substring capture, the 726 escape sequences that are concerned with captured substrings are not 727 relevant. 728 729 730RESTARTING AFTER A PARTIAL MATCH 731 732 When the alternative matching function has given the PCRE_ERROR_PARTIAL 733 return, indicating that the subject partially matched the pattern, you 734 can restart the match with additional subject data by means of the \R 735 escape sequence. For example: 736 737 re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ 738 data> 23ja\P\D 739 Partial match: 23ja 740 data> n05\R\D 741 0: n05 742 743 For further information about partial matching, see the pcrepartial 744 documentation. 745 746 747CALLOUTS 748 749 If the pattern contains any callout requests, pcretest's callout func- 750 tion is called during matching. This works with both matching func- 751 tions. By default, the called function displays the callout number, the 752 start and current positions in the text at the callout time, and the 753 next pattern item to be tested. For example: 754 755 --->pqrabcdef 756 0 ^ ^ \d 757 758 This output indicates that callout number 0 occurred for a match 759 attempt starting at the fourth character of the subject string, when 760 the pointer was at the seventh character of the data, and when the next 761 pattern item was \d. Just one circumflex is output if the start and 762 current positions are the same. 763 764 Callouts numbered 255 are assumed to be automatic callouts, inserted as 765 a result of the /C pattern modifier. In this case, instead of showing 766 the callout number, the offset in the pattern, preceded by a plus, is 767 output. For example: 768 769 re> /\d?[A-E]\*/C 770 data> E* 771 --->E* 772 +0 ^ \d? 773 +3 ^ [A-E] 774 +8 ^^ \* 775 +10 ^ ^ 776 0: E* 777 778 If a pattern contains (*MARK) items, an additional line is output when- 779 ever a change of latest mark is passed to the callout function. For 780 example: 781 782 re> /a(*MARK:X)bc/C 783 data> abc 784 --->abc 785 +0 ^ a 786 +1 ^^ (*MARK:X) 787 +10 ^^ b 788 Latest Mark: X 789 +11 ^ ^ c 790 +12 ^ ^ 791 0: abc 792 793 The mark changes between matching "a" and "b", but stays the same for 794 the rest of the match, so nothing more is output. If, as a result of 795 backtracking, the mark reverts to being unset, the text "<unset>" is 796 output. 797 798 The callout function in pcretest returns zero (carry on matching) by 799 default, but you can use a \C item in a data line (as described above) 800 to change this and other parameters of the callout. 801 802 Inserting callouts can be helpful when using pcretest to check compli- 803 cated regular expressions. For further information about callouts, see 804 the pcrecallout documentation. 805 806 807NON-PRINTING CHARACTERS 808 809 When pcretest is outputting text in the compiled version of a pattern, 810 bytes other than 32-126 are always treated as non-printing characters 811 are are therefore shown as hex escapes. 812 813 When pcretest is outputting text that is a matched part of a subject 814 string, it behaves in the same way, unless a different locale has been 815 set for the pattern (using the /L modifier). In this case, the 816 isprint() function to distinguish printing and non-printing characters. 817 818 819SAVING AND RELOADING COMPILED PATTERNS 820 821 The facilities described in this section are not available when the 822 POSIX interface to PCRE is being used, that is, when the /P pattern 823 modifier is specified. 824 825 When the POSIX interface is not in use, you can cause pcretest to write 826 a compiled pattern to a file, by following the modifiers with > and a 827 file name. For example: 828 829 /pattern/im >/some/file 830 831 See the pcreprecompile documentation for a discussion about saving and 832 re-using compiled patterns. Note that if the pattern was successfully 833 studied with JIT optimization, the JIT data cannot be saved. 834 835 The data that is written is binary. The first eight bytes are the 836 length of the compiled pattern data followed by the length of the 837 optional study data, each written as four bytes in big-endian order 838 (most significant byte first). If there is no study data (either the 839 pattern was not studied, or studying did not return any data), the sec- 840 ond length is zero. The lengths are followed by an exact copy of the 841 compiled pattern. If there is additional study data, this (excluding 842 any JIT data) follows immediately after the compiled pattern. After 843 writing the file, pcretest expects to read a new pattern. 844 845 A saved pattern can be reloaded into pcretest by specifying < and a 846 file name instead of a pattern. The name of the file must not contain a 847 < character, as otherwise pcretest will interpret the line as a pattern 848 delimited by < characters. For example: 849 850 re> </some/file 851 Compiled pattern loaded from /some/file 852 No study data 853 854 If the pattern was previously studied with the JIT optimization, the 855 JIT information cannot be saved and restored, and so is lost. When the 856 pattern has been loaded, pcretest proceeds to read data lines in the 857 usual way. 858 859 You can copy a file written by pcretest to a different host and reload 860 it there, even if the new host has opposite endianness to the one on 861 which the pattern was compiled. For example, you can compile on an i86 862 machine and run on a SPARC machine. When a pattern is reloaded on a 863 host with different endianness, the confirmation message is changed to: 864 865 Compiled pattern (byte-inverted) loaded from /some/file 866 867 The test suite contains some saved pre-compiled patterns with different 868 endianness. These are reloaded using "<!" instead of just "<". This 869 suppresses the "(byte-inverted)" text so that the output is the same on 870 all hosts. It also forces debugging output once the pattern has been 871 reloaded. 872 873 File names for saving and reloading can be absolute or relative, but 874 note that the shell facility of expanding a file name that starts with 875 a tilde (~) is not available. 876 877 The ability to save and reload files in pcretest is intended for test- 878 ing and experimentation. It is not intended for production use because 879 only a single pattern can be written to a file. Furthermore, there is 880 no facility for supplying custom character tables for use with a 881 reloaded pattern. If the original pattern was compiled with custom 882 tables, an attempt to match a subject string using a reloaded pattern 883 is likely to cause pcretest to crash. Finally, if you attempt to load 884 a file that is not in the correct format, the result is undefined. 885 886 887SEE ALSO 888 889 pcre(3), pcre16(3), pcreapi(3), pcrecallout(3), pcrejit, pcrematch- 890 ing(3), pcrepartial(d), pcrepattern(3), pcreprecompile(3). 891 892 893AUTHOR 894 895 Philip Hazel 896 University Computing Service 897 Cambridge CB2 3QH, England. 898 899 900REVISION 901 902 Last updated: 21 February 2012 903 Copyright (c) 1997-2012 University of Cambridge. 904