1<html> 2<head> 3<title>pcre specification</title> 4</head> 5<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> 6<h1>pcre man page</h1> 7<p> 8Return to the <a href="index.html">PCRE index page</a>. 9</p> 10<p> 11This page is part of the PCRE HTML documentation. It was generated automatically 12from the original man page. If there is any nonsense in it, please consult the 13man page, in case the conversion went wrong. 14<br> 15<ul> 16<li><a name="TOC1" href="#SEC1">INTRODUCTION</a> 17<li><a name="TOC2" href="#SEC2">USER DOCUMENTATION</a> 18<li><a name="TOC3" href="#SEC3">LIMITATIONS</a> 19<li><a name="TOC4" href="#SEC4">UTF-8 AND UNICODE PROPERTY SUPPORT</a> 20<li><a name="TOC5" href="#SEC5">AUTHOR</a> 21<li><a name="TOC6" href="#SEC6">REVISION</a> 22</ul> 23<br><a name="SEC1" href="#TOC1">INTRODUCTION</a><br> 24<P> 25The PCRE library is a set of functions that implement regular expression 26pattern matching using the same syntax and semantics as Perl, with just a few 27differences. Some features that appeared in Python and PCRE before they 28appeared in Perl are also available using the Python syntax, there is some 29support for one or two .NET and Oniguruma syntax items, and there is an option 30for requesting some minor changes that give better JavaScript compatibility. 31</P> 32<P> 33The current implementation of PCRE corresponds approximately with Perl 5.10, 34including support for UTF-8 encoded strings and Unicode general category 35properties. However, UTF-8 and Unicode support has to be explicitly enabled; it 36is not the default. The Unicode tables correspond to Unicode release 5.2.0. 37</P> 38<P> 39In addition to the Perl-compatible matching function, PCRE contains an 40alternative function that matches the same compiled patterns in a different 41way. In certain circumstances, the alternative function has some advantages. 42For a discussion of the two matching algorithms, see the 43<a href="pcrematching.html"><b>pcrematching</b></a> 44page. 45</P> 46<P> 47PCRE is written in C and released as a C library. A number of people have 48written wrappers and interfaces of various kinds. In particular, Google Inc. 49have provided a comprehensive C++ wrapper. This is now included as part of the 50PCRE distribution. The 51<a href="pcrecpp.html"><b>pcrecpp</b></a> 52page has details of this interface. Other people's contributions can be found 53in the <i>Contrib</i> directory at the primary FTP site, which is: 54<a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre">ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre</a> 55</P> 56<P> 57Details of exactly which Perl regular expression features are and are not 58supported by PCRE are given in separate documents. See the 59<a href="pcrepattern.html"><b>pcrepattern</b></a> 60and 61<a href="pcrecompat.html"><b>pcrecompat</b></a> 62pages. There is a syntax summary in the 63<a href="pcresyntax.html"><b>pcresyntax</b></a> 64page. 65</P> 66<P> 67Some features of PCRE can be included, excluded, or changed when the library is 68built. The 69<a href="pcre_config.html"><b>pcre_config()</b></a> 70function makes it possible for a client to discover which features are 71available. The features themselves are described in the 72<a href="pcrebuild.html"><b>pcrebuild</b></a> 73page. Documentation about building PCRE for various operating systems can be 74found in the <b>README</b> and <b>NON-UNIX-USE</b> files in the source 75distribution. 76</P> 77<P> 78The library contains a number of undocumented internal functions and data 79tables that are used by more than one of the exported external functions, but 80which are not intended for use by external callers. Their names all begin with 81"_pcre_", which hopefully will not provoke any name clashes. In some 82environments, it is possible to control which external symbols are exported 83when a shared library is built, and in these cases the undocumented symbols are 84not exported. 85</P> 86<br><a name="SEC2" href="#TOC1">USER DOCUMENTATION</a><br> 87<P> 88The user documentation for PCRE comprises a number of different sections. In 89the "man" format, each of these is a separate "man page". In the HTML format, 90each is a separate page, linked from the index page. In the plain text format, 91all the sections, except the <b>pcredemo</b> section, are concatenated, for ease 92of searching. The sections are as follows: 93<pre> 94 pcre this document 95 pcre-config show PCRE installation configuration information 96 pcreapi details of PCRE's native C API 97 pcrebuild options for building PCRE 98 pcrecallout details of the callout feature 99 pcrecompat discussion of Perl compatibility 100 pcrecpp details of the C++ wrapper 101 pcredemo a demonstration C program that uses PCRE 102 pcregrep description of the <b>pcregrep</b> command 103 pcrematching discussion of the two matching algorithms 104 pcrepartial details of the partial matching facility 105 pcrepattern syntax and semantics of supported regular expressions 106 pcreperform discussion of performance issues 107 pcreposix the POSIX-compatible C API 108 pcreprecompile details of saving and re-using precompiled patterns 109 pcresample discussion of the pcredemo program 110 pcrestack discussion of stack usage 111 pcresyntax quick syntax reference 112 pcretest description of the <b>pcretest</b> testing command 113</pre> 114In addition, in the "man" and HTML formats, there is a short page for each 115C library function, listing its arguments and results. 116</P> 117<br><a name="SEC3" href="#TOC1">LIMITATIONS</a><br> 118<P> 119There are some size limitations in PCRE but it is hoped that they will never in 120practice be relevant. 121</P> 122<P> 123The maximum length of a compiled pattern is 65539 (sic) bytes if PCRE is 124compiled with the default internal linkage size of 2. If you want to process 125regular expressions that are truly enormous, you can compile PCRE with an 126internal linkage size of 3 or 4 (see the <b>README</b> file in the source 127distribution and the 128<a href="pcrebuild.html"><b>pcrebuild</b></a> 129documentation for details). In these cases the limit is substantially larger. 130However, the speed of execution is slower. 131</P> 132<P> 133All values in repeating quantifiers must be less than 65536. 134</P> 135<P> 136There is no limit to the number of parenthesized subpatterns, but there can be 137no more than 65535 capturing subpatterns. 138</P> 139<P> 140The maximum length of name for a named subpattern is 32 characters, and the 141maximum number of named subpatterns is 10000. 142</P> 143<P> 144The maximum length of a subject string is the largest positive number that an 145integer variable can hold. However, when using the traditional matching 146function, PCRE uses recursion to handle subpatterns and indefinite repetition. 147This means that the available stack space may limit the size of a subject 148string that can be processed by certain patterns. For a discussion of stack 149issues, see the 150<a href="pcrestack.html"><b>pcrestack</b></a> 151documentation. 152<a name="utf8support"></a></P> 153<br><a name="SEC4" href="#TOC1">UTF-8 AND UNICODE PROPERTY SUPPORT</a><br> 154<P> 155From release 3.3, PCRE has had some support for character strings encoded in 156the UTF-8 format. For release 4.0 this was greatly extended to cover most 157common requirements, and in release 5.0 additional support for Unicode general 158category properties was added. 159</P> 160<P> 161In order process UTF-8 strings, you must build PCRE to include UTF-8 support in 162the code, and, in addition, you must call 163<a href="pcre_compile.html"><b>pcre_compile()</b></a> 164with the PCRE_UTF8 option flag, or the pattern must start with the sequence 165(*UTF8). When either of these is the case, both the pattern and any subject 166strings that are matched against it are treated as UTF-8 strings instead of 167strings of 1-byte characters. 168</P> 169<P> 170If you compile PCRE with UTF-8 support, but do not use it at run time, the 171library will be a bit bigger, but the additional run time overhead is limited 172to testing the PCRE_UTF8 flag occasionally, so should not be very big. 173</P> 174<P> 175If PCRE is built with Unicode character property support (which implies UTF-8 176support), the escape sequences \p{..}, \P{..}, and \X are supported. 177The available properties that can be tested are limited to the general 178category properties such as Lu for an upper case letter or Nd for a decimal 179number, the Unicode script names such as Arabic or Han, and the derived 180properties Any and L&. A full list is given in the 181<a href="pcrepattern.html"><b>pcrepattern</b></a> 182documentation. Only the short names for properties are supported. For example, 183\p{L} matches a letter. Its Perl synonym, \p{Letter}, is not supported. 184Furthermore, in Perl, many properties may optionally be prefixed by "Is", for 185compatibility with Perl 5.6. PCRE does not support this. 186<a name="utf8strings"></a></P> 187<br><b> 188Validity of UTF-8 strings 189</b><br> 190<P> 191When you set the PCRE_UTF8 flag, the strings passed as patterns and subjects 192are (by default) checked for validity on entry to the relevant functions. From 193release 7.3 of PCRE, the check is according the rules of RFC 3629, which are 194themselves derived from the Unicode specification. Earlier releases of PCRE 195followed the rules of RFC 2279, which allows the full range of 31-bit values (0 196to 0x7FFFFFFF). The current check allows only values in the range U+0 to 197U+10FFFF, excluding U+D800 to U+DFFF. 198</P> 199<P> 200The excluded code points are the "Low Surrogate Area" of Unicode, of which the 201Unicode Standard says this: "The Low Surrogate Area does not contain any 202character assignments, consequently no character code charts or namelists are 203provided for this area. Surrogates are reserved for use with UTF-16 and then 204must be used in pairs." The code points that are encoded by UTF-16 pairs are 205available as independent code points in the UTF-8 encoding. (In other words, 206the whole surrogate thing is a fudge for UTF-16 which unfortunately messes up 207UTF-8.) 208</P> 209<P> 210If an invalid UTF-8 string is passed to PCRE, an error return 211(PCRE_ERROR_BADUTF8) is given. In some situations, you may already know that 212your strings are valid, and therefore want to skip these checks in order to 213improve performance. If you set the PCRE_NO_UTF8_CHECK flag at compile time or 214at run time, PCRE assumes that the pattern or subject it is given 215(respectively) contains only valid UTF-8 codes. In this case, it does not 216diagnose an invalid UTF-8 string. 217</P> 218<P> 219If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set, what 220happens depends on why the string is invalid. If the string conforms to the 221"old" definition of UTF-8 (RFC 2279), it is processed as a string of characters 222in the range 0 to 0x7FFFFFFF. In other words, apart from the initial validity 223test, PCRE (when in UTF-8 mode) handles strings according to the more liberal 224rules of RFC 2279. However, if the string does not even conform to RFC 2279, 225the result is undefined. Your program may crash. 226</P> 227<P> 228If you want to process strings of values in the full range 0 to 0x7FFFFFFF, 229encoded in a UTF-8-like manner as per the old RFC, you can set 230PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in this 231situation, you will have to apply your own validity check. 232</P> 233<br><b> 234General comments about UTF-8 mode 235</b><br> 236<P> 2371. An unbraced hexadecimal escape sequence (such as \xb3) matches a two-byte 238UTF-8 character if the value is greater than 127. 239</P> 240<P> 2412. Octal numbers up to \777 are recognized, and match two-byte UTF-8 242characters for values greater than \177. 243</P> 244<P> 2453. Repeat quantifiers apply to complete UTF-8 characters, not to individual 246bytes, for example: \x{100}{3}. 247</P> 248<P> 2494. The dot metacharacter matches one UTF-8 character instead of a single byte. 250</P> 251<P> 2525. The escape sequence \C can be used to match a single byte in UTF-8 mode, 253but its use can lead to some strange effects. This facility is not available in 254the alternative matching function, <b>pcre_dfa_exec()</b>. 255</P> 256<P> 2576. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly 258test characters of any code value, but the characters that PCRE recognizes as 259digits, spaces, or word characters remain the same set as before, all with 260values less than 256. This remains true even when PCRE includes Unicode 261property support, because to do otherwise would slow down PCRE in many common 262cases. If you really want to test for a wider sense of, say, "digit", you 263must use Unicode property tests such as \p{Nd}. Note that this also applies to 264\b, because it is defined in terms of \w and \W. 265</P> 266<P> 2677. Similarly, characters that match the POSIX named character classes are all 268low-valued characters. 269</P> 270<P> 2718. However, the Perl 5.10 horizontal and vertical whitespace matching escapes 272(\h, \H, \v, and \V) do match all the appropriate Unicode characters. 273</P> 274<P> 2759. Case-insensitive matching applies only to characters whose values are less 276than 128, unless PCRE is built with Unicode property support. Even when Unicode 277property support is available, PCRE still uses its own character tables when 278checking the case of low-valued characters, so as not to degrade performance. 279The Unicode property information is used only for characters with higher 280values. Even when Unicode property support is available, PCRE supports 281case-insensitive matching only when there is a one-to-one mapping between a 282letter's cases. There are a small number of many-to-one mappings in Unicode; 283these are not supported by PCRE. 284</P> 285<br><a name="SEC5" href="#TOC1">AUTHOR</a><br> 286<P> 287Philip Hazel 288<br> 289University Computing Service 290<br> 291Cambridge CB2 3QH, England. 292<br> 293</P> 294<P> 295Putting an actual email address here seems to have been a spam magnet, so I've 296taken it away. If you want to email me, use my two initials, followed by the 297two digits 10, at the domain cam.ac.uk. 298</P> 299<br><a name="SEC6" href="#TOC1">REVISION</a><br> 300<P> 301Last updated: 01 March 2010 302<br> 303Copyright © 1997-2010 University of Cambridge. 304<br> 305<p> 306Return to the <a href="index.html">PCRE index page</a>. 307</p> 308