|
1 .TH PCRE 3 |
|
2 .SH NAME |
|
3 PCRE - Perl-compatible regular expressions |
|
4 .SH INTRODUCTION |
|
5 .rs |
|
6 .sp |
|
7 The PCRE library is a set of functions that implement regular expression |
|
8 pattern matching using the same syntax and semantics as Perl, with just a few |
|
9 differences. Certain features that appeared in Python and PCRE before they |
|
10 appeared in Perl are also available using the Python syntax. There is also some |
|
11 support for certain .NET and Oniguruma syntax items, and there is an option for |
|
12 requesting some minor changes that give better JavaScript compatibility. |
|
13 .P |
|
14 The current implementation of PCRE (release 7.x) corresponds approximately with |
|
15 Perl 5.10, including support for UTF-8 encoded strings and Unicode general |
|
16 category properties. However, UTF-8 and Unicode support has to be explicitly |
|
17 enabled; it is not the default. The Unicode tables correspond to Unicode |
|
18 release 5.0.0. |
|
19 .P |
|
20 In addition to the Perl-compatible matching function, PCRE contains an |
|
21 alternative matching function that matches the same compiled patterns in a |
|
22 different way. In certain circumstances, the alternative function has some |
|
23 advantages. For a discussion of the two matching algorithms, see the |
|
24 .\" HREF |
|
25 \fBpcrematching\fP |
|
26 .\" |
|
27 page. |
|
28 .P |
|
29 PCRE is written in C and released as a C library. A number of people have |
|
30 written wrappers and interfaces of various kinds. In particular, Google Inc. |
|
31 have provided a comprehensive C++ wrapper. This is now included as part of the |
|
32 PCRE distribution. The |
|
33 .\" HREF |
|
34 \fBpcrecpp\fP |
|
35 .\" |
|
36 page has details of this interface. Other people's contributions can be found |
|
37 in the \fIContrib\fR directory at the primary FTP site, which is: |
|
38 .sp |
|
39 .\" HTML <a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre"> |
|
40 .\" </a> |
|
41 ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre |
|
42 .P |
|
43 Details of exactly which Perl regular expression features are and are not |
|
44 supported by PCRE are given in separate documents. See the |
|
45 .\" HREF |
|
46 \fBpcrepattern\fR |
|
47 .\" |
|
48 and |
|
49 .\" HREF |
|
50 \fBpcrecompat\fR |
|
51 .\" |
|
52 pages. There is a syntax summary in the |
|
53 .\" HREF |
|
54 \fBpcresyntax\fR |
|
55 .\" |
|
56 page. |
|
57 .P |
|
58 Some features of PCRE can be included, excluded, or changed when the library is |
|
59 built. The |
|
60 .\" HREF |
|
61 \fBpcre_config()\fR |
|
62 .\" |
|
63 function makes it possible for a client to discover which features are |
|
64 available. The features themselves are described in the |
|
65 .\" HREF |
|
66 \fBpcrebuild\fP |
|
67 .\" |
|
68 page. Documentation about building PCRE for various operating systems can be |
|
69 found in the \fBREADME\fP file in the source distribution. |
|
70 .P |
|
71 The library contains a number of undocumented internal functions and data |
|
72 tables that are used by more than one of the exported external functions, but |
|
73 which are not intended for use by external callers. Their names all begin with |
|
74 "_pcre_", which hopefully will not provoke any name clashes. In some |
|
75 environments, it is possible to control which external symbols are exported |
|
76 when a shared library is built, and in these cases the undocumented symbols are |
|
77 not exported. |
|
78 . |
|
79 . |
|
80 .SH "USER DOCUMENTATION" |
|
81 .rs |
|
82 .sp |
|
83 The user documentation for PCRE comprises a number of different sections. In |
|
84 the "man" format, each of these is a separate "man page". In the HTML format, |
|
85 each is a separate page, linked from the index page. In the plain text format, |
|
86 all the sections are concatenated, for ease of searching. The sections are as |
|
87 follows: |
|
88 .sp |
|
89 pcre this document |
|
90 pcre-config show PCRE installation configuration information |
|
91 pcreapi details of PCRE's native C API |
|
92 pcrebuild options for building PCRE |
|
93 pcrecallout details of the callout feature |
|
94 pcrecompat discussion of Perl compatibility |
|
95 pcrecpp details of the C++ wrapper |
|
96 pcregrep description of the \fBpcregrep\fP command |
|
97 pcrematching discussion of the two matching algorithms |
|
98 pcrepartial details of the partial matching facility |
|
99 .\" JOIN |
|
100 pcrepattern syntax and semantics of supported |
|
101 regular expressions |
|
102 pcresyntax quick syntax reference |
|
103 pcreperform discussion of performance issues |
|
104 pcreposix the POSIX-compatible C API |
|
105 pcreprecompile details of saving and re-using precompiled patterns |
|
106 pcresample discussion of the sample program |
|
107 pcrestack discussion of stack usage |
|
108 pcretest description of the \fBpcretest\fP testing command |
|
109 .sp |
|
110 In addition, in the "man" and HTML formats, there is a short page for each |
|
111 C library function, listing its arguments and results. |
|
112 . |
|
113 . |
|
114 .SH LIMITATIONS |
|
115 .rs |
|
116 .sp |
|
117 There are some size limitations in PCRE but it is hoped that they will never in |
|
118 practice be relevant. |
|
119 .P |
|
120 The maximum length of a compiled pattern is 65539 (sic) bytes if PCRE is |
|
121 compiled with the default internal linkage size of 2. If you want to process |
|
122 regular expressions that are truly enormous, you can compile PCRE with an |
|
123 internal linkage size of 3 or 4 (see the \fBREADME\fP file in the source |
|
124 distribution and the |
|
125 .\" HREF |
|
126 \fBpcrebuild\fP |
|
127 .\" |
|
128 documentation for details). In these cases the limit is substantially larger. |
|
129 However, the speed of execution is slower. |
|
130 .P |
|
131 All values in repeating quantifiers must be less than 65536. |
|
132 .P |
|
133 There is no limit to the number of parenthesized subpatterns, but there can be |
|
134 no more than 65535 capturing subpatterns. |
|
135 .P |
|
136 The maximum length of name for a named subpattern is 32 characters, and the |
|
137 maximum number of named subpatterns is 10000. |
|
138 .P |
|
139 The maximum length of a subject string is the largest positive number that an |
|
140 integer variable can hold. However, when using the traditional matching |
|
141 function, PCRE uses recursion to handle subpatterns and indefinite repetition. |
|
142 This means that the available stack space may limit the size of a subject |
|
143 string that can be processed by certain patterns. For a discussion of stack |
|
144 issues, see the |
|
145 .\" HREF |
|
146 \fBpcrestack\fP |
|
147 .\" |
|
148 documentation. |
|
149 . |
|
150 .\" HTML <a name="utf8support"></a> |
|
151 . |
|
152 . |
|
153 .SH "UTF-8 AND UNICODE PROPERTY SUPPORT" |
|
154 .rs |
|
155 .sp |
|
156 From release 3.3, PCRE has had some support for character strings encoded in |
|
157 the UTF-8 format. For release 4.0 this was greatly extended to cover most |
|
158 common requirements, and in release 5.0 additional support for Unicode general |
|
159 category properties was added. |
|
160 .P |
|
161 In order process UTF-8 strings, you must build PCRE to include UTF-8 support in |
|
162 the code, and, in addition, you must call |
|
163 .\" HREF |
|
164 \fBpcre_compile()\fP |
|
165 .\" |
|
166 with the PCRE_UTF8 option flag. When you do this, both the pattern and any |
|
167 subject strings that are matched against it are treated as UTF-8 strings |
|
168 instead of just strings of bytes. |
|
169 .P |
|
170 If you compile PCRE with UTF-8 support, but do not use it at run time, the |
|
171 library will be a bit bigger, but the additional run time overhead is limited |
|
172 to testing the PCRE_UTF8 flag occasionally, so should not be very big. |
|
173 .P |
|
174 If PCRE is built with Unicode character property support (which implies UTF-8 |
|
175 support), the escape sequences \ep{..}, \eP{..}, and \eX are supported. |
|
176 The available properties that can be tested are limited to the general |
|
177 category properties such as Lu for an upper case letter or Nd for a decimal |
|
178 number, the Unicode script names such as Arabic or Han, and the derived |
|
179 properties Any and L&. A full list is given in the |
|
180 .\" HREF |
|
181 \fBpcrepattern\fP |
|
182 .\" |
|
183 documentation. Only the short names for properties are supported. For example, |
|
184 \ep{L} matches a letter. Its Perl synonym, \ep{Letter}, is not supported. |
|
185 Furthermore, in Perl, many properties may optionally be prefixed by "Is", for |
|
186 compatibility with Perl 5.6. PCRE does not support this. |
|
187 . |
|
188 .\" HTML <a name="utf8strings"></a> |
|
189 . |
|
190 .SS "Validity of UTF-8 strings" |
|
191 .rs |
|
192 .sp |
|
193 When you set the PCRE_UTF8 flag, the strings passed as patterns and subjects |
|
194 are (by default) checked for validity on entry to the relevant functions. From |
|
195 release 7.3 of PCRE, the check is according the rules of RFC 3629, which are |
|
196 themselves derived from the Unicode specification. Earlier releases of PCRE |
|
197 followed the rules of RFC 2279, which allows the full range of 31-bit values (0 |
|
198 to 0x7FFFFFFF). The current check allows only values in the range U+0 to |
|
199 U+10FFFF, excluding U+D800 to U+DFFF. |
|
200 .P |
|
201 The excluded code points are the "Low Surrogate Area" of Unicode, of which the |
|
202 Unicode Standard says this: "The Low Surrogate Area does not contain any |
|
203 character assignments, consequently no character code charts or namelists are |
|
204 provided for this area. Surrogates are reserved for use with UTF-16 and then |
|
205 must be used in pairs." The code points that are encoded by UTF-16 pairs are |
|
206 available as independent code points in the UTF-8 encoding. (In other words, |
|
207 the whole surrogate thing is a fudge for UTF-16 which unfortunately messes up |
|
208 UTF-8.) |
|
209 .P |
|
210 If an invalid UTF-8 string is passed to PCRE, an error return |
|
211 (PCRE_ERROR_BADUTF8) is given. In some situations, you may already know that |
|
212 your strings are valid, and therefore want to skip these checks in order to |
|
213 improve performance. If you set the PCRE_NO_UTF8_CHECK flag at compile time or |
|
214 at run time, PCRE assumes that the pattern or subject it is given |
|
215 (respectively) contains only valid UTF-8 codes. In this case, it does not |
|
216 diagnose an invalid UTF-8 string. |
|
217 .P |
|
218 If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set, what |
|
219 happens depends on why the string is invalid. If the string conforms to the |
|
220 "old" definition of UTF-8 (RFC 2279), it is processed as a string of characters |
|
221 in the range 0 to 0x7FFFFFFF. In other words, apart from the initial validity |
|
222 test, PCRE (when in UTF-8 mode) handles strings according to the more liberal |
|
223 rules of RFC 2279. However, if the string does not even conform to RFC 2279, |
|
224 the result is undefined. Your program may crash. |
|
225 .P |
|
226 If you want to process strings of values in the full range 0 to 0x7FFFFFFF, |
|
227 encoded in a UTF-8-like manner as per the old RFC, you can set |
|
228 PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in this |
|
229 situation, you will have to apply your own validity check. |
|
230 . |
|
231 .SS "General comments about UTF-8 mode" |
|
232 .rs |
|
233 .sp |
|
234 1. An unbraced hexadecimal escape sequence (such as \exb3) matches a two-byte |
|
235 UTF-8 character if the value is greater than 127. |
|
236 .P |
|
237 2. Octal numbers up to \e777 are recognized, and match two-byte UTF-8 |
|
238 characters for values greater than \e177. |
|
239 .P |
|
240 3. Repeat quantifiers apply to complete UTF-8 characters, not to individual |
|
241 bytes, for example: \ex{100}{3}. |
|
242 .P |
|
243 4. The dot metacharacter matches one UTF-8 character instead of a single byte. |
|
244 .P |
|
245 5. The escape sequence \eC can be used to match a single byte in UTF-8 mode, |
|
246 but its use can lead to some strange effects. This facility is not available in |
|
247 the alternative matching function, \fBpcre_dfa_exec()\fP. |
|
248 .P |
|
249 6. The character escapes \eb, \eB, \ed, \eD, \es, \eS, \ew, and \eW correctly |
|
250 test characters of any code value, but the characters that PCRE recognizes as |
|
251 digits, spaces, or word characters remain the same set as before, all with |
|
252 values less than 256. This remains true even when PCRE includes Unicode |
|
253 property support, because to do otherwise would slow down PCRE in many common |
|
254 cases. If you really want to test for a wider sense of, say, "digit", you |
|
255 must use Unicode property tests such as \ep{Nd}. |
|
256 .P |
|
257 7. Similarly, characters that match the POSIX named character classes are all |
|
258 low-valued characters. |
|
259 .P |
|
260 8. However, the Perl 5.10 horizontal and vertical whitespace matching escapes |
|
261 (\eh, \eH, \ev, and \eV) do match all the appropriate Unicode characters. |
|
262 .P |
|
263 9. Case-insensitive matching applies only to characters whose values are less |
|
264 than 128, unless PCRE is built with Unicode property support. Even when Unicode |
|
265 property support is available, PCRE still uses its own character tables when |
|
266 checking the case of low-valued characters, so as not to degrade performance. |
|
267 The Unicode property information is used only for characters with higher |
|
268 values. Even when Unicode property support is available, PCRE supports |
|
269 case-insensitive matching only when there is a one-to-one mapping between a |
|
270 letter's cases. There are a small number of many-to-one mappings in Unicode; |
|
271 these are not supported by PCRE. |
|
272 . |
|
273 . |
|
274 .SH AUTHOR |
|
275 .rs |
|
276 .sp |
|
277 .nf |
|
278 Philip Hazel |
|
279 University Computing Service |
|
280 Cambridge CB2 3QH, England. |
|
281 .fi |
|
282 .P |
|
283 Putting an actual email address here seems to have been a spam magnet, so I've |
|
284 taken it away. If you want to email me, use my two initials, followed by the |
|
285 two digits 10, at the domain cam.ac.uk. |
|
286 . |
|
287 . |
|
288 .SH REVISION |
|
289 .rs |
|
290 .sp |
|
291 .nf |
|
292 Last updated: 12 April 2008 |
|
293 Copyright (c) 1997-2008 University of Cambridge. |
|
294 .fi |