You are viewing a plain text version of this content. The canonical link for it is here.
Posted to cvs@httpd.apache.org by pg...@apache.org on 2007/11/26 18:04:37 UTC
svn commit: r598343 [11/22] - in /httpd/httpd/vendor/pcre/current: ./ doc/
doc/html/ testdata/
Modified: httpd/httpd/vendor/pcre/current/doc/pcre.txt
URL: http://svn.apache.org/viewvc/httpd/httpd/vendor/pcre/current/doc/pcre.txt?rev=598343&r1=598342&r2=598343&view=diff
==============================================================================
--- httpd/httpd/vendor/pcre/current/doc/pcre.txt (original)
+++ httpd/httpd/vendor/pcre/current/doc/pcre.txt Mon Nov 26 09:04:19 2007
@@ -6,47 +6,32 @@
separate text files for the pcregrep and pcretest commands.
-----------------------------------------------------------------------------
-
PCRE(3) PCRE(3)
+
NAME
PCRE - Perl-compatible regular expressions
-
INTRODUCTION
The PCRE library is a set of functions that implement regular expres-
sion pattern matching using the same syntax and semantics as Perl, with
- just a few differences. (Certain features that appeared in Python and
- PCRE before they appeared in Perl are also available using the Python
- syntax.)
-
- The current implementation of PCRE (release 7.x) corresponds approxi-
- mately with Perl 5.10, including support for UTF-8 encoded strings and
- Unicode general category properties. However, UTF-8 and Unicode support
- has to be explicitly enabled; it is not the default. The Unicode tables
- correspond to Unicode release 5.0.0.
-
- In addition to the Perl-compatible matching function, PCRE contains an
- alternative matching function that matches the same compiled patterns
- in a different way. In certain circumstances, the alternative function
- has some advantages. For a discussion of the two matching algorithms,
- see the pcrematching page.
-
- PCRE is written in C and released as a C library. A number of people
- have written wrappers and interfaces of various kinds. In particular,
- Google Inc. have provided a comprehensive C++ wrapper. This is now
- included as part of the PCRE distribution. The pcrecpp page has details
- of this interface. Other people's contributions can be found in the
- Contrib directory at the primary FTP site, which is:
+ just a few differences. The current implementation of PCRE (release
+ 5.x) corresponds approximately with Perl 5.8, including support for
+ UTF-8 encoded strings and Unicode general category properties. However,
+ this support has to be explicitly enabled; it is not the default.
+
+ PCRE is written in C and released as a C library. A number of people
+ have written wrappers and interfaces of various kinds. A C++ class is
+ included in these contributions, which can be found in the Contrib
+ directory at the primary FTP site, which is:
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre
- Details of exactly which Perl regular expression features are and are
+ Details of exactly which Perl regular expression features are and are
not supported by PCRE are given in separate documents. See the pcrepat-
- tern and pcrecompat pages. There is a syntax summary in the pcresyntax
- page.
+ tern and pcrecompat pages.
Some features of PCRE can be included, excluded, or changed when the
library is built. The pcre_config() function makes it possible for a
@@ -55,14 +40,6 @@
ing PCRE for various operating systems can be found in the README file
in the source distribution.
- The library contains a number of undocumented internal functions and
- data tables that are used by more than one of the exported external
- functions, but which are not intended for use by external callers.
- Their names all begin with "_pcre_", which hopefully will not provoke
- any name clashes. In some environments, it is possible to control which
- external symbols are exported when a shared library is built, and in
- these cases the undocumented symbols are not exported.
-
USER DOCUMENTATION
@@ -73,27 +50,22 @@
of searching. The sections are as follows:
pcre this document
- pcre-config show PCRE installation configuration information
- pcreapi details of PCRE's native C API
+ pcreapi details of PCRE's native API
pcrebuild options for building PCRE
pcrecallout details of the callout feature
pcrecompat discussion of Perl compatibility
- pcrecpp details of the C++ wrapper
pcregrep description of the pcregrep command
- pcrematching discussion of the two matching algorithms
pcrepartial details of the partial matching facility
pcrepattern syntax and semantics of supported
regular expressions
- pcresyntax quick syntax reference
pcreperform discussion of performance issues
- pcreposix the POSIX-compatible C API
+ pcreposix the POSIX-compatible API
pcreprecompile details of saving and re-using precompiled patterns
pcresample discussion of the sample program
- pcrestack discussion of stack usage
pcretest description of the pcretest testing command
In addition, in the "man" and HTML formats, there is a short page for
- each C library function, listing its arguments and results.
+ each library function, listing its arguments and results.
LIMITATIONS
@@ -107,115 +79,83 @@
PCRE with an internal linkage size of 3 or 4 (see the README file in
the source distribution and the pcrebuild documentation for details).
In these cases the limit is substantially larger. However, the speed
- of execution is slower.
-
- All values in repeating quantifiers must be less than 65536.
+ of execution will be slower.
- There is no limit to the number of parenthesized subpatterns, but there
- can be no more than 65535 capturing subpatterns.
+ All values in repeating quantifiers must be less than 65536. The maxi-
+ mum number of capturing subpatterns is 65535.
- The maximum length of name for a named subpattern is 32 characters, and
- the maximum number of named subpatterns is 10000.
+ There is no limit to the number of non-capturing subpatterns, but the
+ maximum depth of nesting of all kinds of parenthesized subpattern,
+ including capturing subpatterns, assertions, and other types of subpat-
+ tern, is 200.
The maximum length of a subject string is the largest positive number
- that an integer variable can hold. However, when using the traditional
- matching function, PCRE uses recursion to handle subpatterns and indef-
- inite repetition. This means that the available stack space may limit
- the size of a subject string that can be processed by certain patterns.
- For a discussion of stack issues, see the pcrestack documentation.
+ that an integer variable can hold. However, PCRE uses recursion to han-
+ dle subpatterns and indefinite repetition. This means that the avail-
+ able stack space may limit the size of a subject string that can be
+ processed by certain patterns.
UTF-8 AND UNICODE PROPERTY SUPPORT
- From release 3.3, PCRE has had some support for character strings
- encoded in the UTF-8 format. For release 4.0 this was greatly extended
- to cover most common requirements, and in release 5.0 additional sup-
+ From release 3.3, PCRE has had some support for character strings
+ encoded in the UTF-8 format. For release 4.0 this was greatly extended
+ to cover most common requirements, and in release 5.0 additional sup-
port for Unicode general category properties was added.
- In order process UTF-8 strings, you must build PCRE to include UTF-8
- support in the code, and, in addition, you must call pcre_compile()
- with the PCRE_UTF8 option flag. When you do this, both the pattern and
- any subject strings that are matched against it are treated as UTF-8
+ In order process UTF-8 strings, you must build PCRE to include UTF-8
+ support in the code, and, in addition, you must call pcre_compile()
+ with the PCRE_UTF8 option flag. When you do this, both the pattern and
+ any subject strings that are matched against it are treated as UTF-8
strings instead of just strings of bytes.
- If you compile PCRE with UTF-8 support, but do not use it at run time,
- the library will be a bit bigger, but the additional run time overhead
- is limited to testing the PCRE_UTF8 flag occasionally, so should not be
- very big.
+ If you compile PCRE with UTF-8 support, but do not use it at run time,
+ the library will be a bit bigger, but the additional run time overhead
+ is limited to testing the PCRE_UTF8 flag in several places, so should
+ not be very large.
If PCRE is built with Unicode character property support (which implies
- UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup-
+ UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup-
ported. The available properties that can be tested are limited to the
- general category properties such as Lu for an upper case letter or Nd
- for a decimal number, the Unicode script names such as Arabic or Han,
- and the derived properties Any and L&. A full list is given in the
- pcrepattern documentation. Only the short names for properties are sup-
- ported. For example, \p{L} matches a letter. Its Perl synonym, \p{Let-
- ter}, is not supported. Furthermore, in Perl, many properties may
- optionally be prefixed by "Is", for compatibility with Perl 5.6. PCRE
- does not support this.
-
- Validity of UTF-8 strings
-
- When you set the PCRE_UTF8 flag, the strings passed as patterns and
- subjects are (by default) checked for validity on entry to the relevant
- functions. From release 7.3 of PCRE, the check is according the rules
- of RFC 3629, which are themselves derived from the Unicode specifica-
- tion. Earlier releases of PCRE followed the rules of RFC 2279, which
- allows the full range of 31-bit values (0 to 0x7FFFFFFF). The current
- check allows only values in the range U+0 to U+10FFFF, excluding U+D800
- to U+DFFF.
-
- The excluded code points are the "Low Surrogate Area" of Unicode, of
- which the Unicode Standard says this: "The Low Surrogate Area does not
- contain any character assignments, consequently no character code
- charts or namelists are provided for this area. Surrogates are reserved
- for use with UTF-16 and then must be used in pairs." The code points
- that are encoded by UTF-16 pairs are available as independent code
- points in the UTF-8 encoding. (In other words, the whole surrogate
- thing is a fudge for UTF-16 which unfortunately messes up UTF-8.)
-
- If an invalid UTF-8 string is passed to PCRE, an error return
- (PCRE_ERROR_BADUTF8) is given. In some situations, you may already know
- that your strings are valid, and therefore want to skip these checks in
- order to improve performance. If you set the PCRE_NO_UTF8_CHECK flag at
- compile time or at run time, PCRE assumes that the pattern or subject
- it is given (respectively) contains only valid UTF-8 codes. In this
- case, it does not diagnose an invalid UTF-8 string.
-
- If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set,
- what happens depends on why the string is invalid. If the string con-
- forms to the "old" definition of UTF-8 (RFC 2279), it is processed as a
- string of characters in the range 0 to 0x7FFFFFFF. In other words,
- apart from the initial validity test, PCRE (when in UTF-8 mode) handles
- strings according to the more liberal rules of RFC 2279. However, if
- the string does not even conform to RFC 2279, the result is undefined.
- Your program may crash.
-
- If you want to process strings of values in the full range 0 to
- 0x7FFFFFFF, encoded in a UTF-8-like manner as per the old RFC, you can
- set PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in
- this situation, you will have to apply your own validity check.
-
- General comments about UTF-8 mode
+ general category properties such as Lu for an upper case letter or Nd
+ for a decimal number. A full list is given in the pcrepattern documen-
+ tation. The PCRE library is increased in size by about 90K when Unicode
+ property support is included.
+
+ The following comments apply when PCRE is running in UTF-8 mode:
+
+ 1. When you set the PCRE_UTF8 flag, the strings passed as patterns and
+ subjects are checked for validity on entry to the relevant functions.
+ If an invalid UTF-8 string is passed, an error return is given. In some
+ situations, you may already know that your strings are valid, and
+ therefore want to skip these checks in order to improve performance. If
+ you set the PCRE_NO_UTF8_CHECK flag at compile time or at run time,
+ PCRE assumes that the pattern or subject it is given (respectively)
+ contains only valid UTF-8 codes. In this case, it does not diagnose an
+ invalid UTF-8 string. If you pass an invalid UTF-8 string to PCRE when
+ PCRE_NO_UTF8_CHECK is set, the results are undefined. Your program may
+ crash.
+
+ 2. In a pattern, the escape sequence \x{...}, where the contents of the
+ braces is a string of hexadecimal digits, is interpreted as a UTF-8
+ character whose code number is the given hexadecimal number, for exam-
+ ple: \x{1234}. If a non-hexadecimal digit appears between the braces,
+ the item is not recognized. This escape sequence can be used either as
+ a literal, or within a character class.
- 1. An unbraced hexadecimal escape sequence (such as \xb3) matches a
- two-byte UTF-8 character if the value is greater than 127.
+ 3. The original hexadecimal escape sequence, \xhh, matches a two-byte
+ UTF-8 character if the value is greater than 127.
- 2. Octal numbers up to \777 are recognized, and match two-byte UTF-8
- characters for values greater than \177.
-
- 3. Repeat quantifiers apply to complete UTF-8 characters, not to indi-
+ 4. Repeat quantifiers apply to complete UTF-8 characters, not to indi-
vidual bytes, for example: \x{100}{3}.
- 4. The dot metacharacter matches one UTF-8 character instead of a sin-
+ 5. The dot metacharacter matches one UTF-8 character instead of a sin-
gle byte.
- 5. The escape sequence \C can be used to match a single byte in UTF-8
- mode, but its use can lead to some strange effects. This facility is
- not available in the alternative matching function, pcre_dfa_exec().
+ 6. The escape sequence \C can be used to match a single byte in UTF-8
+ mode, but its use can lead to some strange effects.
- 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly
+ 7. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly
test characters of any code value, but the characters that PCRE recog-
nizes as digits, spaces, or word characters remain the same set as
before, all with values less than 256. This remains true even when PCRE
@@ -224,83 +164,52 @@
sense of, say, "digit", you must use Unicode property tests such as
\p{Nd}.
- 7. Similarly, characters that match the POSIX named character classes
+ 8. Similarly, characters that match the POSIX named character classes
are all low-valued characters.
- 8. However, the Perl 5.10 horizontal and vertical whitespace matching
- escapes (\h, \H, \v, and \V) do match all the appropriate Unicode char-
- acters.
-
9. Case-insensitive matching applies only to characters whose values
are less than 128, unless PCRE is built with Unicode property support.
Even when Unicode property support is available, PCRE still uses its
own character tables when checking the case of low-valued characters,
so as not to degrade performance. The Unicode property information is
- used only for characters with higher values. Even when Unicode property
- support is available, PCRE supports case-insensitive matching only when
- there is a one-to-one mapping between a letter's cases. There are a
- small number of many-to-one mappings in Unicode; these are not sup-
- ported by PCRE.
+ used only for characters with higher values.
AUTHOR
- Philip Hazel
- University Computing Service
- Cambridge CB2 3QH, England.
-
- Putting an actual email address here seems to have been a spam magnet,
- so I've taken it away. If you want to email me, use my two initials,
- followed by the two digits 10, at the domain cam.ac.uk.
-
-
-REVISION
-
- Last updated: 09 August 2007
- Copyright (c) 1997-2007 University of Cambridge.
-------------------------------------------------------------------------------
+ Philip Hazel <ph...@cam.ac.uk>
+ University Computing Service,
+ Cambridge CB2 3QG, England.
+ Phone: +44 1223 334714
+
+Last updated: 09 September 2004
+Copyright (c) 1997-2004 University of Cambridge.
+-----------------------------------------------------------------------------
+PCRE(3) PCRE(3)
-PCREBUILD(3) PCREBUILD(3)
NAME
PCRE - Perl-compatible regular expressions
-
PCRE BUILD-TIME OPTIONS
This document describes the optional features of PCRE that can be
- selected when the library is compiled. It assumes use of the configure
- script, where the optional features are selected or deselected by pro-
- viding options to configure before running the make command. However,
- the same options can be selected in both Unix-like and non-Unix-like
- environments using the GUI facility of CMakeSetup if you are using
- CMake instead of configure to build PCRE.
-
- The complete list of options for configure (which includes the standard
- ones such as the selection of the installation directory) can be
- obtained by running
+ selected when the library is compiled. They are all selected, or dese-
+ lected, by providing options to the configure script that is run before
+ the make command. The complete list of options for configure (which
+ includes the standard ones such as the selection of the installation
+ directory) can be obtained by running
./configure --help
- The following sections include descriptions of options whose names
- begin with --enable or --disable. These settings specify changes to the
- defaults for the configure command. Because of the way that configure
- works, --enable and --disable always come in pairs, so the complemen-
- tary option always exists as well, but as it specifies the default, it
- is not described.
-
-
-C++ SUPPORT
-
- By default, the configure script will search for a C++ compiler and C++
- header files. If it finds them, it automatically builds the C++ wrapper
- library for PCRE. You can disable this by adding
-
- --disable-cpp
-
- to the configure command.
+ The following sections describe certain options whose names begin with
+ --enable or --disable. These settings specify changes to the defaults
+ for the configure command. Because of the way that configure works,
+ --enable and --disable always come in pairs, so the complementary
+ option always exists as well, but as it specifies the default, it is
+ not described.
UTF-8 SUPPORT
@@ -309,80 +218,48 @@
--enable-utf8
- to the configure command. Of itself, this does not make PCRE treat
- strings as UTF-8. As well as compiling PCRE with this option, you also
- have have to set the PCRE_UTF8 option when you call the pcre_compile()
+ to the configure command. Of itself, this does not make PCRE treat
+ strings as UTF-8. As well as compiling PCRE with this option, you also
+ have have to set the PCRE_UTF8 option when you call the pcre_compile()
function.
UNICODE CHARACTER PROPERTY SUPPORT
- UTF-8 support allows PCRE to process character values greater than 255
- in the strings that it handles. On its own, however, it does not pro-
+ UTF-8 support allows PCRE to process character values greater than 255
+ in the strings that it handles. On its own, however, it does not pro-
vide any facilities for accessing the properties of such characters. If
- you want to be able to use the pattern escapes \P, \p, and \X, which
+ you want to be able to use the pattern escapes \P, \p, and \X, which
refer to Unicode character properties, you must add
--enable-unicode-properties
- to the configure command. This implies UTF-8 support, even if you have
+ to the configure command. This implies UTF-8 support, even if you have
not explicitly requested it.
- Including Unicode property support adds around 30K of tables to the
- PCRE library. Only the general category properties such as Lu and Nd
- are supported. Details are given in the pcrepattern documentation.
+ Including Unicode property support adds around 90K of tables to the
+ PCRE library, approximately doubling its size. Only the general cate-
+ gory properties such as Lu and Nd are supported. Details are given in
+ the pcrepattern documentation.
CODE VALUE OF NEWLINE
- By default, PCRE interprets character 10 (linefeed, LF) as indicating
- the end of a line. This is the normal newline character on Unix-like
- systems. You can compile PCRE to use character 13 (carriage return, CR)
- instead, by adding
+ By default, PCRE treats character 10 (linefeed) as the newline charac-
+ ter. This is the normal newline character on Unix-like systems. You can
+ compile PCRE to use character 13 (carriage return) instead by adding
--enable-newline-is-cr
- to the configure command. There is also a --enable-newline-is-lf
- option, which explicitly specifies linefeed as the newline character.
-
- Alternatively, you can specify that line endings are to be indicated by
- the two character sequence CRLF. If you want this, add
-
- --enable-newline-is-crlf
-
- to the configure command. There is a fourth option, specified by
-
- --enable-newline-is-anycrlf
-
- which causes PCRE to recognize any of the three sequences CR, LF, or
- CRLF as indicating a line ending. Finally, a fifth option, specified by
-
- --enable-newline-is-any
-
- causes PCRE to recognize any Unicode newline sequence.
-
- Whatever line ending convention is selected when PCRE is built can be
- overridden when the library functions are called. At build time it is
- conventional to use the standard for your operating system.
-
-
-WHAT \R MATCHES
-
- By default, the sequence \R in a pattern matches any Unicode newline
- sequence, whatever has been selected as the line ending sequence. If
- you specify
-
- --enable-bsr-anycrlf
-
- the default is changed so that \R matches only CR, LF, or CRLF. What-
- ever is selected when PCRE is built can be overridden when the library
- functions are called.
+ to the configure command. For completeness there is also a --enable-
+ newline-is-lf option, which explicitly specifies linefeed as the new-
+ line character.
BUILDING SHARED AND STATIC LIBRARIES
- The PCRE building process uses libtool to build both shared and static
- Unix libraries by default. You can suppress one of these by adding one
+ The PCRE building process uses libtool to build both shared and static
+ Unix libraries by default. You can suppress one of these by adding one
of
--disable-shared
@@ -394,9 +271,9 @@
POSIX MALLOC USAGE
When PCRE is called through the POSIX interface (see the pcreposix doc-
- umentation), additional working storage is required for holding the
- pointers to capturing substrings, because PCRE requires three integers
- per substring, whereas the POSIX interface provides only two. If the
+ umentation), additional working storage is required for holding the
+ pointers to capturing substrings, because PCRE requires three integers
+ per substring, whereas the POSIX interface provides only two. If the
number of expected substrings is small, the wrapper function uses space
on the stack, because this is faster than using malloc() for each call.
The default threshold above which the stack is no longer used is 10; it
@@ -407,340 +284,89 @@
to the configure command.
+LIMITING PCRE RESOURCE USAGE
+
+ Internally, PCRE has a function called match(), which it calls repeat-
+ edly (possibly recursively) when matching a pattern. By controlling the
+ maximum number of times this function may be called during a single
+ matching operation, a limit can be placed on the resources used by a
+ single call to pcre_exec(). The limit can be changed at run time, as
+ described in the pcreapi documentation. The default is 10 million, but
+ this can be changed by adding a setting such as
+
+ --with-match-limit=500000
+
+ to the configure command.
+
+
HANDLING VERY LARGE PATTERNS
- Within a compiled pattern, offset values are used to point from one
- part to another (for example, from an opening parenthesis to an alter-
- nation metacharacter). By default, two-byte values are used for these
- offsets, leading to a maximum size for a compiled pattern of around
- 64K. This is sufficient to handle all but the most gigantic patterns.
- Nevertheless, some people do want to process enormous patterns, so it
- is possible to compile PCRE to use three-byte or four-byte offsets by
+ Within a compiled pattern, offset values are used to point from one
+ part to another (for example, from an opening parenthesis to an alter-
+ nation metacharacter). By default, two-byte values are used for these
+ offsets, leading to a maximum size for a compiled pattern of around
+ 64K. This is sufficient to handle all but the most gigantic patterns.
+ Nevertheless, some people do want to process enormous patterns, so it
+ is possible to compile PCRE to use three-byte or four-byte offsets by
adding a setting such as
--with-link-size=3
- to the configure command. The value given must be 2, 3, or 4. Using
- longer offsets slows down the operation of PCRE because it has to load
+ to the configure command. The value given must be 2, 3, or 4. Using
+ longer offsets slows down the operation of PCRE because it has to load
additional bytes when handling them.
+ If you build PCRE with an increased link size, test 2 (and test 5 if
+ you are using UTF-8) will fail. Part of the output of these tests is a
+ representation of the compiled pattern, and this changes with the link
+ size.
+
AVOIDING EXCESSIVE STACK USAGE
- When matching with the pcre_exec() function, PCRE implements backtrack-
- ing by making recursive calls to an internal function called match().
- In environments where the size of the stack is limited, this can se-
- verely limit PCRE's operation. (The Unix environment does not usually
- suffer from this problem, but it may sometimes be necessary to increase
- the maximum stack size. There is a discussion in the pcrestack docu-
- mentation.) An alternative approach to recursion that uses memory from
- the heap to remember data, instead of using recursive function calls,
- has been implemented to work round the problem of limited stack size.
- If you want to build a version of PCRE that works this way, add
+ PCRE implements backtracking while matching by making recursive calls
+ to an internal function called match(). In environments where the size
+ of the stack is limited, this can severely limit PCRE's operation. (The
+ Unix environment does not usually suffer from this problem.) An alter-
+ native approach that uses memory from the heap to remember data,
+ instead of using recursive function calls, has been implemented to work
+ round this problem. If you want to build a version of PCRE that works
+ this way, add
--disable-stack-for-recursion
to the configure command. With this configuration, PCRE will use the
pcre_stack_malloc and pcre_stack_free variables to call memory manage-
- ment functions. By default these point to malloc() and free(), but you
- can replace the pointers so that your own functions are used.
-
- Separate functions are provided rather than using pcre_malloc and
- pcre_free because the usage is very predictable: the block sizes
- requested are always the same, and the blocks are always freed in
- reverse order. A calling program might be able to implement optimized
- functions that perform better than malloc() and free(). PCRE runs
- noticeably more slowly when built in this way. This option affects only
- the pcre_exec() function; it is not relevant for the the
- pcre_dfa_exec() function.
-
-
-LIMITING PCRE RESOURCE USAGE
-
- Internally, PCRE has a function called match(), which it calls repeat-
- edly (sometimes recursively) when matching a pattern with the
- pcre_exec() function. By controlling the maximum number of times this
- function may be called during a single matching operation, a limit can
- be placed on the resources used by a single call to pcre_exec(). The
- limit can be changed at run time, as described in the pcreapi documen-
- tation. The default is 10 million, but this can be changed by adding a
- setting such as
-
- --with-match-limit=500000
-
- to the configure command. This setting has no effect on the
- pcre_dfa_exec() matching function.
-
- In some environments it is desirable to limit the depth of recursive
- calls of match() more strictly than the total number of calls, in order
- to restrict the maximum amount of stack (or heap, if --disable-stack-
- for-recursion is specified) that is used. A second limit controls this;
- it defaults to the value that is set for --with-match-limit, which
- imposes no additional constraints. However, you can set a lower limit
- by adding, for example,
-
- --with-match-limit-recursion=10000
-
- to the configure command. This value can also be overridden at run
- time.
-
-
-CREATING CHARACTER TABLES AT BUILD TIME
-
- PCRE uses fixed tables for processing characters whose code values are
- less than 256. By default, PCRE is built with a set of tables that are
- distributed in the file pcre_chartables.c.dist. These tables are for
- ASCII codes only. If you add
-
- --enable-rebuild-chartables
-
- to the configure command, the distributed tables are no longer used.
- Instead, a program called dftables is compiled and run. This outputs
- the source for new set of tables, created in the default locale of your
- C runtime system. (This method of replacing the tables does not work if
- you are cross compiling, because dftables is run on the local host. If
- you need to create alternative tables when cross compiling, you will
- have to do so "by hand".)
+ ment functions. Separate functions are provided because the usage is
+ very predictable: the block sizes requested are always the same, and
+ the blocks are always freed in reverse order. A calling program might
+ be able to implement optimized functions that perform better than the
+ standard malloc() and free() functions. PCRE runs noticeably more
+ slowly when built in this way.
USING EBCDIC CODE
- PCRE assumes by default that it will run in an environment where the
- character code is ASCII (or Unicode, which is a superset of ASCII).
- This is the case for most computer operating systems. PCRE can, how-
- ever, be compiled to run in an EBCDIC environment by adding
+ PCRE assumes by default that it will run in an environment where the
+ character code is ASCII (or Unicode, which is a superset of ASCII).
+ PCRE can, however, be compiled to run in an EBCDIC environment by
+ adding
--enable-ebcdic
- to the configure command. This setting implies --enable-rebuild-charta-
- bles. You should only use it if you know that you are in an EBCDIC
- environment (for example, an IBM mainframe operating system).
-
-
-SEE ALSO
-
- pcreapi(3), pcre_config(3).
-
-
-AUTHOR
-
- Philip Hazel
- University Computing Service
- Cambridge CB2 3QH, England.
-
-
-REVISION
-
- Last updated: 21 September 2007
- Copyright (c) 1997-2007 University of Cambridge.
-------------------------------------------------------------------------------
-
-
-PCREMATCHING(3) PCREMATCHING(3)
-
-
-NAME
- PCRE - Perl-compatible regular expressions
-
-
-PCRE MATCHING ALGORITHMS
-
- This document describes the two different algorithms that are available
- in PCRE for matching a compiled regular expression against a given sub-
- ject string. The "standard" algorithm is the one provided by the
- pcre_exec() function. This works in the same was as Perl's matching
- function, and provides a Perl-compatible matching operation.
-
- An alternative algorithm is provided by the pcre_dfa_exec() function;
- this operates in a different way, and is not Perl-compatible. It has
- advantages and disadvantages compared with the standard algorithm, and
- these are described below.
-
- When there is only one possible way in which a given subject string can
- match a pattern, the two algorithms give the same answer. A difference
- arises, however, when there are multiple possibilities. For example, if
- the pattern
-
- ^<.*>
-
- is matched against the string
-
- <something> <something else> <something further>
-
- there are three possible answers. The standard algorithm finds only one
- of them, whereas the alternative algorithm finds all three.
-
-
-REGULAR EXPRESSIONS AS TREES
-
- The set of strings that are matched by a regular expression can be rep-
- resented as a tree structure. An unlimited repetition in the pattern
- makes the tree of infinite size, but it is still a tree. Matching the
- pattern to a given subject string (from a given starting point) can be
- thought of as a search of the tree. There are two ways to search a
- tree: depth-first and breadth-first, and these correspond to the two
- matching algorithms provided by PCRE.
-
-
-THE STANDARD MATCHING ALGORITHM
-
- In the terminology of Jeffrey Friedl's book "Mastering Regular Expres-
- sions", the standard algorithm is an "NFA algorithm". It conducts a
- depth-first search of the pattern tree. That is, it proceeds along a
- single path through the tree, checking that the subject matches what is
- required. When there is a mismatch, the algorithm tries any alterna-
- tives at the current point, and if they all fail, it backs up to the
- previous branch point in the tree, and tries the next alternative
- branch at that level. This often involves backing up (moving to the
- left) in the subject string as well. The order in which repetition
- branches are tried is controlled by the greedy or ungreedy nature of
- the quantifier.
-
- If a leaf node is reached, a matching string has been found, and at
- that point the algorithm stops. Thus, if there is more than one possi-
- ble match, this algorithm returns the first one that it finds. Whether
- this is the shortest, the longest, or some intermediate length depends
- on the way the greedy and ungreedy repetition quantifiers are specified
- in the pattern.
-
- Because it ends up with a single path through the tree, it is rela-
- tively straightforward for this algorithm to keep track of the sub-
- strings that are matched by portions of the pattern in parentheses.
- This provides support for capturing parentheses and back references.
-
-
-THE ALTERNATIVE MATCHING ALGORITHM
-
- This algorithm conducts a breadth-first search of the tree. Starting
- from the first matching point in the subject, it scans the subject
- string from left to right, once, character by character, and as it does
- this, it remembers all the paths through the tree that represent valid
- matches. In Friedl's terminology, this is a kind of "DFA algorithm",
- though it is not implemented as a traditional finite state machine (it
- keeps multiple states active simultaneously).
-
- The scan continues until either the end of the subject is reached, or
- there are no more unterminated paths. At this point, terminated paths
- represent the different matching possibilities (if there are none, the
- match has failed). Thus, if there is more than one possible match,
- this algorithm finds all of them, and in particular, it finds the long-
- est. In PCRE, there is an option to stop the algorithm after the first
- match (which is necessarily the shortest) has been found.
-
- Note that all the matches that are found start at the same point in the
- subject. If the pattern
-
- cat(er(pillar)?)
-
- is matched against the string "the caterpillar catchment", the result
- will be the three strings "cat", "cater", and "caterpillar" that start
- at the fourth character of the subject. The algorithm does not automat-
- ically move on to find matches that start at later positions.
-
- There are a number of features of PCRE regular expressions that are not
- supported by the alternative matching algorithm. They are as follows:
-
- 1. Because the algorithm finds all possible matches, the greedy or
- ungreedy nature of repetition quantifiers is not relevant. Greedy and
- ungreedy quantifiers are treated in exactly the same way. However, pos-
- sessive quantifiers can make a difference when what follows could also
- match what is quantified, for example in a pattern like this:
-
- ^a++\w!
-
- This pattern matches "aaab!" but not "aaa!", which would be matched by
- a non-possessive quantifier. Similarly, if an atomic group is present,
- it is matched as if it were a standalone pattern at the current point,
- and the longest match is then "locked in" for the rest of the overall
- pattern.
-
- 2. When dealing with multiple paths through the tree simultaneously, it
- is not straightforward to keep track of captured substrings for the
- different matching possibilities, and PCRE's implementation of this
- algorithm does not attempt to do this. This means that no captured sub-
- strings are available.
-
- 3. Because no substrings are captured, back references within the pat-
- tern are not supported, and cause errors if encountered.
-
- 4. For the same reason, conditional expressions that use a backrefer-
- ence as the condition or test for a specific group recursion are not
- supported.
-
- 5. Because many paths through the tree may be active, the \K escape
- sequence, which resets the start of the match when encountered (but may
- be on some paths and not on others), is not supported. It causes an
- error if encountered.
-
- 6. Callouts are supported, but the value of the capture_top field is
- always 1, and the value of the capture_last field is always -1.
-
- 7. The \C escape sequence, which (in the standard algorithm) matches a
- single byte, even in UTF-8 mode, is not supported because the alterna-
- tive algorithm moves through the subject string one character at a
- time, for all active paths through the tree.
-
- 8. None of the backtracking control verbs such as (*PRUNE) are sup-
- ported.
-
-
-ADVANTAGES OF THE ALTERNATIVE ALGORITHM
-
- Using the alternative matching algorithm provides the following advan-
- tages:
-
- 1. All possible matches (at a single point in the subject) are automat-
- ically found, and in particular, the longest match is found. To find
- more than one match using the standard algorithm, you have to do kludgy
- things with callouts.
-
- 2. There is much better support for partial matching. The restrictions
- on the content of the pattern that apply when using the standard algo-
- rithm for partial matching do not apply to the alternative algorithm.
- For non-anchored patterns, the starting position of a partial match is
- available.
-
- 3. Because the alternative algorithm scans the subject string just
- once, and never needs to backtrack, it is possible to pass very long
- subject strings to the matching function in several pieces, checking
- for partial matching each time.
-
-
-DISADVANTAGES OF THE ALTERNATIVE ALGORITHM
-
- The alternative algorithm suffers from a number of disadvantages:
-
- 1. It is substantially slower than the standard algorithm. This is
- partly because it has to search for all possible matches, but is also
- because it is less susceptible to optimization.
-
- 2. Capturing parentheses and back references are not supported.
-
- 3. Although atomic groups are supported, their use does not provide the
- performance advantage that it does for the standard algorithm.
-
-
-AUTHOR
-
- Philip Hazel
- University Computing Service
- Cambridge CB2 3QH, England.
-
-
-REVISION
+ to the configure command.
- Last updated: 08 August 2007
- Copyright (c) 1997-2007 University of Cambridge.
-------------------------------------------------------------------------------
+Last updated: 09 September 2004
+Copyright (c) 1997-2004 University of Cambridge.
+-----------------------------------------------------------------------------
+PCRE(3) PCRE(3)
-PCREAPI(3) PCREAPI(3)
NAME
PCRE - Perl-compatible regular expressions
-
PCRE NATIVE API
#include <pcre.h>
@@ -749,11 +375,6 @@
const char **errptr, int *erroffset,
const unsigned char *tableptr);
- pcre *pcre_compile2(const char *pattern, int options,
- int *errorcodeptr,
- const char **errptr, int *erroffset,
- const unsigned char *tableptr);
-
pcre_extra *pcre_study(const pcre *code, int options,
const char **errptr);
@@ -761,11 +382,6 @@
const char *subject, int length, int startoffset,
int options, int *ovector, int ovecsize);
- int pcre_dfa_exec(const pcre *code, const pcre_extra *extra,
- const char *subject, int length, int startoffset,
- int options, int *ovector, int ovecsize,
- int *workspace, int wscount);
-
int pcre_copy_named_substring(const pcre *code,
const char *subject, int *ovector,
int stringcount, const char *stringname,
@@ -783,9 +399,6 @@
int pcre_get_stringnumber(const pcre *code,
const char *name);
- int pcre_get_stringtable_entries(const pcre *code,
- const char *name, char **first, char **last);
-
int pcre_get_substring(const char *subject, int *ovector,
int stringcount, int stringnumber,
const char **stringptr);
@@ -804,8 +417,6 @@
int pcre_info(const pcre *code, int *optptr, int *firstcharptr);
- int pcre_refcount(pcre *code, int adjust);
-
int pcre_config(int what, void *where);
char *pcre_version(void);
@@ -824,37 +435,26 @@
PCRE API OVERVIEW
PCRE has its own native API, which is described in this document. There
- are also some wrapper functions that correspond to the POSIX regular
- expression API. These are described in the pcreposix documentation.
- Both of these APIs define a set of C function calls. A C++ wrapper is
- distributed with PCRE. It is documented in the pcrecpp page.
+ is also a set of wrapper functions that correspond to the POSIX regular
+ expression API. These are described in the pcreposix documentation.
- The native API C function prototypes are defined in the header file
- pcre.h, and on Unix systems the library itself is called libpcre. It
+ The native API function prototypes are defined in the header file
+ pcre.h, and on Unix systems the library itself is called libpcre. It
can normally be accessed by adding -lpcre to the command for linking an
application that uses PCRE. The header file defines the macros
PCRE_MAJOR and PCRE_MINOR to contain the major and minor release num-
bers for the library. Applications can use these to include support
for different releases of PCRE.
- The functions pcre_compile(), pcre_compile2(), pcre_study(), and
- pcre_exec() are used for compiling and matching regular expressions in
- a Perl-compatible manner. A sample program that demonstrates the sim-
- plest way of using them is provided in the file called pcredemo.c in
- the source distribution. The pcresample documentation describes how to
- run it.
-
- A second matching function, pcre_dfa_exec(), which is not Perl-compati-
- ble, is also provided. This uses a different algorithm for the match-
- ing. The alternative algorithm finds all possible matches (at a given
- point in the subject), and scans the subject just once. However, this
- algorithm does not return captured substrings. A description of the two
- matching algorithms and their advantages and disadvantages is given in
- the pcrematching documentation.
-
- In addition to the main compiling and matching functions, there are
- convenience functions for extracting captured substrings from a subject
- string that is matched by pcre_exec(). They are:
+ The functions pcre_compile(), pcre_study(), and pcre_exec() are used
+ for compiling and matching regular expressions. A sample program that
+ demonstrates the simplest way of using them is provided in the file
+ called pcredemo.c in the source distribution. The pcresample documenta-
+ tion describes how to run it.
+
+ In addition to the main compiling and matching functions, there are
+ convenience functions for extracting captured substrings from a matched
+ subject string. They are:
pcre_copy_substring()
pcre_copy_named_substring()
@@ -862,17 +462,15 @@
pcre_get_named_substring()
pcre_get_substring_list()
pcre_get_stringnumber()
- pcre_get_stringtable_entries()
pcre_free_substring() and pcre_free_substring_list() are also provided,
to free the memory used for extracted strings.
- The function pcre_maketables() is used to build a set of character
- tables in the current locale for passing to pcre_compile(),
- pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is
- provided for specialist use. Most commonly, no special tables are
- passed, in which case internal tables that are generated when PCRE is
- built are used.
+ The function pcre_maketables() is used to build a set of character
+ tables in the current locale for passing to pcre_compile() or
+ pcre_exec(). This is an optional facility that is provided for spe-
+ cialist use. Most commonly, no special tables are passed, in which case
+ internal tables that are generated when PCRE is built are used.
The function pcre_fullinfo() is used to find out information about a
compiled pattern; pcre_info() is an obsolete version that returns only
@@ -880,10 +478,6 @@
patibility. The function pcre_version() returns a pointer to a string
containing the version of PCRE and its date of release.
- The function pcre_refcount() maintains a reference count in a data
- block containing a compiled pattern. This is provided for the benefit
- of object-oriented applications.
-
The global variables pcre_malloc and pcre_free initially contain the
entry points of the standard malloc() and free() functions, respec-
tively. PCRE calls the memory management functions via these variables,
@@ -893,57 +487,20 @@
The global variables pcre_stack_malloc and pcre_stack_free are also
indirections to memory management functions. These special functions
are used only when PCRE is compiled to use the heap for remembering
- data, instead of recursive function calls, when running the pcre_exec()
- function. See the pcrebuild documentation for details of how to do
- this. It is a non-standard way of building PCRE, for use in environ-
- ments that have limited stacks. Because of the greater use of memory
- management, it runs more slowly. Separate functions are provided so
- that special-purpose external code can be used for this case. When
- used, these functions are always called in a stack-like manner (last
- obtained, first freed), and always for memory blocks of the same size.
- There is a discussion about PCRE's stack usage in the pcrestack docu-
- mentation.
+ data, instead of recursive function calls. This is a non-standard way
+ of building PCRE, for use in environments that have limited stacks.
+ Because of the greater use of memory management, it runs more slowly.
+ Separate functions are provided so that special-purpose external code
+ can be used for this case. When used, these functions are always called
+ in a stack-like manner (last obtained, first freed), and always for
+ memory blocks of the same size.
The global variable pcre_callout initially contains NULL. It can be set
- by the caller to a "callout" function, which PCRE will then call at
- specified points during a matching operation. Details are given in the
+ by the caller to a "callout" function, which PCRE will then call at
+ specified points during a matching operation. Details are given in the
pcrecallout documentation.
-NEWLINES
-
- PCRE supports five different conventions for indicating line breaks in
- strings: a single CR (carriage return) character, a single LF (line-
- feed) character, the two-character sequence CRLF, any of the three pre-
- ceding, or any Unicode newline sequence. The Unicode newline sequences
- are the three just mentioned, plus the single characters VT (vertical
- tab, U+000B), FF (formfeed, U+000C), NEL (next line, U+0085), LS (line
- separator, U+2028), and PS (paragraph separator, U+2029).
-
- Each of the first three conventions is used by at least one operating
- system as its standard newline sequence. When PCRE is built, a default
- can be specified. The default default is LF, which is the Unix stan-
- dard. When PCRE is run, the default can be overridden, either when a
- pattern is compiled, or when it is matched.
-
- At compile time, the newline convention can be specified by the options
- argument of pcre_compile(), or it can be specified by special text at
- the start of the pattern itself; this overrides any other settings. See
- the pcrepattern page for details of the special character sequences.
-
- In the PCRE documentation the word "newline" is used to mean "the char-
- acter or pair of characters that indicate a line break". The choice of
- newline convention affects the handling of the dot, circumflex, and
- dollar metacharacters, the handling of #-comments in /x mode, and, when
- CRLF is a recognized line ending sequence, the match position advance-
- ment for a non-anchored pattern. There is more detail about this in the
- section on pcre_exec() options below.
-
- The choice of newline convention does not affect the interpretation of
- the \n or \r escape sequences, nor does it affect what \R matches,
- which is controlled in a similar way, but by separate options.
-
-
MULTITHREADING
The PCRE functions can be used in multi-threading applications, with
@@ -961,9 +518,7 @@
The compiled form of a regular expression can be saved and re-used at a
later time, possibly by a different program, and even on a host other
than the one on which it was compiled. Details are given in the
- pcreprecompile documentation. However, compiling a regular expression
- with one version of PCRE for use with a different version is not guar-
- anteed to work and may cause crashes.
+ pcreprecompile documentation.
CHECKING BUILD-TIME OPTIONS
@@ -992,56 +547,41 @@
PCRE_CONFIG_NEWLINE
- The output is an integer whose value specifies the default character
- sequence that is recognized as meaning "newline". The four values that
- are supported are: 10 for LF, 13 for CR, 3338 for CRLF, -2 for ANYCRLF,
- and -1 for ANY. The default should normally be the standard sequence
- for your operating system.
-
- PCRE_CONFIG_BSR
-
- The output is an integer whose value indicates what character sequences
- the \R escape sequence matches by default. A value of 0 means that \R
- matches any Unicode line ending sequence; a value of 1 means that \R
- matches only CR, LF, or CRLF. The default can be overridden when a pat-
- tern is compiled or matched.
+ The output is an integer that is set to the value of the code that is
+ used for the newline character. It is either linefeed (10) or carriage
+ return (13), and should normally be the standard character for your
+ operating system.
PCRE_CONFIG_LINK_SIZE
- The output is an integer that contains the number of bytes used for
+ The output is an integer that contains the number of bytes used for
internal linkage in compiled regular expressions. The value is 2, 3, or
- 4. Larger values allow larger regular expressions to be compiled, at
- the expense of slower matching. The default value of 2 is sufficient
- for all but the most massive patterns, since it allows the compiled
+ 4. Larger values allow larger regular expressions to be compiled, at
+ the expense of slower matching. The default value of 2 is sufficient
+ for all but the most massive patterns, since it allows the compiled
pattern to be up to 64K in size.
PCRE_CONFIG_POSIX_MALLOC_THRESHOLD
- The output is an integer that contains the threshold above which the
- POSIX interface uses malloc() for output vectors. Further details are
+ The output is an integer that contains the threshold above which the
+ POSIX interface uses malloc() for output vectors. Further details are
given in the pcreposix documentation.
PCRE_CONFIG_MATCH_LIMIT
The output is an integer that gives the default limit for the number of
- internal matching function calls in a pcre_exec() execution. Further
+ internal matching function calls in a pcre_exec() execution. Further
details are given with pcre_exec() below.
- PCRE_CONFIG_MATCH_LIMIT_RECURSION
-
- The output is an integer that gives the default limit for the depth of
- recursion when calling the internal matching function in a pcre_exec()
- execution. Further details are given with pcre_exec() below.
-
PCRE_CONFIG_STACKRECURSE
- The output is an integer that is set to one if internal recursion when
- running pcre_exec() is implemented by recursive function calls that use
- the stack to remember their state. This is the usual way that PCRE is
- compiled. The output is zero if PCRE was compiled to use blocks of data
- on the heap instead of recursive function calls. In this case,
- pcre_stack_malloc and pcre_stack_free are called to manage memory
- blocks on the heap, thus avoiding the use of the stack.
+ The output is an integer that is set to one if internal recursion is
+ implemented by recursive function calls that use the stack to remember
+ their state. This is the usual way that PCRE is compiled. The output is
+ zero if PCRE was compiled to use blocks of data on the heap instead of
+ recursive function calls. In this case, pcre_stack_malloc and
+ pcre_stack_free are called to manage memory blocks on the heap, thus
+ avoiding the use of the stack.
COMPILING A PATTERN
@@ -1050,62 +590,47 @@
const char **errptr, int *erroffset,
const unsigned char *tableptr);
- pcre *pcre_compile2(const char *pattern, int options,
- int *errorcodeptr,
- const char **errptr, int *erroffset,
- const unsigned char *tableptr);
-
- Either of the functions pcre_compile() or pcre_compile2() can be called
- to compile a pattern into an internal form. The only difference between
- the two interfaces is that pcre_compile2() has an additional argument,
- errorcodeptr, via which a numerical error code can be returned.
-
- The pattern is a C string terminated by a binary zero, and is passed in
- the pattern argument. A pointer to a single block of memory that is
- obtained via pcre_malloc is returned. This contains the compiled code
- and related data. The pcre type is defined for the returned block; this
- is a typedef for a structure whose contents are not externally defined.
- It is up to the caller to free the memory (via pcre_free) when it is no
- longer required.
+ The function pcre_compile() is called to compile a pattern into an
+ internal form. The pattern is a C string terminated by a binary zero,
+ and is passed in the pattern argument. A pointer to a single block of
+ memory that is obtained via pcre_malloc is returned. This contains the
+ compiled code and related data. The pcre type is defined for the
+ returned block; this is a typedef for a structure whose contents are
+ not externally defined. It is up to the caller to free the memory when
+ it is no longer required.
- Although the compiled code of a PCRE regex is relocatable, that is, it
+ Although the compiled code of a PCRE regex is relocatable, that is, it
does not depend on memory location, the complete pcre data block is not
- fully relocatable, because it may contain a copy of the tableptr argu-
+ fully relocatable, because it may contain a copy of the tableptr argu-
ment, which is an address (see below).
- The options argument contains various bit settings that affect the com-
- pilation. It should be zero if no options are required. The available
- options are described below. Some of them, in particular, those that
- are compatible with Perl, can also be set and unset from within the
- pattern (see the detailed description in the pcrepattern documenta-
- tion). For these options, the contents of the options argument speci-
- fies their initial settings at the start of compilation and execution.
- The PCRE_ANCHORED and PCRE_NEWLINE_xxx options can be set at the time
- of matching as well as at compile time.
+ The options argument contains independent bits that affect the compila-
+ tion. It should be zero if no options are required. The available
+ options are described below. Some of them, in particular, those that
+ are compatible with Perl, can also be set and unset from within the
+ pattern (see the detailed description in the pcrepattern documenta-
+ tion). For these options, the contents of the options argument speci-
+ fies their initial settings at the start of compilation and execution.
+ The PCRE_ANCHORED option can be set at the time of matching as well as
+ at compile time.
If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise,
- if compilation of a pattern fails, pcre_compile() returns NULL, and
+ if compilation of a pattern fails, pcre_compile() returns NULL, and
sets the variable pointed to by errptr to point to a textual error mes-
- sage. This is a static string that is part of the library. You must not
- try to free it. The offset from the start of the pattern to the charac-
- ter where the error was discovered is placed in the variable pointed to
- by erroffset, which must not be NULL. If it is, an immediate error is
+ sage. The offset from the start of the pattern to the character where
+ the error was discovered is placed in the variable pointed to by
+ erroffset, which must not be NULL. If it is, an immediate error is
given.
- If pcre_compile2() is used instead of pcre_compile(), and the error-
- codeptr argument is not NULL, a non-zero error code number is returned
- via this argument in the event of an error. This is in addition to the
- textual error message. Error codes and messages are listed below.
-
- If the final argument, tableptr, is NULL, PCRE uses a default set of
- character tables that are built when PCRE is compiled, using the
- default C locale. Otherwise, tableptr must be an address that is the
- result of a call to pcre_maketables(). This value is stored with the
- compiled pattern, and used again by pcre_exec(), unless another table
+ If the final argument, tableptr, is NULL, PCRE uses a default set of
+ character tables that are built when PCRE is compiled, using the
+ default C locale. Otherwise, tableptr must be an address that is the
+ result of a call to pcre_maketables(). This value is stored with the
+ compiled pattern, and used again by pcre_exec(), unless another table
pointer is passed to it. For more discussion, see the section on locale
support below.
- This code fragment shows a typical straightforward call to pcre_com-
+ This code fragment shows a typical straightforward call to pcre_com-
pile():
pcre *re;
@@ -1118,80 +643,58 @@
&erroffset, /* for error offset */
NULL); /* use default character tables */
- The following names for option bits are defined in the pcre.h header
+ The following names for option bits are defined in the pcre.h header
file:
PCRE_ANCHORED
If this bit is set, the pattern is forced to be "anchored", that is, it
- is constrained to match only at the first matching point in the string
- that is being searched (the "subject string"). This effect can also be
- achieved by appropriate constructs in the pattern itself, which is the
+ is constrained to match only at the first matching point in the string
+ that is being searched (the "subject string"). This effect can also be
+ achieved by appropriate constructs in the pattern itself, which is the
only way to do it in Perl.
PCRE_AUTO_CALLOUT
If this bit is set, pcre_compile() automatically inserts callout items,
- all with number 255, before each pattern item. For discussion of the
+ all with number 255, before each pattern item. For discussion of the
callout facility, see the pcrecallout documentation.
- PCRE_BSR_ANYCRLF
- PCRE_BSR_UNICODE
-
- These options (which are mutually exclusive) control what the \R escape
- sequence matches. The choice is either to match only CR, LF, or CRLF,
- or to match any Unicode newline sequence. The default is specified when
- PCRE is built. It can be overridden from within the pattern, or by set-
- ting an option when a compiled pattern is matched.
-
PCRE_CASELESS
- If this bit is set, letters in the pattern match both upper and lower
- case letters. It is equivalent to Perl's /i option, and it can be
- changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE
- always understands the concept of case for characters whose values are
- less than 128, so caseless matching is always possible. For characters
- with higher values, the concept of case is supported if PCRE is com-
- piled with Unicode property support, but not otherwise. If you want to
- use caseless matching for characters 128 and above, you must ensure
- that PCRE is compiled with Unicode property support as well as with
- UTF-8 support.
+ If this bit is set, letters in the pattern match both upper and lower
+ case letters. It is equivalent to Perl's /i option, and it can be
+ changed within a pattern by a (?i) option setting. When running in
+ UTF-8 mode, case support for high-valued characters is available only
+ when PCRE is built with Unicode character property support.
PCRE_DOLLAR_ENDONLY
If this bit is set, a dollar metacharacter in the pattern matches only
at the end of the subject string. Without this option, a dollar also
- matches immediately before a newline at the end of the string (but not
- before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored
- if PCRE_MULTILINE is set. There is no equivalent to this option in
- Perl, and no way to set it within a pattern.
+ matches immediately before the final character if it is a newline (but
+ not before any other newlines). The PCRE_DOLLAR_ENDONLY option is
+ ignored if PCRE_MULTILINE is set. There is no equivalent to this option
+ in Perl, and no way to set it within a pattern.
PCRE_DOTALL
If this bit is set, a dot metacharater in the pattern matches all char-
- acters, including those that indicate newline. Without it, a dot does
- not match when the current position is at a newline. This option is
- equivalent to Perl's /s option, and it can be changed within a pattern
- by a (?s) option setting. A negative class such as [^a] always matches
- newline characters, independent of the setting of this option.
-
- PCRE_DUPNAMES
-
- If this bit is set, names used to identify capturing subpatterns need
- not be unique. This can be helpful for certain types of pattern when it
- is known that only one instance of the named subpattern can ever be
- matched. There are more details of named subpatterns below; see also
- the pcrepattern documentation.
+ acters, including newlines. Without it, newlines are excluded. This
+ option is equivalent to Perl's /s option, and it can be changed within
+ a pattern by a (?s) option setting. A negative class such as [^a]
+ always matches a newline character, independent of the setting of this
+ option.
PCRE_EXTENDED
If this bit is set, whitespace data characters in the pattern are
- totally ignored except when escaped or inside a character class. White-
- space does not include the VT character (code 11). In addition, charac-
- ters between an unescaped # outside a character class and the next new-
- line, inclusive, are also ignored. This is equivalent to Perl's /x
- option, and it can be changed within a pattern by a (?x) option set-
- ting.
+ totally ignored except when escaped or inside a character class.
+ Whitespace does not include the VT character (code 11). In addition,
+ characters between an unescaped # outside a character class and the
+ next newline character, inclusive, are also ignored. This is equivalent
+ to Perl's /x option, and it can be changed within a pattern by a (?x)
+ option setting.
This option makes it possible to include comments inside complicated
patterns. Note, however, that this applies only to data characters.
@@ -1207,71 +710,26 @@
letter that has no special meaning causes an error, thus reserving
these combinations for future expansion. By default, as in Perl, a
backslash followed by a letter with no special meaning is treated as a
- literal. (Perl can, however, be persuaded to give a warning for this.)
- There are at present no other features controlled by this option. It
- can also be set by a (?X) option setting within a pattern.
-
- PCRE_FIRSTLINE
-
- If this option is set, an unanchored pattern is required to match
- before or at the first newline in the subject string, though the
- matched text may continue over the newline.
+ literal. There are at present no other features controlled by this
+ option. It can also be set by a (?X) option setting within a pattern.
PCRE_MULTILINE
- By default, PCRE treats the subject string as consisting of a single
- line of characters (even if it actually contains newlines). The "start
- of line" metacharacter (^) matches only at the start of the string,
- while the "end of line" metacharacter ($) matches only at the end of
+ By default, PCRE treats the subject string as consisting of a single
+ line of characters (even if it actually contains newlines). The "start
+ of line" metacharacter (^) matches only at the start of the string,
+ while the "end of line" metacharacter ($) matches only at the end of
the string, or before a terminating newline (unless PCRE_DOLLAR_ENDONLY
is set). This is the same as Perl.
- When PCRE_MULTILINE it is set, the "start of line" and "end of line"
- constructs match immediately following or immediately before internal
- newlines in the subject string, respectively, as well as at the very
- start and end. This is equivalent to Perl's /m option, and it can be
- changed within a pattern by a (?m) option setting. If there are no new-
- lines in a subject string, or no occurrences of ^ or $ in a pattern,
+ When PCRE_MULTILINE it is set, the "start of line" and "end of line"
+ constructs match immediately following or immediately before any new-
+ line in the subject string, respectively, as well as at the very start
+ and end. This is equivalent to Perl's /m option, and it can be changed
+ within a pattern by a (?m) option setting. If there are no "\n" charac-
+ ters in a subject string, or no occurrences of ^ or $ in a pattern,
setting PCRE_MULTILINE has no effect.
- PCRE_NEWLINE_CR
- PCRE_NEWLINE_LF
- PCRE_NEWLINE_CRLF
- PCRE_NEWLINE_ANYCRLF
- PCRE_NEWLINE_ANY
-
- These options override the default newline definition that was chosen
- when PCRE was built. Setting the first or the second specifies that a
- newline is indicated by a single character (CR or LF, respectively).
- Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the
- two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies
- that any of the three preceding sequences should be recognized. Setting
- PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be
- recognized. The Unicode newline sequences are the three just mentioned,
- plus the single characters VT (vertical tab, U+000B), FF (formfeed,
- U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS
- (paragraph separator, U+2029). The last two are recognized only in
- UTF-8 mode.
-
- The newline setting in the options word uses three bits that are
- treated as a number, giving eight possibilities. Currently only six are
- used (default plus the five values above). This means that if you set
- more than one newline option, the combination may or may not be sensi-
- ble. For example, PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to
- PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and
- cause an error.
-
- The only time that a line break is specially recognized when compiling
- a pattern is if PCRE_EXTENDED is set, and an unescaped # outside a
- character class is encountered. This indicates a comment that lasts
- until after the next line break sequence. In other circumstances, line
- break sequences are treated as literal data, except that in
- PCRE_EXTENDED mode, both CR and LF are treated as whitespace characters
- and are therefore ignored.
-
- The newline option that is set at compile time becomes the default that
- is used for pcre_exec() and pcre_dfa_exec(), but it can be overridden.
-
PCRE_NO_AUTO_CAPTURE
If this option is set, it disables the use of numbered capturing paren-
@@ -1299,119 +757,47 @@
PCRE_NO_UTF8_CHECK
When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is
- automatically checked. There is a discussion about the validity of
- UTF-8 strings in the main pcre page. If an invalid UTF-8 sequence of
- bytes is found, pcre_compile() returns an error. If you already know
- that your pattern is valid, and you want to skip this check for perfor-
- mance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is
- set, the effect of passing an invalid UTF-8 string as a pattern is
- undefined. It may cause your program to crash. Note that this option
- can also be passed to pcre_exec() and pcre_dfa_exec(), to suppress the
- UTF-8 validity checking of subject strings.
-
-
-COMPILATION ERROR CODES
-
- The following table lists the error codes than may be returned by
- pcre_compile2(), along with the error messages that may be returned by
- both compiling functions. As PCRE has developed, some error codes have
- fallen out of use. To avoid confusion, they have not been re-used.
-
- 0 no error
- 1 \ at end of pattern
- 2 \c at end of pattern
- 3 unrecognized character follows \
- 4 numbers out of order in {} quantifier
- 5 number too big in {} quantifier
- 6 missing terminating ] for character class
- 7 invalid escape sequence in character class
- 8 range out of order in character class
- 9 nothing to repeat
- 10 [this code is not in use]
- 11 internal error: unexpected repeat
- 12 unrecognized character after (?
- 13 POSIX named classes are supported only within a class
- 14 missing )
- 15 reference to non-existent subpattern
- 16 erroffset passed as NULL
- 17 unknown option bit(s) set
- 18 missing ) after comment
- 19 [this code is not in use]
- 20 regular expression too large
- 21 failed to get memory
- 22 unmatched parentheses
- 23 internal error: code overflow
- 24 unrecognized character after (?<
- 25 lookbehind assertion is not fixed length
- 26 malformed number or name after (?(
- 27 conditional group contains more than two branches
- 28 assertion expected after (?(
- 29 (?R or (?[+-]digits must be followed by )
- 30 unknown POSIX class name
- 31 POSIX collating elements are not supported
- 32 this version of PCRE is not compiled with PCRE_UTF8 support
- 33 [this code is not in use]
- 34 character value in \x{...} sequence is too large
- 35 invalid condition (?(0)
- 36 \C not allowed in lookbehind assertion
- 37 PCRE does not support \L, \l, \N, \U, or \u
- 38 number after (?C is > 255
- 39 closing ) for (?C expected
- 40 recursive call could loop indefinitely
- 41 unrecognized character after (?P
- 42 syntax error in subpattern name (missing terminator)
- 43 two named subpatterns have the same name
- 44 invalid UTF-8 string
- 45 support for \P, \p, and \X has not been compiled
- 46 malformed \P or \p sequence
- 47 unknown property name after \P or \p
- 48 subpattern name is too long (maximum 32 characters)
- 49 too many named subpatterns (maximum 10,000)
- 50 [this code is not in use]
- 51 octal value is greater than \377 (not in UTF-8 mode)
- 52 internal error: overran compiling workspace
- 53 internal error: previously-checked referenced subpattern not
- found
- 54 DEFINE group contains more than one branch
- 55 repeating a DEFINE group is not allowed
- 56 inconsistent NEWLINE options
- 57 \g is not followed by a braced name or an optionally braced
- non-zero number
- 58 (?+ or (?- or (?(+ or (?(- must be followed by a non-zero number
+ automatically checked. If an invalid UTF-8 sequence of bytes is found,
+ pcre_compile() returns an error. If you already know that your pattern
+ is valid, and you want to skip this check for performance reasons, you
+ can set the PCRE_NO_UTF8_CHECK option. When it is set, the effect of
+ passing an invalid UTF-8 string as a pattern is undefined. It may cause
+ your program to crash. Note that this option can also be passed to
+ pcre_exec(), to suppress the UTF-8 validity checking of subject
+ strings.
STUDYING A PATTERN
- pcre_extra *pcre_study(const pcre *code, int options
+ pcre_extra *pcre_study(const pcre *code, int options,
const char **errptr);
- If a compiled pattern is going to be used several times, it is worth
+ If a compiled pattern is going to be used several times, it is worth
spending more time analyzing it in order to speed up the time taken for
- matching. The function pcre_study() takes a pointer to a compiled pat-
+ matching. The function pcre_study() takes a pointer to a compiled pat-
tern as its first argument. If studying the pattern produces additional
- information that will help speed up matching, pcre_study() returns a
- pointer to a pcre_extra block, in which the study_data field points to
+ information that will help speed up matching, pcre_study() returns a
+ pointer to a pcre_extra block, in which the study_data field points to
the results of the study.
The returned value from pcre_study() can be passed directly to
- pcre_exec(). However, a pcre_extra block also contains other fields
- that can be set by the caller before the block is passed; these are
+ pcre_exec(). However, a pcre_extra block also contains other fields
+ that can be set by the caller before the block is passed; these are
described below in the section on matching a pattern.
- If studying the pattern does not produce any additional information
+ If studying the pattern does not produce any additional information,
pcre_study() returns NULL. In that circumstance, if the calling program
- wants to pass any of the other fields to pcre_exec(), it must set up
+ wants to pass any of the other fields to pcre_exec(), it must set up
its own pcre_extra block.
- The second argument of pcre_study() contains option bits. At present,
+ The second argument of pcre_study() contains option bits. At present,
no options are defined, and this argument should always be zero.
- The third argument for pcre_study() is a pointer for an error message.
- If studying succeeds (even if no data is returned), the variable it
- points to is set to NULL. Otherwise it is set to point to a textual
- error message. This is a static string that is part of the library. You
- must not try to free it. You should test the error pointer for NULL
- after calling pcre_study(), to be sure that it has run successfully.
+ The third argument for pcre_study() is a pointer for an error message.
+ If studying succeeds (even if no data is returned), the variable it
+ points to is set to NULL. Otherwise it points to a textual error mes-
+ sage. You should therefore test the error pointer for NULL after call-
+ ing pcre_study(), to be sure that it has run successfully.
This is a typical call to pcre_study():
@@ -1430,25 +816,17 @@
PCRE handles caseless matching, and determines whether characters are
letters, digits, or whatever, by reference to a set of tables, indexed
- by character value. When running in UTF-8 mode, this applies only to
+ by character value. (When running in UTF-8 mode, this applies only to
characters with codes less than 128. Higher-valued codes never match
escapes such as \w or \d, but can be tested with \p if PCRE is built
- with Unicode character property support. The use of locales with Uni-
- code is discouraged. If you are handling characters with codes greater
- than 128, you should either use UTF-8 and Unicode, or use locales, but
- not try to mix the two.
-
- PCRE contains an internal set of tables that are used when the final
- argument of pcre_compile() is NULL. These are sufficient for many
- applications. Normally, the internal tables recognize only ASCII char-
- acters. However, when PCRE is built, it is possible to cause the inter-
- nal tables to be rebuilt in the default "C" locale of the local system,
- which may cause them to be different.
-
- The internal tables can always be overridden by tables supplied by the
- application that calls PCRE. These may be created in a different locale
- from the default. As more and more applications change to using Uni-
- code, the need for this locale support is expected to die away.
+ with Unicode character property support.)
+
+ An internal set of tables is created in the default C locale when PCRE
+ is built. This is used when the final argument of pcre_compile() is
+ NULL, and is sufficient for many applications. An alternative set of
+ tables can, however, be supplied. These may be created in a different
+ locale from the default. As more and more applications change to using
+ Unicode, the need for this locale support is expected to die away.
External tables are built by calling the pcre_maketables() function,
which has no arguments, in the relevant locale. The result can then be
@@ -1461,23 +839,20 @@
tables = pcre_maketables();
re = pcre_compile(..., tables);
- The locale name "fr_FR" is used on Linux and other Unix-like systems;
- if you are using Windows, the name for the French locale is "french".
-
- When pcre_maketables() runs, the tables are built in memory that is
- obtained via pcre_malloc. It is the caller's responsibility to ensure
- that the memory containing the tables remains available for as long as
+ When pcre_maketables() runs, the tables are built in memory that is
+ obtained via pcre_malloc. It is the caller's responsibility to ensure
+ that the memory containing the tables remains available for as long as
it is needed.
The pointer that is passed to pcre_compile() is saved with the compiled
- pattern, and the same tables are used via this pointer by pcre_study()
+ pattern, and the same tables are used via this pointer by pcre_study()
and normally also by pcre_exec(). Thus, by default, for any single pat-
tern, compilation, studying and matching all happen in the same locale,
but different patterns can be compiled in different locales.
- It is possible to pass a table pointer or NULL (indicating the use of
- the internal tables) to pcre_exec(). Although not intended for this
- purpose, this facility could be used to match a pattern in a different
+ It is possible to pass a table pointer or NULL (indicating the use of
+ the internal tables) to pcre_exec(). Although not intended for this
+ purpose, this facility could be used to match a pattern in a different
locale from the one in which it was compiled. Passing table pointers at
run time is discussed below in the section on matching a pattern.
@@ -1487,15 +862,15 @@
int pcre_fullinfo(const pcre *code, const pcre_extra *extra,
int what, void *where);
- The pcre_fullinfo() function returns information about a compiled pat-
+ The pcre_fullinfo() function returns information about a compiled pat-
tern. It replaces the obsolete pcre_info() function, which is neverthe-
less retained for backwards compability (and is documented below).
- The first argument for pcre_fullinfo() is a pointer to the compiled
- pattern. The second argument is the result of pcre_study(), or NULL if
- the pattern was not studied. The third argument specifies which piece
- of information is required, and the fourth argument is a pointer to a
- variable to receive the data. The yield of the function is zero for
+ The first argument for pcre_fullinfo() is a pointer to the compiled
+ pattern. The second argument is the result of pcre_study(), or NULL if
+ the pattern was not studied. The third argument specifies which piece
[... 4815 lines stripped ...]