|
|
<html><head><title>pcre2test specification</title></head><body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"><h1>pcre2test man page</h1><p>Return to the <a href="index.html">PCRE2 index page</a>.</p><p>This page is part of the PCRE2 HTML documentation. It was generatedautomatically from the original man page. If there is any nonsense in it,please consult the man page, in case the conversion went wrong.<br><ul><li><a name="TOC1" href="#SEC1">SYNOPSIS</a><li><a name="TOC2" href="#SEC2">PCRE2's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a><li><a name="TOC3" href="#SEC3">INPUT ENCODING</a><li><a name="TOC4" href="#SEC4">COMMAND LINE OPTIONS</a><li><a name="TOC5" href="#SEC5">DESCRIPTION</a><li><a name="TOC6" href="#SEC6">COMMAND LINES</a><li><a name="TOC7" href="#SEC7">MODIFIER SYNTAX</a><li><a name="TOC8" href="#SEC8">PATTERN SYNTAX</a><li><a name="TOC9" href="#SEC9">SUBJECT LINE SYNTAX</a><li><a name="TOC10" href="#SEC10">PATTERN MODIFIERS</a><li><a name="TOC11" href="#SEC11">SUBJECT MODIFIERS</a><li><a name="TOC12" href="#SEC12">THE ALTERNATIVE MATCHING FUNCTION</a><li><a name="TOC13" href="#SEC13">DEFAULT OUTPUT FROM pcre2test</a><li><a name="TOC14" href="#SEC14">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a><li><a name="TOC15" href="#SEC15">RESTARTING AFTER A PARTIAL MATCH</a><li><a name="TOC16" href="#SEC16">CALLOUTS</a><li><a name="TOC17" href="#SEC17">NON-PRINTING CHARACTERS</a><li><a name="TOC18" href="#SEC18">SAVING AND RESTORING COMPILED PATTERNS</a><li><a name="TOC19" href="#SEC19">SEE ALSO</a><li><a name="TOC20" href="#SEC20">AUTHOR</a><li><a name="TOC21" href="#SEC21">REVISION</a></ul><br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br><P><b>pcre2test [options] [input file [output file]]</b><br><br><b>pcre2test</b> is a test program for the PCRE2 regular expression libraries,but it can also be used for experimenting with regular expressions. Thisdocument describes the features of the test program; for details of the regularexpressions themselves, see the<a href="pcre2pattern.html"><b>pcre2pattern</b></a>documentation. For details of the PCRE2 library function calls and theiroptions, see the<a href="pcre2api.html"><b>pcre2api</b></a>documentation.</P><P>The input for <b>pcre2test</b> is a sequence of regular expression patterns andsubject strings to be matched. There are also command lines for settingdefaults and controlling some special actions. The output shows the result ofeach match attempt. Modifiers on external or internal command lines, thepatterns, and the subject lines specify PCRE2 function options, control how thesubject is processed, and what output is produced.</P><P>There are many obscure modifiers, some of which are specifically designed foruse in conjunction with the test script and data files that are distributed aspart of PCRE2. All the modifiers are documented here, some without muchjustification, but many of them are unlikely to be of use except when testingthe libraries.</P><br><a name="SEC2" href="#TOC1">PCRE2's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a><br><P>Different versions of the PCRE2 library can be built to support characterstrings that are encoded in 8-bit, 16-bit, or 32-bit code units. One, two, orall three of these libraries may be simultaneously installed. The<b>pcre2test</b> program can be used to test all the libraries. However, its owninput and output are always in 8-bit format. When testing the 16-bit or 32-bitlibraries, patterns and subject strings are converted to 16-bit or 32-bitformat before being passed to the library functions. Results are converted backto 8-bit code units for output.</P><P>In the rest of this document, the names of library functions and structuresare given in generic form, for example, <b>pcre2_compile()</b>. The actualnames used in the libraries have a suffix _8, _16, or _32, as appropriate.<a name="inputencoding"></a></P><br><a name="SEC3" href="#TOC1">INPUT ENCODING</a><br><P>Input to <b>pcre2test</b> is processed line by line, either by calling the Clibrary's <b>fgets()</b> function, or via the <b>libreadline</b> or <b>libedit</b>library. In some Windows environments character 26 (hex 1A) causes an immediateend of file, and no further data is read, so this character should be avoidedunless you really want that action.</P><P>The input is processed using C's string functions, so must not contain binaryzeros, even though in Unix-like environments, <b>fgets()</b> treats any bytesother than newline as data characters. An error is generated if a binary zerois encountered. By default subject lines are processed for backslash escapes,which makes it possible to include any data value in strings that are passed tothe library for matching. For patterns, there is a facility for specifying someor all of the 8-bit input characters as hexadecimal pairs, which makes itpossible to include binary zeros.</P><br><b>Input for the 16-bit and 32-bit libraries</b><br><P>When testing the 16-bit or 32-bit libraries, there is a need to be able togenerate character code points greater than 255 in the strings that are passedto the library. For subject lines, backslash escapes can be used. In addition,when the <b>utf</b> modifier (see<a href="#optionmodifiers">"Setting compilation options"</a>below) is set, the pattern and any following subject lines are interpreted asUTF-8 strings and translated to UTF-16 or UTF-32 as appropriate.</P><P>For non-UTF testing of wide characters, the <b>utf8_input</b> modifier can beused. This is mutually exclusive with <b>utf</b>, and is allowed only in 16-bitor 32-bit mode. It causes the pattern and following subject lines to be treatedas UTF-8 according to the original definition (RFC 2279), which allows forcharacter values up to 0x7fffffff. Each character is placed in one 16-bit or32-bit code unit (in the 16-bit case, values greater than 0xffff cause an errorto occur).</P><P>UTF-8 (in its original definition) is not capable of encoding values greaterthan 0x7fffffff, but such values can be handled by the 32-bit library. Whentesting this library in non-UTF mode with <b>utf8_input</b> set, if anycharacter is preceded by the byte 0xff (which is an invalid byte in UTF-8)0x80000000 is added to the character's value. This is the only way of passingsuch code points in a pattern string. For subject strings, using an escapesequence is preferable.</P><br><a name="SEC4" href="#TOC1">COMMAND LINE OPTIONS</a><br><P><b>-8</b>If the 8-bit library has been built, this option causes it to be used (this isthe default). If the 8-bit library has not been built, this option causes anerror.</P><P><b>-16</b>If the 16-bit library has been built, this option causes it to be used. If the8-bit library has not been built, this is the default. If the 16-bit libraryhas not been built, this option causes an error.</P><P><b>-32</b>If the 32-bit library has been built, this option causes it to be used. If noother library has been built, this is the default. If the 32-bit library hasnot been built, this option causes an error.</P><P><b>-ac</b>Behave as if each pattern has the <b>auto_callout</b> modifier, that is, insertautomatic callouts into every pattern that is compiled.</P><P><b>-AC</b>As for <b>-ac</b>, but in addition behave as if each subject line has the<b>callout_extra</b> modifier, that is, show additional information fromcallouts.</P><P><b>-b</b>Behave as if each pattern has the <b>fullbincode</b> modifier; the fullinternal binary form of the pattern is output after compilation.</P><P><b>-C</b>Output the version number of the PCRE2 library, and all available informationabout the optional features that are included, and then exit with zero exitcode. All other options are ignored. If both -C and -LM are present, whicheveris first is recognized.</P><P><b>-C</b> <i>option</i>Output information about a specific build-time option, then exit. Thisfunctionality is intended for use in scripts such as <b>RunTest</b>. Thefollowing options output the value and set the exit code as indicated:<pre> ebcdic-nl the code for LF (= NL) in an EBCDIC environment: 0x15 or 0x25 0 if used in an ASCII environment exit code is always 0 linksize the configured internal link size (2, 3, or 4) exit code is set to the link size newline the default newline setting: CR, LF, CRLF, ANYCRLF, ANY, or NUL exit code is always 0 bsr the default setting for what \R matches: ANYCRLF or ANY exit code is always 0</pre>The following options output 1 for true or 0 for false, and set the exit codeto the same value:<pre> backslash-C \C is supported (not locked out) ebcdic compiled for an EBCDIC environment jit just-in-time support is available pcre2-16 the 16-bit library was built pcre2-32 the 32-bit library was built pcre2-8 the 8-bit library was built unicode Unicode support is available</pre>If an unknown option is given, an error message is output; the exit code is 0.</P><P><b>-d</b>Behave as if each pattern has the <b>debug</b> modifier; the internalform and information about the compiled pattern is output after compilation;<b>-d</b> is equivalent to <b>-b -i</b>.</P><P><b>-dfa</b>Behave as if each subject line has the <b>dfa</b> modifier; matching is doneusing the <b>pcre2_dfa_match()</b> function instead of the default<b>pcre2_match()</b>.</P><P><b>-error</b> <i>number[,number,...]</i>Call <b>pcre2_get_error_message()</b> for each of the error numbers in thecomma-separated list, display the resulting messages on the standard output,then exit with zero exit code. The numbers may be positive or negative. This isa convenience facility for PCRE2 maintainers.</P><P><b>-help</b>Output a brief summary these options and then exit.</P><P><b>-i</b>Behave as if each pattern has the <b>info</b> modifier; information about thecompiled pattern is given after compilation.</P><P><b>-jit</b>Behave as if each pattern line has the <b>jit</b> modifier; after successfulcompilation, each pattern is passed to the just-in-time compiler, if available.</P><P><b>-jitfast</b>Behave as if each pattern line has the <b>jitfast</b> modifier; aftersuccessful compilation, each pattern is passed to the just-in-time compiler, ifavailable, and each subject line is passed directly to the JIT matcher via its"fast path".</P><P><b>-jitverify</b>Behave as if each pattern line has the <b>jitverify</b> modifier; aftersuccessful compilation, each pattern is passed to the just-in-time compiler, ifavailable, and the use of JIT for matching is verified.</P><P><b>-LM</b>List modifiers: write a list of available pattern and subject modifiers to thestandard output, then exit with zero exit code. All other options are ignored.If both -C and any -Lx options are present, whichever is first is recognized.</P><P><b>-LP</b>List properties: write a list of recognized Unicode properties to the standardoutput, then exit with zero exit code. All other options are ignored. If both-C and any -Lx options are present, whichever is first is recognized.</P><P><b>-LS</b>List scripts: write a list of recognized Unicode script names to the standardoutput, then exit with zero exit code. All other options are ignored. If both-C and any -Lx options are present, whichever is first is recognized.</P><P><b>-pattern</b> <i>modifier-list</i>Behave as if each pattern line contains the given modifiers.</P><P><b>-q</b>Do not output the version number of <b>pcre2test</b> at the start of execution.</P><P><b>-S</b> <i>size</i>On Unix-like systems, set the size of the run-time stack to <i>size</i>mebibytes (units of 1024*1024 bytes).</P><P><b>-subject</b> <i>modifier-list</i>Behave as if each subject line contains the given modifiers.</P><P><b>-t</b>Run each compile and match many times with a timer, and output the resultingtimes per compile or match. When JIT is used, separate times are given for theinitial compile and the JIT compile. You can control the number of iterationsthat are used for timing by following <b>-t</b> with a number (as a separateitem on the command line). For example, "-t 1000" iterates 1000 times. Thedefault is to iterate 500,000 times.</P><P><b>-tm</b>This is like <b>-t</b> except that it times only the matching phase, not thecompile phase.</P><P><b>-T</b> <b>-TM</b>These behave like <b>-t</b> and <b>-tm</b>, but in addition, at the end of a run,the total times for all compiles and matches are output.</P><P><b>-version</b>Output the PCRE2 version number and then exit.</P><br><a name="SEC5" href="#TOC1">DESCRIPTION</a><br><P>If <b>pcre2test</b> is given two filename arguments, it reads from the first andwrites to the second. If the first name is "-", input is taken from thestandard input. If <b>pcre2test</b> is given only one argument, it reads fromthat file and writes to stdout. Otherwise, it reads from stdin and writes tostdout.</P><P>When <b>pcre2test</b> is built, a configuration option can specify that itshould be linked with the <b>libreadline</b> or <b>libedit</b> library. When thisis done, if the input is from a terminal, it is read using the <b>readline()</b>function. This provides line-editing and history facilities. The output fromthe <b>-help</b> option states whether or not <b>readline()</b> will be used.</P><P>The program handles any number of tests, each of which consists of a set ofinput lines. Each set starts with a regular expression pattern, followed by anynumber of subject lines to be matched against that pattern. In between sets oftest data, command lines that begin with # may appear. This file format, withsome restrictions, can also be processed by the <b>perltest.sh</b> script thatis distributed with PCRE2 as a means of checking that the behaviour of PCRE2and Perl is the same. For a specification of <b>perltest.sh</b>, see thecomments near its beginning. See also the #perltest command below.</P><P>When the input is a terminal, <b>pcre2test</b> prompts for each line of input,using "re>" to prompt for regular expression patterns, and "data>" to promptfor subject lines. Command lines starting with # can be entered only inresponse to the "re>" prompt.</P><P>Each subject line is matched separately and independently. If you want to domulti-line matches, you have to use the \n escape sequence (or \r or \r\n,etc., depending on the newline setting) in a single line of input to encode thenewline sequences. There is no limit on the length of subject lines; the inputbuffer is automatically extended if it is too small. There are replicationfeatures that makes it possible to generate long repetitive pattern or subjectlines without having to supply them explicitly.</P><P>An empty line or the end of the file signals the end of the subject lines for atest, at which point a new pattern or command line is expected if there isstill input to be read.</P><br><a name="SEC6" href="#TOC1">COMMAND LINES</a><br><P>In between sets of test data, a line that begins with # is interpreted as acommand line. If the first character is followed by white space or anexclamation mark, the line is treated as a comment, and ignored. Otherwise, thefollowing commands are recognized:<pre> #forbid_utf</pre>Subsequent patterns automatically have the PCRE2_NEVER_UTF and PCRE2_NEVER_UCPoptions set, which locks out the use of the PCRE2_UTF and PCRE2_UCP options andthe use of (*UTF) and (*UCP) at the start of patterns. This command also forcesan error if a subsequent pattern contains any occurrences of \P, \p, or \X,which are still supported when PCRE2_UTF is not set, but which require Unicodeproperty support to be included in the library.</P><P>This is a trigger guard that is used in test files to ensure that UTF orUnicode property tests are not accidentally added to files that are used whenUnicode support is not included in the library. Setting PCRE2_NEVER_UTF andPCRE2_NEVER_UCP as a default can also be obtained by the use of <b>#pattern</b>;the difference is that <b>#forbid_utf</b> cannot be unset, and the automaticoptions are not displayed in pattern information, to avoid cluttering up testoutput.<pre> #load <filename></pre>This command is used to load a set of precompiled patterns from a file, asdescribed in the section entitled "Saving and restoring compiled patterns"<a href="#saverestore">below.</a><pre> #loadtables <filename></pre>This command is used to load a set of binary character tables that can beaccessed by the tables=3 qualifier. Such tables can be created by the<b>pcre2_dftables</b> program with the -b option.<pre> #newline_default [<newline-list>]</pre>When PCRE2 is built, a default newline convention can be specified. Thisdetermines which characters and/or character pairs are recognized as indicatinga newline in a pattern or subject string. The default can be overridden when apattern is compiled. The standard test files contain tests of various newlineconventions, but the majority of the tests expect a single linefeed to berecognized as a newline by default. Without special action the tests would failwhen PCRE2 is compiled with either CR or CRLF as the default newline.</P><P>The #newline_default command specifies a list of newline types that areacceptable as the default. The types must be one of CR, LF, CRLF, ANYCRLF,ANY, or NUL (in upper or lower case), for example:<pre> #newline_default LF Any anyCRLF</pre>If the default newline is in the list, this command has no effect. Otherwise,except when testing the POSIX API, a <b>newline</b> modifier that specifies thefirst newline convention in the list (LF in the above example) is added to anypattern that does not already have a <b>newline</b> modifier. If the newlinelist is empty, the feature is turned off. This command is present in a numberof the standard test input files.</P><P>When the POSIX API is being tested there is no way to override the defaultnewline convention, though it is possible to set the newline convention fromwithin the pattern. A warning is given if the <b>posix</b> or <b>posix_nosub</b>modifier is used when <b>#newline_default</b> would set a default for thenon-POSIX API.<pre> #pattern <modifier-list></pre>This command sets a default modifier list that applies to all subsequentpatterns. Modifiers on a pattern can change these settings.<pre> #perltest</pre>This line is used in test files that can also be processed by <b>perltest.sh</b>to confirm that Perl gives the same results as PCRE2. Subsequent tests arechecked for the use of <b>pcre2test</b> features that are incompatible with the<b>perltest.sh</b> script.</P><P>Patterns must use '/' as their delimiter, and only certain modifiers aresupported. Comment lines, #pattern commands, and #subject commands that set orunset "mark" are recognized and acted on. The #perltest, #forbid_utf, and#newline_default commands, which are needed in the relevant pcre2test files,are silently ignored. All other command lines are ignored, but give a warningmessage. The <b>#perltest</b> command helps detect tests that are accidentallyput in the wrong file or use the wrong delimiter. For more details of the<b>perltest.sh</b> script see the comments it contains.<pre> #pop [<modifiers>] #popcopy [<modifiers>]</pre>These commands are used to manipulate the stack of compiled patterns, asdescribed in the section entitled "Saving and restoring compiled patterns"<a href="#saverestore">below.</a><pre> #save <filename></pre>This command is used to save a set of compiled patterns to a file, as describedin the section entitled "Saving and restoring compiled patterns"<a href="#saverestore">below.</a><pre> #subject <modifier-list></pre>This command sets a default modifier list that applies to all subsequentsubject lines. Modifiers on a subject line can change these settings.</P><br><a name="SEC7" href="#TOC1">MODIFIER SYNTAX</a><br><P>Modifier lists are used with both pattern and subject lines. Items in a listare separated by commas followed by optional white space. Trailing whitespacein a modifier list is ignored. Some modifiers may be given for both patternsand subject lines, whereas others are valid only for one or the other. Eachmodifier has a long name, for example "anchored", and some of them must befollowed by an equals sign and a value, for example, "offset=12". Values cannotcontain comma characters, but may contain spaces. Modifiers that do not takevalues may be preceded by a minus sign to turn off a previous setting.</P><P>A few of the more common modifiers can also be specified as single letters, forexample "i" for "caseless". In documentation, following the Perl convention,these are written with a slash ("the /i modifier") for clarity. Abbreviatedmodifiers must all be concatenated in the first item of a modifier list. If thefirst item is not recognized as a long modifier name, it is interpreted as asequence of these abbreviations. For example:<pre> /abc/ig,newline=cr,jit=3</pre>This is a pattern line whose modifier list starts with two one-letter modifiers(/i and /g). The lower-case abbreviated modifiers are the same as used in Perl.</P><br><a name="SEC8" href="#TOC1">PATTERN SYNTAX</a><br><P>A pattern line must start with one of the following characters (common symbols,excluding pattern meta-characters):<pre> / ! " ' ` - = _ : ; , % & @ ~</pre>This is interpreted as the pattern's delimiter. A regular expression may becontinued over several input lines, in which case the newline characters areincluded within it. It is possible to include the delimiter as a literal withinthe pattern by escaping it with a backslash, for example<pre> /abc\/def/</pre>If you do this, the escape and the delimiter form part of the pattern, butsince the delimiters are all non-alphanumeric, the inclusion of the backslashdoes not affect the pattern's interpretation. Note, however, that this trickdoes not work within \Q...\E literal bracketing because the backslash willitself be interpreted as a literal. If the terminating delimiter is immediatelyfollowed by a backslash, for example,<pre> /abc/\</pre>a backslash is added to the end of the pattern. This is done to provide a wayof testing the error condition that arises if a pattern finishes with abackslash, because<pre> /abc\/</pre>is interpreted as the first line of a pattern that starts with "abc/", causingpcre2test to read the next line as a continuation of the regular expression.</P><P>A pattern can be followed by a modifier list (details below).</P><br><a name="SEC9" href="#TOC1">SUBJECT LINE SYNTAX</a><br><P>Before each subject line is passed to <b>pcre2_match()</b>,<b>pcre2_dfa_match()</b>, or <b>pcre2_jit_match()</b>, leading and trailing whitespace is removed, and the line is scanned for backslash escapes, unless the<b>subject_literal</b> modifier was set for the pattern. The following provide ameans of encoding non-printing characters in a visible way:<pre> \a alarm (BEL, \x07) \b backspace (\x08) \e escape (\x27) \f form feed (\x0c) \n newline (\x0a) \r carriage return (\x0d) \t tab (\x09) \v vertical tab (\x0b) \nnn octal character (up to 3 octal digits); always a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode \o{dd...} octal character (any number of octal digits} \xhh hexadecimal byte (up to 2 hex digits) \x{hh...} hexadecimal character (any number of hex digits)</pre>The use of \x{hh...} is not dependent on the use of the <b>utf</b> modifier onthe pattern. It is recognized always. There may be any number of hexadecimaldigits inside the braces; invalid values provoke error messages.</P><P>Note that \xhh specifies one byte rather than one character in UTF-8 mode;this makes it possible to construct invalid UTF-8 sequences for testingpurposes. On the other hand, \x{hh} is interpreted as a UTF-8 character inUTF-8 mode, generating more than one byte if the value is greater than 127.When testing the 8-bit library not in UTF-8 mode, \x{hh} generates one bytefor values less than 256, and causes an error for greater values.</P><P>In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes itpossible to construct invalid UTF-16 sequences for testing purposes.</P><P>In UTF-32 mode, all 4- to 8-digit \x{...} values are accepted. This makes itpossible to construct invalid UTF-32 sequences for testing purposes.</P><P>There is a special backslash sequence that specifies replication of one or morecharacters:<pre> \[<characters>]{<count>}</pre>This makes it possible to test long strings without having to provide them aspart of the file. For example:<pre> \[abc]{4}</pre>is converted to "abcabcabcabc". This feature does not support nesting. Toinclude a closing square bracket in the characters, code it as \x5D.</P><P>A backslash followed by an equals sign marks the end of the subject string andthe start of a modifier list. For example:<pre> abc\=notbol,notempty</pre>If the subject string is empty and \= is followed by whitespace, the line istreated as a comment line, and is not used for matching. For example:<pre> \= This is a comment. abc\= This is an invalid modifier list.</pre>A backslash followed by any other non-alphanumeric character just escapes thatcharacter. A backslash followed by anything else causes an error. However, ifthe very last character in the line is a backslash (and there is no modifierlist), it is ignored. This gives a way of passing an empty line as data, sincea real empty line terminates the data input.</P><P>If the <b>subject_literal</b> modifier is set for a pattern, all subject linesthat follow are treated as literals, with no special treatment of backslashes.No replication is possible, and any subject modifiers must be set as defaultsby a <b>#subject</b> command.</P><br><a name="SEC10" href="#TOC1">PATTERN MODIFIERS</a><br><P>There are several types of modifier that can appear in pattern lines. Exceptwhere noted below, they may also be used in <b>#pattern</b> commands. Apattern's modifier list can add to or override default modifiers that were setby a previous <b>#pattern</b> command.<a name="optionmodifiers"></a></P><br><b>Setting compilation options</b><br><P>The following modifiers set options for <b>pcre2_compile()</b>. Most of them setbits in the options argument of that function, but those whose names start withPCRE2_EXTRA are additional options that are set in the compile context.Some of these options have single-letter abbreviations. There is specialhandling for /x: if a second x is present, PCRE2_EXTENDED is converted intoPCRE2_EXTENDED_MORE as in Perl. A third appearance adds PCRE2_EXTENDED as well,though this makes no difference to the way <b>pcre2_compile()</b> behaves. See<a href="pcre2api.html"><b>pcre2api</b></a>for a description of the effects of these options.<pre> allow_empty_class set PCRE2_ALLOW_EMPTY_CLASS allow_lookaround_bsk set PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK allow_surrogate_escapes set PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES alt_bsux set PCRE2_ALT_BSUX alt_circumflex set PCRE2_ALT_CIRCUMFLEX alt_verbnames set PCRE2_ALT_VERBNAMES anchored set PCRE2_ANCHORED /a ascii_all set all ASCII options ascii_bsd set PCRE2_EXTRA_ASCII_BSD ascii_bss set PCRE2_EXTRA_ASCII_BSS ascii_bsw set PCRE2_EXTRA_ASCII_BSW ascii_digit set PCRE2_EXTRA_ASCII_DIGIT ascii_posix set PCRE2_EXTRA_ASCII_POSIX auto_callout set PCRE2_AUTO_CALLOUT bad_escape_is_literal set PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL /i caseless set PCRE2_CASELESS /r caseless_restrict set PCRE2_EXTRA_CASELESS_RESTRICT dollar_endonly set PCRE2_DOLLAR_ENDONLY /s dotall set PCRE2_DOTALL dupnames set PCRE2_DUPNAMES endanchored set PCRE2_ENDANCHORED escaped_cr_is_lf set PCRE2_EXTRA_ESCAPED_CR_IS_LF /x extended set PCRE2_EXTENDED /xx extended_more set PCRE2_EXTENDED_MORE extra_alt_bsux set PCRE2_EXTRA_ALT_BSUX firstline set PCRE2_FIRSTLINE literal set PCRE2_LITERAL match_line set PCRE2_EXTRA_MATCH_LINE match_invalid_utf set PCRE2_MATCH_INVALID_UTF match_unset_backref set PCRE2_MATCH_UNSET_BACKREF match_word set PCRE2_EXTRA_MATCH_WORD /m multiline set PCRE2_MULTILINE never_backslash_c set PCRE2_NEVER_BACKSLASH_C never_ucp set PCRE2_NEVER_UCP never_utf set PCRE2_NEVER_UTF /n no_auto_capture set PCRE2_NO_AUTO_CAPTURE no_auto_possess set PCRE2_NO_AUTO_POSSESS no_dotstar_anchor set PCRE2_NO_DOTSTAR_ANCHOR no_start_optimize set PCRE2_NO_START_OPTIMIZE no_utf_check set PCRE2_NO_UTF_CHECK ucp set PCRE2_UCP ungreedy set PCRE2_UNGREEDY use_offset_limit set PCRE2_USE_OFFSET_LIMIT utf set PCRE2_UTF</pre>As well as turning on the PCRE2_UTF option, the <b>utf</b> modifier causes allnon-printing characters in output strings to be printed using the \x{hh...}notation. Otherwise, those less than 0x100 are output in hex without the curlybrackets. Setting <b>utf</b> in 16-bit or 32-bit mode also causes pattern andsubject strings to be translated to UTF-16 or UTF-32, respectively, beforebeing passed to library functions.<a name="controlmodifiers"></a></P><br><b>Setting compilation controls</b><br><P>The following modifiers affect the compilation process or request informationabout the pattern. There are single-letter abbreviations for some that areheavily used in the test files.<pre> bsr=[anycrlf|unicode] specify \R handling /B bincode show binary code without lengths callout_info show callout information convert=<options> request foreign pattern conversion convert_glob_escape=c set glob escape character convert_glob_separator=c set glob separator character convert_length set convert buffer length debug same as info,fullbincode framesize show matching frame size fullbincode show binary code with lengths /I info show info about compiled pattern hex unquoted characters are hexadecimal jit[=<number>] use JIT jitfast use JIT fast path jitverify verify JIT use locale=<name> use this locale max_pattern_compiled ) set maximum compiled pattern _length=<n> ) length (bytes) max_pattern_length=<n> set maximum pattern length (code units) max_varlookbehind=<n> set maximum variable lookbehind length memory show memory used newline=<type> set newline type null_context compile with a NULL context null_pattern pass pattern as NULL parens_nest_limit=<n> set maximum parentheses depth posix use the POSIX API posix_nosub use the POSIX API with REG_NOSUB push push compiled pattern onto the stack pushcopy push a copy onto the stack stackguard=<number> test the stackguard feature subject_literal treat all subject lines as literal tables=[0|1|2|3] select internal tables use_length do not zero-terminate the pattern utf8_input treat input as UTF-8</pre>The effects of these modifiers are described in the following sections.</P><br><b>Newline and \R handling</b><br><P>The <b>bsr</b> modifier specifies what \R in a pattern should match. If it isset to "anycrlf", \R matches CR, LF, or CRLF only. If it is set to "unicode",\R matches any Unicode newline sequence. The default can be specified whenPCRE2 is built; if it is not, the default is set to Unicode.</P><P>The <b>newline</b> modifier specifies which characters are to be interpreted asnewlines, both in the pattern and in subject lines. The type must be one of CR,LF, CRLF, ANYCRLF, ANY, or NUL (in upper or lower case).</P><br><b>Information about a pattern</b><br><P>The <b>debug</b> modifier is a shorthand for <b>info,fullbincode</b>, requestingall available information.</P><P>The <b>bincode</b> modifier causes a representation of the compiled code to beoutput after compilation. This information does not contain length and offsetvalues, which ensures that the same output is generated for different internallink sizes and different code unit widths. By using <b>bincode</b>, the sameregression tests can be used in different environments.</P><P>The <b>fullbincode</b> modifier, by contrast, <i>does</i> include length andoffset values. This is used in a few special tests that run only for specificcode unit widths and link sizes, and is also useful for one-off tests.</P><P>The <b>info</b> modifier requests information about the compiled pattern(whether it is anchored, has a fixed first character, and so on). Theinformation is obtained from the <b>pcre2_pattern_info()</b> function. Here aresome typical examples:<pre> re> /(?i)(^a|^b)/m,info Capture group count = 1 Compile options: multiline Overall options: caseless multiline First code unit at start or follows newline Subject length lower bound = 1
re> /(?i)abc/info Capture group count = 0 Compile options: <none> Overall options: caseless First code unit = 'a' (caseless) Last code unit = 'c' (caseless) Subject length lower bound = 3</pre>"Compile options" are those specified by modifiers; "overall options" haveadded options that are taken or deduced from the pattern. If both sets ofoptions are the same, just a single "options" line is output; if there are nooptions, the line is omitted. "First code unit" is where any match must start;if there is more than one they are listed as "starting code units". "Last codeunit" is the last literal code unit that must be present in any match. This isnot necessarily the last character. These lines are omitted if no starting orending code units are recorded. The subject length line is omitted when<b>no_start_optimize</b> is set because the minimum length is not calculatedwhen it can never be used.</P><P>The <b>framesize</b> modifier shows the size, in bytes, of each storage frameused by <b>pcre2_match()</b> for handling backtracking. The size depends on thenumber of capturing parentheses in the pattern. A vector of these frames isused at matching time; its overall size is shown when the <b>heaframes_size</b>subject modifier is set.</P><P>The <b>callout_info</b> modifier requests information about all the callouts inthe pattern. A list of them is output at the end of any other information thatis requested. For each callout, either its number or string is given, followedby the item that follows it in the pattern.</P><br><b>Passing a NULL context</b><br><P>Normally, <b>pcre2test</b> passes a context block to <b>pcre2_compile()</b>. Ifthe <b>null_context</b> modifier is set, however, NULL is passed. This is fortesting that <b>pcre2_compile()</b> behaves correctly in this case (it usesdefault values).</P><br><b>Passing a NULL pattern</b><br><P>The <b>null_pattern</b> modifier is for testing the behaviour of<b>pcre2_compile()</b> when the pattern argument is NULL. The length valuepassed is the default PCRE2_ZERO_TERMINATED unless <b>use_length</b> is set.Any length other than zero causes an error.</P><br><b>Specifying pattern characters in hexadecimal</b><br><P>The <b>hex</b> modifier specifies that the characters of the pattern, except forsubstrings enclosed in single or double quotes, are to be interpreted as pairsof hexadecimal digits. This feature is provided as a way of creating patternsthat contain binary zeros and other non-printing characters. White space ispermitted between pairs of digits. For example, this pattern contains threecharacters:<pre> /ab 32 59/hex</pre>Parts of such a pattern are taken literally if quoted. This pattern containsnine characters, only two of which are specified in hexadecimal:<pre> /ab "literal" 32/hex</pre>Either single or double quotes may be used. There is no way of includingthe delimiter within a substring. The <b>hex</b> and <b>expand</b> modifiers aremutually exclusive.</P><br><b>Specifying the pattern's length</b><br><P>By default, patterns are passed to the compiling functions as zero-terminatedstrings but can be passed by length instead of being zero-terminated. The<b>use_length</b> modifier causes this to happen. Using a length happensautomatically (whether or not <b>use_length</b> is set) when <b>hex</b> is set,because patterns specified in hexadecimal may contain binary zeros.</P><P>If <b>hex</b> or <b>use_length</b> is used with the POSIX wrapper API (see<a href="#posixwrapper">"Using the POSIX wrapper API"</a>below), the REG_PEND extension is used to pass the pattern's length.</P><br><b>Specifying a maximum for variable lookbehinds</b><br><P>Variable lookbehind assertions are supported only if, for each one, there is amaximum length (in characters) that it can match. There is a limit on this,whose default can be set at build time, with an ultimate default of 255. The<b>max_varlookbehind</b> modifier uses the <b>pcre2_set_max_varlookbehind()</b>function to change the limit. Lookbehinds whose branches each match a fixedlength are limited to 65535 characters per branch.</P><br><b>Specifying wide characters in 16-bit and 32-bit modes</b><br><P>In 16-bit and 32-bit modes, all input is automatically treated as UTF-8 andtranslated to UTF-16 or UTF-32 when the <b>utf</b> modifier is set. For testingthe 16-bit and 32-bit libraries in non-UTF mode, the <b>utf8_input</b> modifiercan be used. It is mutually exclusive with <b>utf</b>. Input lines areinterpreted as UTF-8 as a means of specifying wide characters. More details aregiven in<a href="#inputencoding">"Input encoding"</a>above.</P><br><b>Generating long repetitive patterns</b><br><P>Some tests use long patterns that are very repetitive. Instead of creating avery long input line for such a pattern, you can use a special repetitionfeature, similar to the one described for subject lines above. If the<b>expand</b> modifier is present on a pattern, parts of the pattern that havethe form<pre> \[<characters>]{<count>}</pre>are expanded before the pattern is passed to <b>pcre2_compile()</b>. Forexample, \[AB]{6000} is expanded to "ABAB..." 6000 times. This constructioncannot be nested. An initial "\[" sequence is recognized only if "]{" followedby decimal digits and "}" is found later in the pattern. If not, the charactersremain in the pattern unaltered. The <b>expand</b> and <b>hex</b> modifiers aremutually exclusive.</P><P>If part of an expanded pattern looks like an expansion, but is really part ofthe actual pattern, unwanted expansion can be avoided by giving two values inthe quantifier. For example, \[AB]{6000,6000} is not recognized as anexpansion item.</P><P>If the <b>info</b> modifier is set on an expanded pattern, the result of theexpansion is included in the information that is output.</P><br><b>JIT compilation</b><br><P>Just-in-time (JIT) compiling is a heavyweight optimization that can greatlyspeed up pattern matching. See the<a href="pcre2jit.html"><b>pcre2jit</b></a>documentation for details. JIT compiling happens, optionally, after a patternhas been successfully compiled into an internal form. The JIT compiler convertsthis to optimized machine code. It needs to know whether the match-time optionsPCRE2_PARTIAL_HARD and PCRE2_PARTIAL_SOFT are going to be used, becausedifferent code is generated for the different cases. See the <b>partial</b>modifier in "Subject Modifiers"<a href="#subjectmodifiers">below</a>for details of how these options are specified for each match attempt.</P><P>JIT compilation is requested by the <b>jit</b> pattern modifier, which mayoptionally be followed by an equals sign and a number in the range 0 to 7.The three bits that make up the number specify which of the three JIT operatingmodes are to be compiled:<pre> 1 compile JIT code for non-partial matching 2 compile JIT code for soft partial matching 4 compile JIT code for hard partial matching</pre>The possible values for the <b>jit</b> modifier are therefore:<pre> 0 disable JIT 1 normal matching only 2 soft partial matching only 3 normal and soft partial matching 4 hard partial matching only 6 soft and hard partial matching only 7 all three modes</pre>If no number is given, 7 is assumed. The phrase "partial matching" means a callto <b>pcre2_match()</b> with either the PCRE2_PARTIAL_SOFT or thePCRE2_PARTIAL_HARD option set. Note that such a call may return a completematch; the options enable the possibility of a partial match, but do notrequire it. Note also that if you request JIT compilation only for partialmatching (for example, jit=2) but do not set the <b>partial</b> modifier on asubject line, that match will not use JIT code because none was compiled fornon-partial matching.</P><P>If JIT compilation is successful, the compiled JIT code will automatically beused when an appropriate type of match is run, except when incompatiblerun-time options are specified. For more details, see the<a href="pcre2jit.html"><b>pcre2jit</b></a>documentation. See also the <b>jitstack</b> modifier below for a way ofsetting the size of the JIT stack.</P><P>If the <b>jitfast</b> modifier is specified, matching is done using the JIT"fast path" interface, <b>pcre2_jit_match()</b>, which skips some of the sanitychecks that are done by <b>pcre2_match()</b>, and of course does not work whenJIT is not supported. If <b>jitfast</b> is specified without <b>jit</b>, jit=7 isassumed.</P><P>If the <b>jitverify</b> modifier is specified, information about the compiledpattern shows whether JIT compilation was or was not successful. If<b>jitverify</b> is specified without <b>jit</b>, jit=7 is assumed. If JITcompilation is successful when <b>jitverify</b> is set, the text "(JIT)" isadded to the first output line after a match or non match when JIT-compiledcode was actually used in the match.</P><br><b>Setting a locale</b><br><P>The <b>locale</b> modifier must specify the name of a locale, for example:<pre> /pattern/locale=fr_FR</pre>The given locale is set, <b>pcre2_maketables()</b> is called to build a set ofcharacter tables for the locale, and this is then passed to<b>pcre2_compile()</b> when compiling the regular expression. The same tablesare used when matching the following subject lines. The <b>locale</b> modifierapplies only to the pattern on which it appears, but can be given in a<b>#pattern</b> command if a default is needed. Setting a locale and alternatecharacter tables are mutually exclusive.</P><br><b>Showing pattern memory</b><br><P>The <b>memory</b> modifier causes the size in bytes of the memory used to holdthe compiled pattern to be output. This does not include the size of the<b>pcre2_code</b> block; it is just the actual compiled data. If the pattern issubsequently passed to the JIT compiler, the size of the JIT compiled code isalso output. Here is an example:<pre> re> /a(b)c/jit,memory Memory allocation (code space): 21 Memory allocation (JIT code): 1910
</PRE></P><br><b>Limiting nested parentheses</b><br><P>The <b>parens_nest_limit</b> modifier sets a limit on the depth of nestedparentheses in a pattern. Breaching the limit causes a compilation error.The default for the library is set when PCRE2 is built, but <b>pcre2test</b>sets its own default of 220, which is required for running the standard testsuite.</P><br><b>Limiting the pattern length</b><br><P>The <b>max_pattern_length</b> modifier sets a limit, in code units, to thelength of pattern that <b>pcre2_compile()</b> will accept. Breaching the limitcauses a compilation error. The default is the largest number a PCRE2_SIZEvariable can hold (essentially unlimited).</P><br><b>Limiting the size of a compiled pattern</b><br><P>The <b>max_pattern_compiled_length</b> modifier sets a limit, in bytes, to theamount of memory used by a compiled pattern. Breaching the limit causes acompilation error. The default is the largest number a PCRE2_SIZE variable canhold (essentially unlimited).<a name="posixwrapper"></a></P><br><b>Using the POSIX wrapper API</b><br><P>The <b>posix</b> and <b>posix_nosub</b> modifiers cause <b>pcre2test</b> to callPCRE2 via the POSIX wrapper API rather than its native API. When<b>posix_nosub</b> is used, the POSIX option REG_NOSUB is passed to<b>regcomp()</b>. The POSIX wrapper supports only the 8-bit library. Note thatit does not imply POSIX matching semantics; for more detail see the<a href="pcre2posix.html"><b>pcre2posix</b></a>documentation. The following pattern modifiers set options for the<b>regcomp()</b> function:<pre> caseless REG_ICASE multiline REG_NEWLINE dotall REG_DOTALL ) ungreedy REG_UNGREEDY ) These options are not part of ucp REG_UCP ) the POSIX standard utf REG_UTF8 )</pre>The <b>regerror_buffsize</b> modifier specifies a size for the error buffer thatis passed to <b>regerror()</b> in the event of a compilation error. For example:<pre> /abc/posix,regerror_buffsize=20</pre>This provides a means of testing the behaviour of <b>regerror()</b> when thebuffer is too small for the error message. If this modifier has not been set, alarge buffer is used.</P><P>The <b>aftertext</b> and <b>allaftertext</b> subject modifiers work as describedbelow. All other modifiers are either ignored, with a warning message, or causean error.</P><P>The pattern is passed to <b>regcomp()</b> as a zero-terminated string bydefault, but if the <b>use_length</b> or <b>hex</b> modifiers are set, theREG_PEND extension is used to pass it by length.</P><br><b>Testing the stack guard feature</b><br><P>The <b>stackguard</b> modifier is used to test the use of<b>pcre2_set_compile_recursion_guard()</b>, a function that is provided toenable stack availability to be checked during compilation (see the<a href="pcre2api.html"><b>pcre2api</b></a>documentation for details). If the number specified by the modifier is greaterthan zero, <b>pcre2_set_compile_recursion_guard()</b> is called to set upcallback from <b>pcre2_compile()</b> to a local function. The argument itreceives is the current nesting parenthesis depth; if this is greater than thevalue given by the modifier, non-zero is returned, causing the compilation tobe aborted.</P><br><b>Using alternative character tables</b><br><P>The value specified for the <b>tables</b> modifier must be one of the digits 0,1, 2, or 3. It causes a specific set of built-in character tables to be passedto <b>pcre2_compile()</b>. This is used in the PCRE2 tests to check behaviourwith different character tables. The digit specifies the tables as follows:<pre> 0 do not pass any special character tables 1 the default ASCII tables, as distributed in pcre2_chartables.c.dist 2 a set of tables defining ISO 8859 characters 3 a set of tables loaded by the #loadtables command</pre>In tables 2, some characters whose codes are greater than 128 are identified asletters, digits, spaces, etc. Tables 3 can be used only after a<b>#loadtables</b> command has loaded them from a binary file. Setting alternatecharacter tables and a locale are mutually exclusive.</P><br><b>Setting certain match controls</b><br><P>The following modifiers are really subject modifiers, and are described under"Subject Modifiers" below. However, they may be included in a pattern'smodifier list, in which case they are applied to every subject line that isprocessed with that pattern. These modifiers do not affect the compilationprocess.<pre> aftertext show text after match allaftertext show text after captures allcaptures show all captures allvector show the entire ovector allusedtext show all consulted text altglobal alternative global matching /g global global matching heapframes_size show match data heapframes size jitstack=<n> set size of JIT stack mark show mark values replace=<string> specify a replacement string startchar show starting character when relevant substitute_callout use substitution callouts substitute_extended use PCRE2_SUBSTITUTE_EXTENDED substitute_literal use PCRE2_SUBSTITUTE_LITERAL substitute_matched use PCRE2_SUBSTITUTE_MATCHED substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY substitute_skip=<n> skip substitution <n> substitute_stop=<n> skip substitution <n> and following substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY</pre>These modifiers may not appear in a <b>#pattern</b> command. If you want them asdefaults, set them in a <b>#subject</b> command.</P><br><b>Specifying literal subject lines</b><br><P>If the <b>subject_literal</b> modifier is present on a pattern, all the subjectlines that it matches are taken as literal strings, with no interpretation ofbackslashes. It is not possible to set subject modifiers on such lines, but anythat are set as defaults by a <b>#subject</b> command are recognized.</P><br><b>Saving a compiled pattern</b><br><P>When a pattern with the <b>push</b> modifier is successfully compiled, it ispushed onto a stack of compiled patterns, and <b>pcre2test</b> expects the nextline to contain a new pattern (or a command) instead of a subject line. Thisfacility is used when saving compiled patterns to a file, as described in thesection entitled "Saving and restoring compiled patterns"<a href="#saverestore">below.</a>If <b>pushcopy</b> is used instead of <b>push</b>, a copy of the compiledpattern is stacked, leaving the original as current, ready to match thefollowing input lines. This provides a way of testing the<b>pcre2_code_copy()</b> function.The <b>push</b> and <b>pushcopy </b> modifiers are incompatible with compilationmodifiers such as <b>global</b> that act at match time. Any that are specifiedare ignored (for the stacked copy), with a warning message, except for<b>replace</b>, which causes an error. Note that <b>jitverify</b>, which isallowed, does not carry through to any subsequent matching that uses a stackedpattern.</P><br><b>Testing foreign pattern conversion</b><br><P>The experimental foreign pattern conversion functions in PCRE2 can be tested bysetting the <b>convert</b> modifier. Its argument is a colon-separated list ofoptions, which set the equivalent option for the <b>pcre2_pattern_convert()</b>function:<pre> glob PCRE2_CONVERT_GLOB glob_no_starstar PCRE2_CONVERT_GLOB_NO_STARSTAR glob_no_wild_separator PCRE2_CONVERT_GLOB_NO_WILD_SEPARATOR posix_basic PCRE2_CONVERT_POSIX_BASIC posix_extended PCRE2_CONVERT_POSIX_EXTENDED unset Unset all options</pre>The "unset" value is useful for turning off a default that has been set by a<b>#pattern</b> command. When one of these options is set, the input pattern ispassed to <b>pcre2_pattern_convert()</b>. If the conversion is successful, theresult is reflected in the output and then passed to <b>pcre2_compile()</b>. Thenormal <b>utf</b> and <b>no_utf_check</b> options, if set, cause thePCRE2_CONVERT_UTF and PCRE2_CONVERT_NO_UTF_CHECK options to be passed to<b>pcre2_pattern_convert()</b>.</P><P>By default, the conversion function is allowed to allocate a buffer for itsoutput. However, if the <b>convert_length</b> modifier is set to a value greaterthan zero, <b>pcre2test</b> passes a buffer of the given length. This makes itpossible to test the length check.</P><P>The <b>convert_glob_escape</b> and <b>convert_glob_separator</b> modifiers can beused to specify the escape and separator characters for glob processing,overriding the defaults, which are operating-system dependent.<a name="subjectmodifiers"></a></P><br><a name="SEC11" href="#TOC1">SUBJECT MODIFIERS</a><br><P>The modifiers that can appear in subject lines and the <b>#subject</b>command are of two types.</P><br><b>Setting match options</b><br><P>The following modifiers set options for <b>pcre2_match()</b> or<b>pcre2_dfa_match()</b>. See<a href="pcreapi.html"><b>pcreapi</b></a>for a description of their effects.<pre> anchored set PCRE2_ANCHORED endanchored set PCRE2_ENDANCHORED dfa_restart set PCRE2_DFA_RESTART dfa_shortest set PCRE2_DFA_SHORTEST disable_recurseloop_check set PCRE2_DISABLE_RECURSELOOP_CHECK no_jit set PCRE2_NO_JIT no_utf_check set PCRE2_NO_UTF_CHECK notbol set PCRE2_NOTBOL notempty set PCRE2_NOTEMPTY notempty_atstart set PCRE2_NOTEMPTY_ATSTART noteol set PCRE2_NOTEOL partial_hard (or ph) set PCRE2_PARTIAL_HARD partial_soft (or ps) set PCRE2_PARTIAL_SOFT</pre>The partial matching modifiers are provided with abbreviations because theyappear frequently in tests.</P><P>If the <b>posix</b> or <b>posix_nosub</b> modifier was present on the pattern,causing the POSIX wrapper API to be used, the only option-setting modifiersthat have any effect are <b>notbol</b>, <b>notempty</b>, and <b>noteol</b>,causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively, to be passed to<b>regexec()</b>. The other modifiers are ignored, with a warning message.</P><P>There is one additional modifier that can be used with the POSIX wrapper. It isignored (with a warning) if used for non-POSIX matching.<pre> posix_startend=<n>[:<m>]</pre>This causes the subject string to be passed to <b>regexec()</b> using theREG_STARTEND option, which uses offsets to specify which part of the string issearched. If only one number is given, the end offset is passed as the end ofthe subject string. For more detail of REG_STARTEND, see the<a href="pcre2posix.html"><b>pcre2posix</b></a>documentation. If the subject string contains binary zeros (coded as escapessuch as \x{00} because <b>pcre2test</b> does not support actual binary zeros inits input), you must use <b>posix_startend</b> to specify its length.</P><br><b>Setting match controls</b><br><P>The following modifiers affect the matching process or request additionalinformation. Some of them may also be specified on a pattern line (see above),in which case they apply to every subject line that is matched against thatpattern, but can be overridden by modifiers on the subject.<pre> aftertext show text after match allaftertext show text after captures allcaptures show all captures allvector show the entire ovector allusedtext show all consulted text (non-JIT only) altglobal alternative global matching callout_capture show captures at callout time callout_data=<n> set a value to pass via callouts callout_error=<n>[:<m>] control callout error callout_extra show extra callout information callout_fail=<n>[:<m>] control callout failure callout_no_where do not show position of a callout callout_none do not supply a callout function copy=<number or name> copy captured substring depth_limit=<n> set a depth limit dfa use <b>pcre2_dfa_match()</b> find_limits find heap, match and depth limits find_limits_noheap find match and depth limits get=<number or name> extract captured substring getall extract all captured substrings /g global global matching heapframes_size show match data heapframes size heap_limit=<n> set a limit on heap memory (Kbytes) jitstack=<n> set size of JIT stack mark show mark values match_limit=<n> set a match limit memory show heap memory usage null_context match with a NULL context null_replacement substitute with NULL replacement null_subject match with NULL subject offset=<n> set starting offset offset_limit=<n> set offset limit ovector=<n> set size of output vector recursion_limit=<n> obsolete synonym for depth_limit replace=<string> specify a replacement string startchar show startchar when relevant startoffset=<n> same as offset=<n> substitute_callout use substitution callouts substitute_extedded use PCRE2_SUBSTITUTE_EXTENDED substitute_literal use PCRE2_SUBSTITUTE_LITERAL substitute_matched use PCRE2_SUBSTITUTE_MATCHED substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY substitute_skip=<n> skip substitution number n substitute_stop=<n> skip substitution number n and greater substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY zero_terminate pass the subject as zero-terminated</pre>The effects of these modifiers are described in the following sections. Whenmatching via the POSIX wrapper API, the <b>aftertext</b>, <b>allaftertext</b>,and <b>ovector</b> subject modifiers work as described below. All othermodifiers are either ignored, with a warning message, or cause an error.</P><br><b>Showing more text</b><br><P>The <b>aftertext</b> modifier requests that as well as outputting the part ofthe subject string that matched the entire pattern, <b>pcre2test</b> should inaddition output the remainder of the subject string. This is useful for testswhere the subject contains multiple copies of the same substring. The<b>allaftertext</b> modifier requests the same action for captured substrings aswell as the main matched substring. In each case the remainder is output on thefollowing line with a plus character following the capture number.</P><P>The <b>allusedtext</b> modifier requests that all the text that was consultedduring a successful pattern match by the interpreter should be shown, for bothfull and partial matches. This feature is not supported for JIT matching, andif requested with JIT it is ignored (with a warning message). Setting thismodifier affects the output if there is a lookbehind at the start of a match,or, for a complete match, a lookahead at the end, or if \K is used in thepattern. Characters that precede or follow the start and end of the actualmatch are indicated in the output by '<' or '>' characters underneath them.Here is an example:<pre> re> /(?<=pqr)abc(?=xyz)/ data> 123pqrabcxyz456\=allusedtext 0: pqrabcxyz <<< >>> data> 123pqrabcxy\=ph,allusedtext Partial match: pqrabcxy <<<</pre>The first, complete match shows that the matched string is "abc", with thepreceding and following strings "pqr" and "xyz" having been consulted duringthe match (when processing the assertions). The partial match can indicate onlythe preceding string.</P><P>The <b>startchar</b> modifier requests that the starting character for the matchbe indicated, if it is different to the start of the matched string. The onlytime when this occurs is when \K has been processed as part of the match. Inthis situation, the output for the matched string is displayed from thestarting character instead of from the match point, with circumflex charactersunder the earlier characters. For example:<pre> re> /abc\Kxyz/ data> abcxyz\=startchar 0: abcxyz ^^^</pre>Unlike <b>allusedtext</b>, the <b>startchar</b> modifier can be used with JIT.However, these two modifiers are mutually exclusive.</P><br><b>Showing the value of all capture groups</b><br><P>The <b>allcaptures</b> modifier requests that the values of all potentialcaptured parentheses be output after a match. By default, only those up to thehighest one actually used in the match are output (corresponding to the returncode from <b>pcre2_match()</b>). Groups that did not take part in the matchare output as "<unset>". This modifier is not relevant for DFA matching (whichdoes no capturing) and does not apply when <b>replace</b> is specified; it isignored, with a warning message, if present.</P><br><b>Showing the entire ovector, for all outcomes</b><br><P>The <b>allvector</b> modifier requests that the entire ovector be shown,whatever the outcome of the match. Compare <b>allcaptures</b>, which shows onlyup to the maximum number of capture groups for the pattern, and then only for asuccessful complete non-DFA match. This modifier, which acts after any matchresult, and also for DFA matching, provides a means of checking that there areno unexpected modifications to ovector fields. Before each match attempt, theovector is filled with a special value, and if this is found in both elementsof a capturing pair, "<unchanged>" is output. After a successful match, thisapplies to all groups after the maximum capture group for the pattern. In othercases it applies to the entire ovector. After a partial match, the first twoelements are the only ones that should be set. After a DFA match, the amount ofovector that is used depends on the number of matches that were found.</P><br><b>Testing pattern callouts</b><br><P>A callout function is supplied when <b>pcre2test</b> calls the library matchingfunctions, unless <b>callout_none</b> is specified. Its behaviour can becontrolled by various modifiers listed above whose names begin with<b>callout_</b>. Details are given in the section entitled "Callouts"<a href="#callouts">below.</a>Testing callouts from <b>pcre2_substitute()</b> is described separately in"Testing the substitution function"<a href="#substitution">below.</a></P><br><b>Finding all matches in a string</b><br><P>Searching for all possible matches within a subject can be requested by the<b>global</b> or <b>altglobal</b> modifier. After finding a match, the matchingfunction is called again to search the remainder of the subject. The differencebetween <b>global</b> and <b>altglobal</b> is that the former uses the<i>start_offset</i> argument to <b>pcre2_match()</b> or <b>pcre2_dfa_match()</b>to start searching at a new point within the entire string (which is what Perldoes), whereas the latter passes over a shortened subject. This makes adifference to the matching process if the pattern begins with a lookbehindassertion (including \b or \B).</P><P>If an empty string is matched, the next match is done with thePCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search foranother, non-empty, match at the same point in the subject. If this matchfails, the start offset is advanced, and the normal match is retried. Thisimitates the way Perl handles such cases when using the <b>/g</b> modifier orthe <b>split()</b> function. Normally, the start offset is advanced by onecharacter, but if the newline convention recognizes CRLF as a newline, and thecurrent character is CR followed by LF, an advance of two characters occurs.</P><br><b>Testing substring extraction functions</b><br><P>The <b>copy</b> and <b>get</b> modifiers can be used to test the<b>pcre2_substring_copy_xxx()</b> and <b>pcre2_substring_get_xxx()</b> functions.They can be given more than once, and each can specify a capture group name ornumber, for example:<pre> abcd\=copy=1,copy=3,get=G1</pre>If the <b>#subject</b> command is used to set default copy and/or get lists,these can be unset by specifying a negative number to cancel all numberedgroups and an empty name to cancel all named groups.</P><P>The <b>getall</b> modifier tests <b>pcre2_substring_list_get()</b>, whichextracts all captured substrings.</P><P>If the subject line is successfully matched, the substrings extracted by theconvenience functions are output with C, G, or L after the string numberinstead of a colon. This is in addition to the normal full list. The stringlength (that is, the return from the extraction function) is given inparentheses after each substring, followed by the name when the extraction wasby name.<a name="substitution"></a></P><br><b>Testing the substitution function</b><br><P>If the <b>replace</b> modifier is set, the <b>pcre2_substitute()</b> function iscalled instead of one of the matching functions (or after one call of<b>pcre2_match()</b> in the case of PCRE2_SUBSTITUTE_MATCHED). Note thatreplacement strings cannot contain commas, because a comma signifies the end ofa modifier. This is not thought to be an issue in a test program.</P><P>Specifying a completely empty replacement string disables this modifier.However, it is possible to specify an empty replacement by providing a bufferlength, as described below, for an otherwise empty replacement.</P><P>Unlike subject strings, <b>pcre2test</b> does not process replacement stringsfor escape sequences. In UTF mode, a replacement string is checked to see if itis a valid UTF-8 string. If so, it is correctly converted to a UTF string ofthe appropriate code unit width. If it is not a valid UTF-8 string, theindividual code units are copied directly. This provides a means of passing aninvalid UTF-8 string for testing purposes.</P><P>The following modifiers set options (in additional to the normal match options)for <b>pcre2_substitute()</b>:<pre> global PCRE2_SUBSTITUTE_GLOBAL substitute_extended PCRE2_SUBSTITUTE_EXTENDED substitute_literal PCRE2_SUBSTITUTE_LITERAL substitute_matched PCRE2_SUBSTITUTE_MATCHED substitute_overflow_length PCRE2_SUBSTITUTE_OVERFLOW_LENGTH substitute_replacement_only PCRE2_SUBSTITUTE_REPLACEMENT_ONLY substitute_unknown_unset PCRE2_SUBSTITUTE_UNKNOWN_UNSET substitute_unset_empty PCRE2_SUBSTITUTE_UNSET_EMPTY</pre>See the<a href="pcre2api.html"><b>pcre2api</b></a>documentation for details of these options.</P><P>After a successful substitution, the modified string is output, preceded by thenumber of replacements. This may be zero if there were no matches. Here is asimple example of a substitution test:<pre> /abc/replace=xxx =abc=abc= 1: =xxx=abc= =abc=abc=\=global 2: =xxx=xxx=</pre>Subject and replacement strings should be kept relatively short (fewer than 256characters) for substitution tests, as fixed-size buffers are used. To make iteasy to test for buffer overflow, if the replacement string starts with anumber in square brackets, that number is passed to <b>pcre2_substitute()</b> asthe size of the output buffer, with the replacement string starting at the nextcharacter. Here is an example that tests the edge case:<pre> /abc/ 123abc123\=replace=[10]XYZ 1: 123XYZ123 123abc123\=replace=[9]XYZ Failed: error -47: no more memory</pre>The default action of <b>pcre2_substitute()</b> is to returnPCRE2_ERROR_NOMEMORY when the output buffer is too small. However, if thePCRE2_SUBSTITUTE_OVERFLOW_LENGTH option is set (by using the<b>substitute_overflow_length</b> modifier), <b>pcre2_substitute()</b> continuesto go through the motions of matching and substituting (but not doing anycallouts), in order to compute the size of buffer that is required. When thishappens, <b>pcre2test</b> shows the required buffer length (which includes spacefor the trailing zero) as part of the error message. For example:<pre> /abc/substitute_overflow_length 123abc123\=replace=[9]XYZ Failed: error -47: no more memory: 10 code units are needed</pre>A replacement string is ignored with POSIX and DFA matching. Specifying partialmatching provokes an error return ("bad option value") from<b>pcre2_substitute()</b>.</P><br><b>Testing substitute callouts</b><br><P>If the <b>substitute_callout</b> modifier is set, a substitution calloutfunction is set up. The <b>null_context</b> modifier must not be set, becausethe address of the callout function is passed in a match context. When thecallout function is called (after each substitution), details of the inputand output strings are output. For example:<pre> /abc/g,replace=<$0>,substitute_callout abcdefabcpqr 1(1) Old 0 3 "abc" New 0 5 "<abc>" 2(1) Old 6 9 "abc" New 8 13 "<abc>" 2: <abc>def<abc>pqr</pre>The first number on each callout line is the count of matches. Theparenthesized number is the number of pairs that are set in the ovector (thatis, one more than the number of capturing groups that were set). Then arelisted the offsets of the old substring, its contents, and the same for thereplacement.</P><P>By default, the substitution callout function returns zero, which accepts thereplacement and causes matching to continue if /g was used. Two furthermodifiers can be used to test other return values. If <b>substitute_skip</b> isset to a value greater than zero the callout function returns +1 for the matchof that number, and similarly <b>substitute_stop</b> returns -1. These cause thereplacement to be rejected, and -1 causes no further matching to take place. Ifeither of them are set, <b>substitute_callout</b> is assumed. For example:<pre> /abc/g,replace=<$0>,substitute_skip=1 abcdefabcpqr 1(1) Old 0 3 "abc" New 0 5 "<abc> SKIPPED" 2(1) Old 6 9 "abc" New 6 11 "<abc>" 2: abcdef<abc>pqr abcdefabcpqr\=substitute_stop=1 1(1) Old 0 3 "abc" New 0 5 "<abc> STOPPED" 1: abcdefabcpqr</pre>If both are set for the same number, stop takes precedence. Only a single skipor stop is supported, which is sufficient for testing that the feature works.</P><br><b>Setting the JIT stack size</b><br><P>The <b>jitstack</b> modifier provides a way of setting the maximum stack sizethat is used by the just-in-time optimization code. It is ignored if JIToptimization is not being used. The value is a number of kibibytes (units of1024 bytes). Setting zero reverts to the default of 32KiB. Providing a stackthat is larger than the default is necessary only for very complicatedpatterns. If <b>jitstack</b> is set non-zero on a subject line it overrides anyvalue that was set on the pattern.</P><br><b>Setting heap, match, and depth limits</b><br><P>The <b>heap_limit</b>, <b>match_limit</b>, and <b>depth_limit</b> modifiers setthe appropriate limits in the match context. These values are ignored when the<b>find_limits</b> or <b>find_limits_noheap</b> modifier is specified.</P><br><b>Finding minimum limits</b><br><P>If the <b>find_limits</b> modifier is present on a subject line, <b>pcre2test</b>calls the relevant matching function several times, setting different values inthe match context via <b>pcre2_set_heap_limit()</b>,<b>pcre2_set_match_limit()</b>, or <b>pcre2_set_depth_limit()</b> until it findsthe smallest value for each parameter that allows the match to complete withouta "limit exceeded" error. The match itself may succeed or fail. An alternativemodifier, <b>find_limits_noheap</b>, omits the heap limit. This is used in thestandard tests, because the minimum heap limit varies between systems. If JITis being used, only the match limit is relevant, and the other two areautomatically omitted.</P><P>When using this modifier, the pattern should not contain any limit settingssuch as (*LIMIT_MATCH=...) within it. If such a setting is present and islower than the minimum matching value, the minimum value cannot be foundbecause <b>pcre2_set_match_limit()</b> etc. are only able to reduce the value ofan in-pattern limit; they cannot increase it.</P><P>For non-DFA matching, the minimum <i>depth_limit</i> number is a measure of howmuch nested backtracking happens (that is, how deeply the pattern's tree issearched). In the case of DFA matching, <i>depth_limit</i> controls the depth ofrecursive calls of the internal function that is used for handling patternrecursion, lookaround assertions, and atomic groups.</P><P>For non-DFA matching, the <i>match_limit</i> number is a measure of the amountof backtracking that takes place, and learning the minimum value can beinstructive. For most simple matches, the number is quite small, but forpatterns with very large numbers of matching possibilities, it can become largevery quickly with increasing length of subject string. In the case of DFAmatching, <i>match_limit</i> controls the total number of calls, both recursiveand non-recursive, to the internal matching function, thus controlling theoverall amount of computing resource that is used.</P><P>For both kinds of matching, the <i>heap_limit</i> number, which is in kibibytes(units of 1024 bytes), limits the amount of heap memory used for matching.</P><br><b>Showing MARK names</b><br><P>The <b>mark</b> modifier causes the names from backtracking control verbs thatare returned from calls to <b>pcre2_match()</b> to be displayed. If a mark isreturned for a match, non-match, or partial match, <b>pcre2test</b> shows it.For a match, it is on a line by itself, tagged with "MK:". Otherwise, itis added to the non-match message.</P><br><b>Showing memory usage</b><br><P>The <b>memory</b> modifier causes <b>pcre2test</b> to log the sizes of all heapmemory allocation and freeing calls that occur during a call to<b>pcre2_match()</b> or <b>pcre2_dfa_match()</b>. In the latter case, heap memoryis used only when a match requires more internal workspace that the defaultallocation on the stack, so in many cases there will be no output. No heapmemory is allocated during matching with JIT. For this modifier to work, the<b>null_context</b> modifier must not be set on both the pattern and thesubject, though it can be set on one or the other.</P><br><b>Showing the heap frame overall vector size</b><br><P>The <b>heapframes_size</b> modifier is relevant for matches using<b>pcre2_match()</b> without JIT. After a match has run (whether successful ornot) the size, in bytes, of the allocated heap frames vector that is leftattached to the match data block is shown. If the matching action involvedseveral calls to <b>pcre2_match()</b> (for example, global matching or fortiming) only the final value is shown.</P><P>This modifier is ignored, with a warning, for POSIX or DFA matching. JITmatching does not use the heap frames vector, so the size is always zero,unless there was a previous non-JIT match. Note that specifing a size of zerofor the output vector (see below) causes <b>pcre2test</b> to free its match datablock (and associated heap frames vector) and allocate a new one.</P><br><b>Setting a starting offset</b><br><P>The <b>offset</b> modifier sets an offset in the subject string at whichmatching starts. Its value is a number of code units, not characters.</P><br><b>Setting an offset limit</b><br><P>The <b>offset_limit</b> modifier sets a limit for unanchored matches. If a matchcannot be found starting at or before this offset in the subject, a "no match"return is given. The data value is a number of code units, not characters. Whenthis modifier is used, the <b>use_offset_limit</b> modifier must have been setfor the pattern; if not, an error is generated.</P><br><b>Setting the size of the output vector</b><br><P>The <b>ovector</b> modifier applies only to the subject line in which itappears, though of course it can also be used to set a default in a<b>#subject</b> command. It specifies the number of pairs of offsets that areavailable for storing matching information. The default is 15.</P><P>A value of zero is useful when testing the POSIX API because it causes<b>regexec()</b> to be called with a NULL capture vector. When not testing thePOSIX API, a value of zero is used to cause<b>pcre2_match_data_create_from_pattern()</b> to be called, in order to create anew match block of exactly the right size for the pattern. (It is not possibleto create a match block with a zero-length ovector; there is always at leastone pair of offsets.) The old match data block is freed.</P><br><b>Passing the subject as zero-terminated</b><br><P>By default, the subject string is passed to a native API matching function withits correct length. In order to test the facility for passing a zero-terminatedstring, the <b>zero_terminate</b> modifier is provided. It causes the length tobe passed as PCRE2_ZERO_TERMINATED. When matching via the POSIX interface,this modifier is ignored, with a warning.</P><P>When testing <b>pcre2_substitute()</b>, this modifier also has the effect ofpassing the replacement string as zero-terminated.</P><br><b>Passing a NULL context, subject, or replacement</b><br><P>Normally, <b>pcre2test</b> passes a context block to <b>pcre2_match()</b>,<b>pcre2_dfa_match()</b>, <b>pcre2_jit_match()</b> or <b>pcre2_substitute()</b>.If the <b>null_context</b> modifier is set, however, NULL is passed. This is fortesting that the matching and substitution functions behave correctly in thiscase (they use default values). This modifier cannot be used with the<b>find_limits</b>, <b>find_limits_noheap</b>, or <b>substitute_callout</b>modifiers.</P><P>Similarly, for testing purposes, if the <b>null_subject</b> or<b>null_replacement</b> modifier is set, the subject or replacement stringpointers are passed as NULL, respectively, to the relevant functions.</P><br><a name="SEC12" href="#TOC1">THE ALTERNATIVE MATCHING FUNCTION</a><br><P>By default, <b>pcre2test</b> uses the standard PCRE2 matching function,<b>pcre2_match()</b> to match each subject line. PCRE2 also supports analternative matching function, <b>pcre2_dfa_match()</b>, which operates in adifferent way, and has some restrictions. The differences between the twofunctions are described in the<a href="pcre2matching.html"><b>pcre2matching</b></a>documentation.</P><P>If the <b>dfa</b> modifier is set, the alternative matching function is used.This function finds all possible matches at a given point in the subject. If,however, the <b>dfa_shortest</b> modifier is set, processing stops after thefirst match is found. This is always the shortest possible match.</P><br><a name="SEC13" href="#TOC1">DEFAULT OUTPUT FROM pcre2test</a><br><P>This section describes the output when the normal matching function,<b>pcre2_match()</b>, is being used.</P><P>When a match succeeds, <b>pcre2test</b> outputs the list of captured substrings,starting with number 0 for the string that matched the whole pattern.Otherwise, it outputs "No match" when the return is PCRE2_ERROR_NOMATCH, or"Partial match:" followed by the partially matching substring when thereturn is PCRE2_ERROR_PARTIAL. (Note that this is theentire substring that was inspected during the partial match; it may includecharacters before the actual match start if a lookbehind assertion, \K, \b,or \B was involved.)</P><P>For any other return, <b>pcre2test</b> outputs the PCRE2 negative error numberand a short descriptive phrase. If the error is a failed UTF string check, thecode unit offset of the start of the failing character is also output. Here isan example of an interactive <b>pcre2test</b> run.<pre> $ pcre2test PCRE2 version 10.22 2016-07-29
re> /^abc(\d+)/ data> abc123 0: abc123 1: 123 data> xyz No match</pre>Unset capturing substrings that are not followed by one that is set are notshown by <b>pcre2test</b> unless the <b>allcaptures</b> modifier is specified. Inthe following example, there are two capturing substrings, but when the firstdata line is matched, the second, unset substring is not shown. An "internal"unset substring is shown as "<unset>", as for the second data line.<pre> re> /(a)|(b)/ data> a 0: a 1: a data> b 0: b 1: <unset> 2: b</pre>If the strings contain any non-printing characters, they are output as \xhhescapes if the value is less than 256 and UTF mode is not set. Otherwise theyare output as \x{hh...} escapes. See below for the definition of non-printingcharacters. If the <b>aftertext</b> modifier is set, the output for substring 0is followed by the rest of the subject string, identified by "0+" like this:<pre> re> /cat/aftertext data> cataract 0: cat 0+ aract</pre>If global matching is requested, the results of successive matching attemptsare output in sequence, like this:<pre> re> /\Bi(\w\w)/g data> Mississippi 0: iss 1: ss 0: iss 1: ss 0: ipp 1: pp</pre>"No match" is output only if the first match attempt fails. Here is an exampleof a failure message (the offset 4 that is specified by the <b>offset</b>modifier is past the end of the subject string):<pre> re> /xyz/ data> xyz\=offset=4 Error -24 (bad offset value)</PRE></P><P>Note that whereas patterns can be continued over several lines (a plain ">"prompt is used for continuations), subject lines may not. However newlines canbe included in a subject by means of the \n escape (or \r, \r\n, etc.,depending on the newline sequence setting).</P><br><a name="SEC14" href="#TOC1">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a><br><P>When the alternative matching function, <b>pcre2_dfa_match()</b>, is used, theoutput consists of a list of all the matches that start at the first point inthe subject where there is at least one match. For example:<pre> re> /(tang|tangerine|tan)/ data> yellow tangerine\=dfa 0: tangerine 1: tang 2: tan</pre>Using the normal matching function on this data finds only "tang". Thelongest matching string is always given first (and numbered zero). After aPCRE2_ERROR_PARTIAL return, the output is "Partial match:", followed by thepartially matching substring. Note that this is the entire substring that wasinspected during the partial match; it may include characters before the actualmatch start if a lookbehind assertion, \b, or \B was involved. (\K is notsupported for DFA matching.)</P><P>If global matching is requested, the search for further matches resumesat the end of the longest match. For example:<pre> re> /(tang|tangerine|tan)/g data> yellow tangerine and tangy sultana\=dfa 0: tangerine 1: tang 2: tan 0: tang 1: tan 0: tan</pre>The alternative matching function does not support substring capture, so themodifiers that are concerned with captured substrings are not relevant.</P><br><a name="SEC15" href="#TOC1">RESTARTING AFTER A PARTIAL MATCH</a><br><P>When the alternative matching function has given the PCRE2_ERROR_PARTIALreturn, indicating that the subject partially matched the pattern, you canrestart the match with additional subject data by means of the<b>dfa_restart</b> modifier. For example:<pre> re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ data> 23ja\=ps,dfa Partial match: 23ja data> n05\=dfa,dfa_restart 0: n05</pre>For further information about partial matching, see the<a href="pcre2partial.html"><b>pcre2partial</b></a>documentation.<a name="callouts"></a></P><br><a name="SEC16" href="#TOC1">CALLOUTS</a><br><P>If the pattern contains any callout requests, <b>pcre2test</b>'s calloutfunction is called during matching unless <b>callout_none</b> is specified. Thisworks with both matching functions, and with JIT, though there are somedifferences in behaviour. The output for callouts with numerical arguments andthose with string arguments is slightly different.</P><br><b>Callouts with numerical arguments</b><br><P>By default, the callout function displays the callout number, the start andcurrent positions in the subject text at the callout time, and the next patternitem to be tested. For example:<pre> --->pqrabcdef 0 ^ ^ \d</pre>This output indicates that callout number 0 occurred for a match attemptstarting at the fourth character of the subject string, when the pointer was atthe seventh character, and when the next pattern item was \d. Justone circumflex is output if the start and current positions are the same, or ifthe current position precedes the start position, which can happen if thecallout is in a lookbehind assertion.</P><P>Callouts numbered 255 are assumed to be automatic callouts, inserted as aresult of the <b>auto_callout</b> pattern modifier. In this case, instead ofshowing the callout number, the offset in the pattern, preceded by a plus, isoutput. For example:<pre> re> /\d?[A-E]\*/auto_callout data> E* --->E* +0 ^ \d? +3 ^ [A-E] +8 ^^ \* +10 ^ ^ 0: E*</pre>If a pattern contains (*MARK) items, an additional line is output whenevera change of latest mark is passed to the callout function. For example:<pre> re> /a(*MARK:X)bc/auto_callout data> abc --->abc +0 ^ a +1 ^^ (*MARK:X) +10 ^^ b Latest Mark: X +11 ^ ^ c +12 ^ ^ 0: abc</pre>The mark changes between matching "a" and "b", but stays the same for the restof the match, so nothing more is output. If, as a result of backtracking, themark reverts to being unset, the text "<unset>" is output.</P><br><b>Callouts with string arguments</b><br><P>The output for a callout with a string argument is similar, except that insteadof outputting a callout number before the position indicators, the calloutstring and its offset in the pattern string are output before the reflection ofthe subject string, and the subject string is reflected for each callout. Forexample:<pre> re> /^ab(?C'first')cd(?C"second")ef/ data> abcdefg Callout (7): 'first' --->abcdefg ^ ^ c Callout (20): "second" --->abcdefg ^ ^ e 0: abcdef
</PRE></P><br><b>Callout modifiers</b><br><P>The callout function in <b>pcre2test</b> returns zero (carry on matching) bydefault, but you can use a <b>callout_fail</b> modifier in a subject line tochange this and other parameters of the callout (see below).</P><P>If the <b>callout_capture</b> modifier is set, the current captured groups areoutput when a callout occurs. This is useful only for non-DFA matching, as<b>pcre2_dfa_match()</b> does not support capturing, so no captures are evershown.</P><P>The normal callout output, showing the callout number or pattern offset (asdescribed above) is suppressed if the <b>callout_no_where</b> modifier is set.</P><P>When using the interpretive matching function <b>pcre2_match()</b> without JIT,setting the <b>callout_extra</b> modifier causes additional output from<b>pcre2test</b>'s callout function to be generated. For the first callout in amatch attempt at a new starting position in the subject, "New match attempt" isoutput. If there has been a backtrack since the last callout (or start ofmatching if this is the first callout), "Backtrack" is output, followed by "Noother matching paths" if the backtrack ended the previous match attempt. Forexample:<pre> re> /(a+)b/auto_callout,no_start_optimize,no_auto_possess data> aac\=callout_extra New match attempt --->aac +0 ^ ( +1 ^ a+ +3 ^ ^ ) +4 ^ ^ b Backtrack --->aac +3 ^^ ) +4 ^^ b Backtrack No other matching paths New match attempt --->aac +0 ^ ( +1 ^ a+ +3 ^^ ) +4 ^^ b Backtrack No other matching paths New match attempt --->aac +0 ^ ( +1 ^ a+ Backtrack No other matching paths New match attempt --->aac +0 ^ ( +1 ^ a+ No match</pre>Notice that various optimizations must be turned off if you want all possiblematching paths to be scanned. If <b>no_start_optimize</b> is not used, there isan immediate "no match", without any callouts, because the startingoptimization fails to find "b" in the subject, which it knows must be presentfor any match. If <b>no_auto_possess</b> is not used, the "a+" item is turnedinto "a++", which reduces the number of backtracks.</P><P>The <b>callout_extra</b> modifier has no effect if used with the DFA matchingfunction, or with JIT.</P><br><b>Return values from callouts</b><br><P>The default return from the callout function is zero, which allows matching tocontinue. The <b>callout_fail</b> modifier can be given one or two numbers. Ifthere is only one number, 1 is returned instead of 0 (causing matching tobacktrack) when a callout of that number is reached. If two numbers (<n>:<m>)are given, 1 is returned when callout <n> is reached and there have been atleast <m> callouts. The <b>callout_error</b> modifier is similar, except thatPCRE2_ERROR_CALLOUT is returned, causing the entire matching process to beaborted. If both these modifiers are set for the same callout number,<b>callout_error</b> takes precedence. Note that callouts with string argumentsare always given the number zero.</P><P>The <b>callout_data</b> modifier can be given an unsigned or a negative number.This is set as the "user data" that is passed to the matching function, andpassed back when the callout function is invoked. Any value other than zero isused as a return from <b>pcre2test</b>'s callout function.</P><P>Inserting callouts can be helpful when using <b>pcre2test</b> to checkcomplicated regular expressions. For further information about callouts, seethe<a href="pcre2callout.html"><b>pcre2callout</b></a>documentation.</P><br><a name="SEC17" href="#TOC1">NON-PRINTING CHARACTERS</a><br><P>When <b>pcre2test</b> is outputting text in the compiled version of a pattern,bytes other than 32-126 are always treated as non-printing characters and aretherefore shown as hex escapes.</P><P>When <b>pcre2test</b> is outputting text that is a matched part of a subjectstring, it behaves in the same way, unless a different locale has been set forthe pattern (using the <b>locale</b> modifier). In this case, the<b>isprint()</b> function is used to distinguish printing and non-printingcharacters.<a name="saverestore"></a></P><br><a name="SEC18" href="#TOC1">SAVING AND RESTORING COMPILED PATTERNS</a><br><P>It is possible to save compiled patterns on disc or elsewhere, and reload themlater, subject to a number of restrictions. JIT data cannot be saved. The hoston which the patterns are reloaded must be running the same version of PCRE2,with the same code unit width, and must also have the same endianness, pointerwidth and PCRE2_SIZE type. Before compiled patterns can be saved they must beserialized, that is, converted to a stream of bytes. A single byte stream maycontain any number of compiled patterns, but they must all use the samecharacter tables. A single copy of the tables is included in the byte stream(its size is 1088 bytes).</P><P>The functions whose names begin with <b>pcre2_serialize_</b> are usedfor serializing and de-serializing. They are described in the<a href="pcre2serialize.html"><b>pcre2serialize</b></a>documentation. In this section we describe the features of <b>pcre2test</b> thatcan be used to test these functions.</P><P>Note that "serialization" in PCRE2 does not convert compiled patterns to anabstract format like Java or .NET. It just makes a reloadable byte code stream.Hence the restrictions on reloading mentioned above.</P><P>In <b>pcre2test</b>, when a pattern with <b>push</b> modifier is successfullycompiled, it is pushed onto a stack of compiled patterns, and <b>pcre2test</b>expects the next line to contain a new pattern (or command) instead of asubject line. By contrast, the <b>pushcopy</b> modifier causes a copy of thecompiled pattern to be stacked, leaving the original available for immediatematching. By using <b>push</b> and/or <b>pushcopy</b>, a number of patterns canbe compiled and retained. These modifiers are incompatible with <b>posix</b>,and control modifiers that act at match time are ignored (with a message) forthe stacked patterns. The <b>jitverify</b> modifier applies only at compiletime.</P><P>The command<pre> #save <filename></pre>causes all the stacked patterns to be serialized and the result written to thenamed file. Afterwards, all the stacked patterns are freed. The command<pre> #load <filename></pre>reads the data in the file, and then arranges for it to be de-serialized, withthe resulting compiled patterns added to the pattern stack. The pattern on thetop of the stack can be retrieved by the #pop command, which must be followedby lines of subjects that are to be matched with the pattern, terminated asusual by an empty line or end of file. This command may be followed by amodifier list containing only<a href="#controlmodifiers">control modifiers</a>that act after a pattern has been compiled. In particular, <b>hex</b>,<b>posix</b>, <b>posix_nosub</b>, <b>push</b>, and <b>pushcopy</b> are not allowed,nor are any<a href="#optionmodifiers">option-setting modifiers.</a>The JIT modifiers are, however permitted. Here is an example that saves andreloads two patterns.<pre> /abc/push /xyz/push #save tempfile #load tempfile #pop info xyz
#pop jit,bincode abc</pre>If <b>jitverify</b> is used with #pop, it does not automatically imply<b>jit</b>, which is different behaviour from when it is used on a pattern.</P><P>The #popcopy command is analogous to the <b>pushcopy</b> modifier in that itmakes current a copy of the topmost stack pattern, leaving the original stillon the stack.</P><br><a name="SEC19" href="#TOC1">SEE ALSO</a><br><P><b>pcre2</b>(3), <b>pcre2api</b>(3), <b>pcre2callout</b>(3),<b>pcre2jit</b>, <b>pcre2matching</b>(3), <b>pcre2partial</b>(d),<b>pcre2pattern</b>(3), <b>pcre2serialize</b>(3).</P><br><a name="SEC20" href="#TOC1">AUTHOR</a><br><P>Philip Hazel<br>Retired from University Computing Service<br>Cambridge, England.<br></P><br><a name="SEC21" href="#TOC1">REVISION</a><br><P>Last updated: 24 April 2024<br>Copyright © 1997-2024 University of Cambridge.<br><p>Return to the <a href="index.html">PCRE2 index page</a>.</p>
|
