tools/pcre/README
author David Anderson <dvander@alliedmods.net>
Thu Oct 29 00:06:12 2009 -0700 (2009-10-29)
changeset 2844 87f951bece2d
parent 2762 d2d467bf1b5a
child 3831 babd0eee3c5e
permissions -rw-r--r--
Fix iloop race when using AddCommandListener and SourceMod unloads (bug 4077, r=pred).
dvander@2762
     1
README file for PCRE (Perl-compatible regular expression library)
dvander@2762
     2
-----------------------------------------------------------------
dvander@2762
     3
dvander@2762
     4
The latest release of PCRE is always available in three alternative formats
dvander@2762
     5
from:
dvander@2762
     6
dvander@2762
     7
  ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
dvander@2762
     8
  ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.bz2
dvander@2762
     9
  ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.zip
dvander@2762
    10
dvander@2762
    11
There is a mailing list for discussion about the development of PCRE at
dvander@2762
    12
dvander@2762
    13
  [email protected]
dvander@2762
    14
dvander@2762
    15
Please read the NEWS file if you are upgrading from a previous release.
dvander@2762
    16
The contents of this README file are:
dvander@2762
    17
dvander@2762
    18
  The PCRE APIs
dvander@2762
    19
  Documentation for PCRE
dvander@2762
    20
  Contributions by users of PCRE
dvander@2762
    21
  Building PCRE on non-Unix systems
dvander@2762
    22
  Building PCRE on Unix-like systems
dvander@2762
    23
  Retrieving configuration information on Unix-like systems
dvander@2762
    24
  Shared libraries on Unix-like systems
dvander@2762
    25
  Cross-compiling on Unix-like systems
dvander@2762
    26
  Using HP's ANSI C++ compiler (aCC)
dvander@2762
    27
  Making new tarballs
dvander@2762
    28
  Testing PCRE
dvander@2762
    29
  Character tables
dvander@2762
    30
  File manifest
dvander@2762
    31
dvander@2762
    32
dvander@2762
    33
The PCRE APIs
dvander@2762
    34
-------------
dvander@2762
    35
dvander@2762
    36
PCRE is written in C, and it has its own API. The distribution also includes a
dvander@2762
    37
set of C++ wrapper functions (see the pcrecpp man page for details), courtesy
dvander@2762
    38
of Google Inc.
dvander@2762
    39
dvander@2762
    40
In addition, there is a set of C wrapper functions that are based on the POSIX
dvander@2762
    41
regular expression API (see the pcreposix man page). These end up in the
dvander@2762
    42
library called libpcreposix. Note that this just provides a POSIX calling
dvander@2762
    43
interface to PCRE; the regular expressions themselves still follow Perl syntax
dvander@2762
    44
and semantics. The POSIX API is restricted, and does not give full access to
dvander@2762
    45
all of PCRE's facilities.
dvander@2762
    46
dvander@2762
    47
The header file for the POSIX-style functions is called pcreposix.h. The
dvander@2762
    48
official POSIX name is regex.h, but I did not want to risk possible problems
dvander@2762
    49
with existing files of that name by distributing it that way. To use PCRE with
dvander@2762
    50
an existing program that uses the POSIX API, pcreposix.h will have to be
dvander@2762
    51
renamed or pointed at by a link.
dvander@2762
    52
dvander@2762
    53
If you are using the POSIX interface to PCRE and there is already a POSIX regex
dvander@2762
    54
library installed on your system, as well as worrying about the regex.h header
dvander@2762
    55
file (as mentioned above), you must also take care when linking programs to
dvander@2762
    56
ensure that they link with PCRE's libpcreposix library. Otherwise they may pick
dvander@2762
    57
up the POSIX functions of the same name from the other library.
dvander@2762
    58
dvander@2762
    59
One way of avoiding this confusion is to compile PCRE with the addition of
dvander@2762
    60
-Dregcomp=PCREregcomp (and similarly for the other POSIX functions) to the
dvander@2762
    61
compiler flags (CFLAGS if you are using "configure" -- see below). This has the
dvander@2762
    62
effect of renaming the functions so that the names no longer clash. Of course,
dvander@2762
    63
you have to do the same thing for your applications, or write them using the
dvander@2762
    64
new names.
dvander@2762
    65
dvander@2762
    66
dvander@2762
    67
Documentation for PCRE
dvander@2762
    68
----------------------
dvander@2762
    69
dvander@2762
    70
If you install PCRE in the normal way on a Unix-like system, you will end up
dvander@2762
    71
with a set of man pages whose names all start with "pcre". The one that is just
dvander@2762
    72
called "pcre" lists all the others. In addition to these man pages, the PCRE
dvander@2762
    73
documentation is supplied in two other forms:
dvander@2762
    74
dvander@2762
    75
  1. There are files called doc/pcre.txt, doc/pcregrep.txt, and
dvander@2762
    76
     doc/pcretest.txt in the source distribution. The first of these is a
dvander@2762
    77
     concatenation of the text forms of all the section 3 man pages except
dvander@2762
    78
     those that summarize individual functions. The other two are the text
dvander@2762
    79
     forms of the section 1 man pages for the pcregrep and pcretest commands.
dvander@2762
    80
     These text forms are provided for ease of scanning with text editors or
dvander@2762
    81
     similar tools. They are installed in <prefix>/share/doc/pcre, where
dvander@2762
    82
     <prefix> is the installation prefix (defaulting to /usr/local).
dvander@2762
    83
dvander@2762
    84
  2. A set of files containing all the documentation in HTML form, hyperlinked
dvander@2762
    85
     in various ways, and rooted in a file called index.html, is distributed in
dvander@2762
    86
     doc/html and installed in <prefix>/share/doc/pcre/html.
dvander@2762
    87
dvander@2762
    88
Users of PCRE have contributed files containing the documentation for various
dvander@2762
    89
releases in CHM format. These can be found in the Contrib directory of the FTP
dvander@2762
    90
site (see next section).
dvander@2762
    91
dvander@2762
    92
dvander@2762
    93
Contributions by users of PCRE
dvander@2762
    94
------------------------------
dvander@2762
    95
dvander@2762
    96
You can find contributions from PCRE users in the directory
dvander@2762
    97
dvander@2762
    98
  ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
dvander@2762
    99
dvander@2762
   100
There is a README file giving brief descriptions of what they are. Some are
dvander@2762
   101
complete in themselves; others are pointers to URLs containing relevant files.
dvander@2762
   102
Some of this material is likely to be well out-of-date. Several of the earlier
dvander@2762
   103
contributions provided support for compiling PCRE on various flavours of
dvander@2762
   104
Windows (I myself do not use Windows). Nowadays there is more Windows support
dvander@2762
   105
in the standard distribution, so these contibutions have been archived.
dvander@2762
   106
dvander@2762
   107
dvander@2762
   108
Building PCRE on non-Unix systems
dvander@2762
   109
---------------------------------
dvander@2762
   110
dvander@2762
   111
For a non-Unix system, please read the comments in the file NON-UNIX-USE,
dvander@2762
   112
though if your system supports the use of "configure" and "make" you may be
dvander@2762
   113
able to build PCRE in the same way as for Unix-like systems. PCRE can also be
dvander@2762
   114
configured in many platform environments using the GUI facility of CMake's
dvander@2762
   115
CMakeSetup. It creates Makefiles, solution files, etc.
dvander@2762
   116
dvander@2762
   117
PCRE has been compiled on many different operating systems. It should be
dvander@2762
   118
straightforward to build PCRE on any system that has a Standard C compiler and
dvander@2762
   119
library, because it uses only Standard C functions.
dvander@2762
   120
dvander@2762
   121
dvander@2762
   122
Building PCRE on Unix-like systems
dvander@2762
   123
----------------------------------
dvander@2762
   124
dvander@2762
   125
If you are using HP's ANSI C++ compiler (aCC), please see the special note
dvander@2762
   126
in the section entitled "Using HP's ANSI C++ compiler (aCC)" below.
dvander@2762
   127
dvander@2762
   128
The following instructions assume the use of the widely used "configure, make,
dvander@2762
   129
make install" process. There is also support for CMake in the PCRE
dvander@2762
   130
distribution; there are some comments about using CMake in the NON-UNIX-USE
dvander@2762
   131
file, though it can also be used in Unix-like systems.
dvander@2762
   132
dvander@2762
   133
To build PCRE on a Unix-like system, first run the "configure" command from the
dvander@2762
   134
PCRE distribution directory, with your current directory set to the directory
dvander@2762
   135
where you want the files to be created. This command is a standard GNU
dvander@2762
   136
"autoconf" configuration script, for which generic instructions are supplied in
dvander@2762
   137
the file INSTALL.
dvander@2762
   138
dvander@2762
   139
Most commonly, people build PCRE within its own distribution directory, and in
dvander@2762
   140
this case, on many systems, just running "./configure" is sufficient. However,
dvander@2762
   141
the usual methods of changing standard defaults are available. For example:
dvander@2762
   142
dvander@2762
   143
CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
dvander@2762
   144
dvander@2762
   145
specifies that the C compiler should be run with the flags '-O2 -Wall' instead
dvander@2762
   146
of the default, and that "make install" should install PCRE under /opt/local
dvander@2762
   147
instead of the default /usr/local.
dvander@2762
   148
dvander@2762
   149
If you want to build in a different directory, just run "configure" with that
dvander@2762
   150
directory as current. For example, suppose you have unpacked the PCRE source
dvander@2762
   151
into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
dvander@2762
   152
dvander@2762
   153
cd /build/pcre/pcre-xxx
dvander@2762
   154
/source/pcre/pcre-xxx/configure
dvander@2762
   155
dvander@2762
   156
PCRE is written in C and is normally compiled as a C library. However, it is
dvander@2762
   157
possible to build it as a C++ library, though the provided building apparatus
dvander@2762
   158
does not have any features to support this.
dvander@2762
   159
dvander@2762
   160
There are some optional features that can be included or omitted from the PCRE
dvander@2762
   161
library. You can read more about them in the pcrebuild man page.
dvander@2762
   162
dvander@2762
   163
. If you want to suppress the building of the C++ wrapper library, you can add
dvander@2762
   164
  --disable-cpp to the "configure" command. Otherwise, when "configure" is run,
dvander@2762
   165
  it will try to find a C++ compiler and C++ header files, and if it succeeds,
dvander@2762
   166
  it will try to build the C++ wrapper.
dvander@2762
   167
dvander@2762
   168
. If you want to make use of the support for UTF-8 Unicode character strings in
dvander@2762
   169
  PCRE, you must add --enable-utf8 to the "configure" command. Without it, the
dvander@2762
   170
  code for handling UTF-8 is not included in the library. Even when included,
dvander@2762
   171
  it still has to be enabled by an option at run time. When PCRE is compiled
dvander@2762
   172
  with this option, its input can only either be ASCII or UTF-8, even when
dvander@2762
   173
  running on EBCDIC platforms. It is not possible to use both --enable-utf8 and
dvander@2762
   174
  --enable-ebcdic at the same time.
dvander@2762
   175
dvander@2762
   176
. If, in addition to support for UTF-8 character strings, you want to include
dvander@2762
   177
  support for the \P, \p, and \X sequences that recognize Unicode character
dvander@2762
   178
  properties, you must add --enable-unicode-properties to the "configure"
dvander@2762
   179
  command. This adds about 30K to the size of the library (in the form of a
dvander@2762
   180
  property table); only the basic two-letter properties such as Lu are
dvander@2762
   181
  supported.
dvander@2762
   182
dvander@2762
   183
. You can build PCRE to recognize either CR or LF or the sequence CRLF or any
dvander@2762
   184
  of the preceding, or any of the Unicode newline sequences as indicating the
dvander@2762
   185
  end of a line. Whatever you specify at build time is the default; the caller
dvander@2762
   186
  of PCRE can change the selection at run time. The default newline indicator
dvander@2762
   187
  is a single LF character (the Unix standard). You can specify the default
dvander@2762
   188
  newline indicator by adding --enable-newline-is-cr or --enable-newline-is-lf
dvander@2762
   189
  or --enable-newline-is-crlf or --enable-newline-is-anycrlf or
dvander@2762
   190
  --enable-newline-is-any to the "configure" command, respectively.
dvander@2762
   191
dvander@2762
   192
  If you specify --enable-newline-is-cr or --enable-newline-is-crlf, some of
dvander@2762
   193
  the standard tests will fail, because the lines in the test files end with
dvander@2762
   194
  LF. Even if the files are edited to change the line endings, there are likely
dvander@2762
   195
  to be some failures. With --enable-newline-is-anycrlf or
dvander@2762
   196
  --enable-newline-is-any, many tests should succeed, but there may be some
dvander@2762
   197
  failures.
dvander@2762
   198
dvander@2762
   199
. By default, the sequence \R in a pattern matches any Unicode line ending
dvander@2762
   200
  sequence. This is independent of the option specifying what PCRE considers to
dvander@2762
   201
  be the end of a line (see above). However, the caller of PCRE can restrict \R
dvander@2762
   202
  to match only CR, LF, or CRLF. You can make this the default by adding
dvander@2762
   203
  --enable-bsr-anycrlf to the "configure" command (bsr = "backslash R").
dvander@2762
   204
dvander@2762
   205
. When called via the POSIX interface, PCRE uses malloc() to get additional
dvander@2762
   206
  storage for processing capturing parentheses if there are more than 10 of
dvander@2762
   207
  them in a pattern. You can increase this threshold by setting, for example,
dvander@2762
   208
dvander@2762
   209
  --with-posix-malloc-threshold=20
dvander@2762
   210
dvander@2762
   211
  on the "configure" command.
dvander@2762
   212
dvander@2762
   213
. PCRE has a counter that can be set to limit the amount of resources it uses.
dvander@2762
   214
  If the limit is exceeded during a match, the match fails. The default is ten
dvander@2762
   215
  million. You can change the default by setting, for example,
dvander@2762
   216
dvander@2762
   217
  --with-match-limit=500000
dvander@2762
   218
dvander@2762
   219
  on the "configure" command. This is just the default; individual calls to
dvander@2762
   220
  pcre_exec() can supply their own value. There is more discussion on the
dvander@2762
   221
  pcreapi man page.
dvander@2762
   222
dvander@2762
   223
. There is a separate counter that limits the depth of recursive function calls
dvander@2762
   224
  during a matching process. This also has a default of ten million, which is
dvander@2762
   225
  essentially "unlimited". You can change the default by setting, for example,
dvander@2762
   226
dvander@2762
   227
  --with-match-limit-recursion=500000
dvander@2762
   228
dvander@2762
   229
  Recursive function calls use up the runtime stack; running out of stack can
dvander@2762
   230
  cause programs to crash in strange ways. There is a discussion about stack
dvander@2762
   231
  sizes in the pcrestack man page.
dvander@2762
   232
dvander@2762
   233
. The default maximum compiled pattern size is around 64K. You can increase
dvander@2762
   234
  this by adding --with-link-size=3 to the "configure" command. You can
dvander@2762
   235
  increase it even more by setting --with-link-size=4, but this is unlikely
dvander@2762
   236
  ever to be necessary. Increasing the internal link size will reduce
dvander@2762
   237
  performance.
dvander@2762
   238
dvander@2762
   239
. You can build PCRE so that its internal match() function that is called from
dvander@2762
   240
  pcre_exec() does not call itself recursively. Instead, it uses memory blocks
dvander@2762
   241
  obtained from the heap via the special functions pcre_stack_malloc() and
dvander@2762
   242
  pcre_stack_free() to save data that would otherwise be saved on the stack. To
dvander@2762
   243
  build PCRE like this, use
dvander@2762
   244
dvander@2762
   245
  --disable-stack-for-recursion
dvander@2762
   246
dvander@2762
   247
  on the "configure" command. PCRE runs more slowly in this mode, but it may be
dvander@2762
   248
  necessary in environments with limited stack sizes. This applies only to the
dvander@2762
   249
  pcre_exec() function; it does not apply to pcre_dfa_exec(), which does not
dvander@2762
   250
  use deeply nested recursion. There is a discussion about stack sizes in the
dvander@2762
   251
  pcrestack man page.
dvander@2762
   252
dvander@2762
   253
. For speed, PCRE uses four tables for manipulating and identifying characters
dvander@2762
   254
  whose code point values are less than 256. By default, it uses a set of
dvander@2762
   255
  tables for ASCII encoding that is part of the distribution. If you specify
dvander@2762
   256
dvander@2762
   257
  --enable-rebuild-chartables
dvander@2762
   258
dvander@2762
   259
  a program called dftables is compiled and run in the default C locale when
dvander@2762
   260
  you obey "make". It builds a source file called pcre_chartables.c. If you do
dvander@2762
   261
  not specify this option, pcre_chartables.c is created as a copy of
dvander@2762
   262
  pcre_chartables.c.dist. See "Character tables" below for further information.
dvander@2762
   263
dvander@2762
   264
. It is possible to compile PCRE for use on systems that use EBCDIC as their
dvander@2762
   265
  character code (as opposed to ASCII) by specifying
dvander@2762
   266
dvander@2762
   267
  --enable-ebcdic
dvander@2762
   268
dvander@2762
   269
  This automatically implies --enable-rebuild-chartables (see above). However,
dvander@2762
   270
  when PCRE is built this way, it always operates in EBCDIC. It cannot support
dvander@2762
   271
  both EBCDIC and UTF-8.
dvander@2762
   272
dvander@2762
   273
. It is possible to compile pcregrep to use libz and/or libbz2, in order to
dvander@2762
   274
  read .gz and .bz2 files (respectively), by specifying one or both of
dvander@2762
   275
dvander@2762
   276
  --enable-pcregrep-libz
dvander@2762
   277
  --enable-pcregrep-libbz2
dvander@2762
   278
dvander@2762
   279
  Of course, the relevant libraries must be installed on your system.
dvander@2762
   280
dvander@2762
   281
. It is possible to compile pcretest so that it links with the libreadline
dvander@2762
   282
  library, by specifying
dvander@2762
   283
dvander@2762
   284
  --enable-pcretest-libreadline
dvander@2762
   285
dvander@2762
   286
  If this is done, when pcretest's input is from a terminal, it reads it using
dvander@2762
   287
  the readline() function. This provides line-editing and history facilities.
dvander@2762
   288
  Note that libreadline is GPL-licenced, so if you distribute a binary of
dvander@2762
   289
  pcretest linked in this way, there may be licensing issues.
dvander@2762
   290
dvander@2762
   291
  Setting this option causes the -lreadline option to be added to the pcretest
dvander@2762
   292
  build. In many operating environments with a sytem-installed readline
dvander@2762
   293
  library this is sufficient. However, in some environments (e.g. if an
dvander@2762
   294
  unmodified distribution version of readline is in use), it may be necessary
dvander@2762
   295
  to specify something like LIBS="-lncurses" as well. This is because, to quote
dvander@2762
   296
  the readline INSTALL, "Readline uses the termcap functions, but does not link
dvander@2762
   297
  with the termcap or curses library itself, allowing applications which link
dvander@2762
   298
  with readline the to choose an appropriate library." If you get error
dvander@2762
   299
  messages about missing functions tgetstr, tgetent, tputs, tgetflag, or tgoto,
dvander@2762
   300
  this is the problem, and linking with the ncurses library should fix it.
dvander@2762
   301
dvander@2762
   302
The "configure" script builds the following files for the basic C library:
dvander@2762
   303
dvander@2762
   304
. Makefile is the makefile that builds the library
dvander@2762
   305
. config.h contains build-time configuration options for the library
dvander@2762
   306
. pcre.h is the public PCRE header file
dvander@2762
   307
. pcre-config is a script that shows the settings of "configure" options
dvander@2762
   308
. libpcre.pc is data for the pkg-config command
dvander@2762
   309
. libtool is a script that builds shared and/or static libraries
dvander@2762
   310
. RunTest is a script for running tests on the basic C library
dvander@2762
   311
. RunGrepTest is a script for running tests on the pcregrep command
dvander@2762
   312
dvander@2762
   313
Versions of config.h and pcre.h are distributed in the PCRE tarballs under
dvander@2762
   314
the names config.h.generic and pcre.h.generic. These are provided for the
dvander@2762
   315
benefit of those who have to built PCRE without the benefit of "configure". If
dvander@2762
   316
you use "configure", the .generic versions are not used.
dvander@2762
   317
dvander@2762
   318
If a C++ compiler is found, the following files are also built:
dvander@2762
   319
dvander@2762
   320
. libpcrecpp.pc is data for the pkg-config command
dvander@2762
   321
. pcrecpparg.h is a header file for programs that call PCRE via the C++ wrapper
dvander@2762
   322
. pcre_stringpiece.h is the header for the C++ "stringpiece" functions
dvander@2762
   323
dvander@2762
   324
The "configure" script also creates config.status, which is an executable
dvander@2762
   325
script that can be run to recreate the configuration, and config.log, which
dvander@2762
   326
contains compiler output from tests that "configure" runs.
dvander@2762
   327
dvander@2762
   328
Once "configure" has run, you can run "make". It builds two libraries, called
dvander@2762
   329
libpcre and libpcreposix, a test program called pcretest, and the pcregrep
dvander@2762
   330
command. If a C++ compiler was found on your system, "make" also builds the C++
dvander@2762
   331
wrapper library, which is called libpcrecpp, and some test programs called
dvander@2762
   332
pcrecpp_unittest, pcre_scanner_unittest, and pcre_stringpiece_unittest.
dvander@2762
   333
Building the C++ wrapper can be disabled by adding --disable-cpp to the
dvander@2762
   334
"configure" command.
dvander@2762
   335
dvander@2762
   336
The command "make check" runs all the appropriate tests. Details of the PCRE
dvander@2762
   337
tests are given below in a separate section of this document.
dvander@2762
   338
dvander@2762
   339
You can use "make install" to install PCRE into live directories on your
dvander@2762
   340
system. The following are installed (file names are all relative to the
dvander@2762
   341
<prefix> that is set when "configure" is run):
dvander@2762
   342
dvander@2762
   343
  Commands (bin):
dvander@2762
   344
    pcretest
dvander@2762
   345
    pcregrep
dvander@2762
   346
    pcre-config
dvander@2762
   347
dvander@2762
   348
  Libraries (lib):
dvander@2762
   349
    libpcre
dvander@2762
   350
    libpcreposix
dvander@2762
   351
    libpcrecpp (if C++ support is enabled)
dvander@2762
   352
dvander@2762
   353
  Configuration information (lib/pkgconfig):
dvander@2762
   354
    libpcre.pc
dvander@2762
   355
    libpcrecpp.pc (if C++ support is enabled)
dvander@2762
   356
dvander@2762
   357
  Header files (include):
dvander@2762
   358
    pcre.h
dvander@2762
   359
    pcreposix.h
dvander@2762
   360
    pcre_scanner.h      )
dvander@2762
   361
    pcre_stringpiece.h  ) if C++ support is enabled
dvander@2762
   362
    pcrecpp.h           )
dvander@2762
   363
    pcrecpparg.h        )
dvander@2762
   364
dvander@2762
   365
  Man pages (share/man/man{1,3}):
dvander@2762
   366
    pcregrep.1
dvander@2762
   367
    pcretest.1
dvander@2762
   368
    pcre.3
dvander@2762
   369
    pcre*.3 (lots more pages, all starting "pcre")
dvander@2762
   370
dvander@2762
   371
  HTML documentation (share/doc/pcre/html):
dvander@2762
   372
    index.html
dvander@2762
   373
    *.html (lots more pages, hyperlinked from index.html)
dvander@2762
   374
dvander@2762
   375
  Text file documentation (share/doc/pcre):
dvander@2762
   376
    AUTHORS
dvander@2762
   377
    COPYING
dvander@2762
   378
    ChangeLog
dvander@2762
   379
    LICENCE
dvander@2762
   380
    NEWS
dvander@2762
   381
    README
dvander@2762
   382
    pcre.txt       (a concatenation of the man(3) pages)
dvander@2762
   383
    pcretest.txt   the pcretest man page
dvander@2762
   384
    pcregrep.txt   the pcregrep man page
dvander@2762
   385
dvander@2762
   386
If you want to remove PCRE from your system, you can run "make uninstall".
dvander@2762
   387
This removes all the files that "make install" installed. However, it does not
dvander@2762
   388
remove any directories, because these are often shared with other programs.
dvander@2762
   389
dvander@2762
   390
dvander@2762
   391
Retrieving configuration information on Unix-like systems
dvander@2762
   392
---------------------------------------------------------
dvander@2762
   393
dvander@2762
   394
Running "make install" installs the command pcre-config, which can be used to
dvander@2762
   395
recall information about the PCRE configuration and installation. For example:
dvander@2762
   396
dvander@2762
   397
  pcre-config --version
dvander@2762
   398
dvander@2762
   399
prints the version number, and
dvander@2762
   400
dvander@2762
   401
  pcre-config --libs
dvander@2762
   402
dvander@2762
   403
outputs information about where the library is installed. This command can be
dvander@2762
   404
included in makefiles for programs that use PCRE, saving the programmer from
dvander@2762
   405
having to remember too many details.
dvander@2762
   406
dvander@2762
   407
The pkg-config command is another system for saving and retrieving information
dvander@2762
   408
about installed libraries. Instead of separate commands for each library, a
dvander@2762
   409
single command is used. For example:
dvander@2762
   410
dvander@2762
   411
  pkg-config --cflags pcre
dvander@2762
   412
dvander@2762
   413
The data is held in *.pc files that are installed in a directory called
dvander@2762
   414
<prefix>/lib/pkgconfig.
dvander@2762
   415
dvander@2762
   416
dvander@2762
   417
Shared libraries on Unix-like systems
dvander@2762
   418
-------------------------------------
dvander@2762
   419
dvander@2762
   420
The default distribution builds PCRE as shared libraries and static libraries,
dvander@2762
   421
as long as the operating system supports shared libraries. Shared library
dvander@2762
   422
support relies on the "libtool" script which is built as part of the
dvander@2762
   423
"configure" process.
dvander@2762
   424
dvander@2762
   425
The libtool script is used to compile and link both shared and static
dvander@2762
   426
libraries. They are placed in a subdirectory called .libs when they are newly
dvander@2762
   427
built. The programs pcretest and pcregrep are built to use these uninstalled
dvander@2762
   428
libraries (by means of wrapper scripts in the case of shared libraries). When
dvander@2762
   429
you use "make install" to install shared libraries, pcregrep and pcretest are
dvander@2762
   430
automatically re-built to use the newly installed shared libraries before being
dvander@2762
   431
installed themselves. However, the versions left in the build directory still
dvander@2762
   432
use the uninstalled libraries.
dvander@2762
   433
dvander@2762
   434
To build PCRE using static libraries only you must use --disable-shared when
dvander@2762
   435
configuring it. For example:
dvander@2762
   436
dvander@2762
   437
./configure --prefix=/usr/gnu --disable-shared
dvander@2762
   438
dvander@2762
   439
Then run "make" in the usual way. Similarly, you can use --disable-static to
dvander@2762
   440
build only shared libraries.
dvander@2762
   441
dvander@2762
   442
dvander@2762
   443
Cross-compiling on Unix-like systems
dvander@2762
   444
------------------------------------
dvander@2762
   445
dvander@2762
   446
You can specify CC and CFLAGS in the normal way to the "configure" command, in
dvander@2762
   447
order to cross-compile PCRE for some other host. However, you should NOT
dvander@2762
   448
specify --enable-rebuild-chartables, because if you do, the dftables.c source
dvander@2762
   449
file is compiled and run on the local host, in order to generate the inbuilt
dvander@2762
   450
character tables (the pcre_chartables.c file). This will probably not work,
dvander@2762
   451
because dftables.c needs to be compiled with the local compiler, not the cross
dvander@2762
   452
compiler.
dvander@2762
   453
dvander@2762
   454
When --enable-rebuild-chartables is not specified, pcre_chartables.c is created
dvander@2762
   455
by making a copy of pcre_chartables.c.dist, which is a default set of tables
dvander@2762
   456
that assumes ASCII code. Cross-compiling with the default tables should not be
dvander@2762
   457
a problem.
dvander@2762
   458
dvander@2762
   459
If you need to modify the character tables when cross-compiling, you should
dvander@2762
   460
move pcre_chartables.c.dist out of the way, then compile dftables.c by hand and
dvander@2762
   461
run it on the local host to make a new version of pcre_chartables.c.dist.
dvander@2762
   462
Then when you cross-compile PCRE this new version of the tables will be used.
dvander@2762
   463
dvander@2762
   464
dvander@2762
   465
Using HP's ANSI C++ compiler (aCC)
dvander@2762
   466
----------------------------------
dvander@2762
   467
dvander@2762
   468
Unless C++ support is disabled by specifying the "--disable-cpp" option of the
dvander@2762
   469
"configure" script, you must include the "-AA" option in the CXXFLAGS
dvander@2762
   470
environment variable in order for the C++ components to compile correctly.
dvander@2762
   471
dvander@2762
   472
Also, note that the aCC compiler on PA-RISC platforms may have a defect whereby
dvander@2762
   473
needed libraries fail to get included when specifying the "-AA" compiler
dvander@2762
   474
option. If you experience unresolved symbols when linking the C++ programs,
dvander@2762
   475
use the workaround of specifying the following environment variable prior to
dvander@2762
   476
running the "configure" script:
dvander@2762
   477
dvander@2762
   478
  CXXLDFLAGS="-lstd_v2 -lCsup_v2"
dvander@2762
   479
dvander@2762
   480
dvander@2762
   481
Making new tarballs
dvander@2762
   482
-------------------
dvander@2762
   483
dvander@2762
   484
The command "make dist" creates three PCRE tarballs, in tar.gz, tar.bz2, and
dvander@2762
   485
zip formats. The command "make distcheck" does the same, but then does a trial
dvander@2762
   486
build of the new distribution to ensure that it works.
dvander@2762
   487
dvander@2762
   488
If you have modified any of the man page sources in the doc directory, you
dvander@2762
   489
should first run the PrepareRelease script before making a distribution. This
dvander@2762
   490
script creates the .txt and HTML forms of the documentation from the man pages.
dvander@2762
   491
dvander@2762
   492
dvander@2762
   493
Testing PCRE
dvander@2762
   494
------------
dvander@2762
   495
dvander@2762
   496
To test the basic PCRE library on a Unix system, run the RunTest script that is
dvander@2762
   497
created by the configuring process. There is also a script called RunGrepTest
dvander@2762
   498
that tests the options of the pcregrep command. If the C++ wrapper library is
dvander@2762
   499
built, three test programs called pcrecpp_unittest, pcre_scanner_unittest, and
dvander@2762
   500
pcre_stringpiece_unittest are also built.
dvander@2762
   501
dvander@2762
   502
Both the scripts and all the program tests are run if you obey "make check" or
dvander@2762
   503
"make test". For other systems, see the instructions in NON-UNIX-USE.
dvander@2762
   504
dvander@2762
   505
The RunTest script runs the pcretest test program (which is documented in its
dvander@2762
   506
own man page) on each of the testinput files in the testdata directory in
dvander@2762
   507
turn, and compares the output with the contents of the corresponding testoutput
dvander@2762
   508
files. A file called testtry is used to hold the main output from pcretest
dvander@2762
   509
(testsavedregex is also used as a working file). To run pcretest on just one of
dvander@2762
   510
the test files, give its number as an argument to RunTest, for example:
dvander@2762
   511
dvander@2762
   512
  RunTest 2
dvander@2762
   513
dvander@2762
   514
The first test file can also be fed directly into the perltest.pl script to
dvander@2762
   515
check that Perl gives the same results. The only difference you should see is
dvander@2762
   516
in the first few lines, where the Perl version is given instead of the PCRE
dvander@2762
   517
version.
dvander@2762
   518
dvander@2762
   519
The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
dvander@2762
   520
pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
dvander@2762
   521
detection, and run-time flags that are specific to PCRE, as well as the POSIX
dvander@2762
   522
wrapper API. It also uses the debugging flags to check some of the internals of
dvander@2762
   523
pcre_compile().
dvander@2762
   524
dvander@2762
   525
If you build PCRE with a locale setting that is not the standard C locale, the
dvander@2762
   526
character tables may be different (see next paragraph). In some cases, this may
dvander@2762
   527
cause failures in the second set of tests. For example, in a locale where the
dvander@2762
   528
isprint() function yields TRUE for characters in the range 128-255, the use of
dvander@2762
   529
[:isascii:] inside a character class defines a different set of characters, and
dvander@2762
   530
this shows up in this test as a difference in the compiled code, which is being
dvander@2762
   531
listed for checking. Where the comparison test output contains [\x00-\x7f] the
dvander@2762
   532
test will contain [\x00-\xff], and similarly in some other cases. This is not a
dvander@2762
   533
bug in PCRE.
dvander@2762
   534
dvander@2762
   535
The third set of tests checks pcre_maketables(), the facility for building a
dvander@2762
   536
set of character tables for a specific locale and using them instead of the
dvander@2762
   537
default tables. The tests make use of the "fr_FR" (French) locale. Before
dvander@2762
   538
running the test, the script checks for the presence of this locale by running
dvander@2762
   539
the "locale" command. If that command fails, or if it doesn't include "fr_FR"
dvander@2762
   540
in the list of available locales, the third test cannot be run, and a comment
dvander@2762
   541
is output to say why. If running this test produces instances of the error
dvander@2762
   542
dvander@2762
   543
  ** Failed to set locale "fr_FR"
dvander@2762
   544
dvander@2762
   545
in the comparison output, it means that locale is not available on your system,
dvander@2762
   546
despite being listed by "locale". This does not mean that PCRE is broken.
dvander@2762
   547
dvander@2762
   548
[If you are trying to run this test on Windows, you may be able to get it to
dvander@2762
   549
work by changing "fr_FR" to "french" everywhere it occurs. Alternatively, use
dvander@2762
   550
RunTest.bat. The version of RunTest.bat included with PCRE 7.4 and above uses
dvander@2762
   551
Windows versions of test 2. More info on using RunTest.bat is included in the
dvander@2762
   552
document entitled NON-UNIX-USE.]
dvander@2762
   553
dvander@2762
   554
The fourth test checks the UTF-8 support. It is not run automatically unless
dvander@2762
   555
PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
dvander@2762
   556
running "configure". This file can be also fed directly to the perltest script,
dvander@2762
   557
provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
dvander@2762
   558
commented in the script, can be be used.)
dvander@2762
   559
dvander@2762
   560
The fifth test checks error handling with UTF-8 encoding, and internal UTF-8
dvander@2762
   561
features of PCRE that are not relevant to Perl.
dvander@2762
   562
dvander@2762
   563
The sixth test checks the support for Unicode character properties. It it not
dvander@2762
   564
run automatically unless PCRE is built with Unicode property support. To to
dvander@2762
   565
this you must set --enable-unicode-properties when running "configure".
dvander@2762
   566
dvander@2762
   567
The seventh, eighth, and ninth tests check the pcre_dfa_exec() alternative
dvander@2762
   568
matching function, in non-UTF-8 mode, UTF-8 mode, and UTF-8 mode with Unicode
dvander@2762
   569
property support, respectively. The eighth and ninth tests are not run
dvander@2762
   570
automatically unless PCRE is build with the relevant support.
dvander@2762
   571
dvander@2762
   572
dvander@2762
   573
Character tables
dvander@2762
   574
----------------
dvander@2762
   575
dvander@2762
   576
For speed, PCRE uses four tables for manipulating and identifying characters
dvander@2762
   577
whose code point values are less than 256. The final argument of the
dvander@2762
   578
pcre_compile() function is a pointer to a block of memory containing the
dvander@2762
   579
concatenated tables. A call to pcre_maketables() can be used to generate a set
dvander@2762
   580
of tables in the current locale. If the final argument for pcre_compile() is
dvander@2762
   581
passed as NULL, a set of default tables that is built into the binary is used.
dvander@2762
   582
dvander@2762
   583
The source file called pcre_chartables.c contains the default set of tables. By
dvander@2762
   584
default, this is created as a copy of pcre_chartables.c.dist, which contains
dvander@2762
   585
tables for ASCII coding. However, if --enable-rebuild-chartables is specified
dvander@2762
   586
for ./configure, a different version of pcre_chartables.c is built by the
dvander@2762
   587
program dftables (compiled from dftables.c), which uses the ANSI C character
dvander@2762
   588
handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to
dvander@2762
   589
build the table sources. This means that the default C locale which is set for
dvander@2762
   590
your system will control the contents of these default tables. You can change
dvander@2762
   591
the default tables by editing pcre_chartables.c and then re-building PCRE. If
dvander@2762
   592
you do this, you should take care to ensure that the file does not get
dvander@2762
   593
automatically re-generated. The best way to do this is to move
dvander@2762
   594
pcre_chartables.c.dist out of the way and replace it with your customized
dvander@2762
   595
tables.
dvander@2762
   596
dvander@2762
   597
When the dftables program is run as a result of --enable-rebuild-chartables,
dvander@2762
   598
it uses the default C locale that is set on your system. It does not pay
dvander@2762
   599
attention to the LC_xxx environment variables. In other words, it uses the
dvander@2762
   600
system's default locale rather than whatever the compiling user happens to have
dvander@2762
   601
set. If you really do want to build a source set of character tables in a
dvander@2762
   602
locale that is specified by the LC_xxx variables, you can run the dftables
dvander@2762
   603
program by hand with the -L option. For example:
dvander@2762
   604
dvander@2762
   605
  ./dftables -L pcre_chartables.c.special
dvander@2762
   606
dvander@2762
   607
The first two 256-byte tables provide lower casing and case flipping functions,
dvander@2762
   608
respectively. The next table consists of three 32-byte bit maps which identify
dvander@2762
   609
digits, "word" characters, and white space, respectively. These are used when
dvander@2762
   610
building 32-byte bit maps that represent character classes for code points less
dvander@2762
   611
than 256.
dvander@2762
   612
dvander@2762
   613
The final 256-byte table has bits indicating various character types, as
dvander@2762
   614
follows:
dvander@2762
   615
dvander@2762
   616
    1   white space character
dvander@2762
   617
    2   letter
dvander@2762
   618
    4   decimal digit
dvander@2762
   619
    8   hexadecimal digit
dvander@2762
   620
   16   alphanumeric or '_'
dvander@2762
   621
  128   regular expression metacharacter or binary zero
dvander@2762
   622
dvander@2762
   623
You should not alter the set of characters that contain the 128 bit, as that
dvander@2762
   624
will cause PCRE to malfunction.
dvander@2762
   625
dvander@2762
   626
dvander@2762
   627
File manifest
dvander@2762
   628
-------------
dvander@2762
   629
dvander@2762
   630
The distribution should contain the following files:
dvander@2762
   631
dvander@2762
   632
(A) Source files of the PCRE library functions and their headers:
dvander@2762
   633
dvander@2762
   634
  dftables.c              auxiliary program for building pcre_chartables.c
dvander@2762
   635
                            when --enable-rebuild-chartables is specified
dvander@2762
   636
dvander@2762
   637
  pcre_chartables.c.dist  a default set of character tables that assume ASCII
dvander@2762
   638
                            coding; used, unless --enable-rebuild-chartables is
dvander@2762
   639
                            specified, by copying to pcre_chartables.c
dvander@2762
   640
dvander@2762
   641
  pcreposix.c             )
dvander@2762
   642
  pcre_compile.c          )
dvander@2762
   643
  pcre_config.c           )
dvander@2762
   644
  pcre_dfa_exec.c         )
dvander@2762
   645
  pcre_exec.c             )
dvander@2762
   646
  pcre_fullinfo.c         )
dvander@2762
   647
  pcre_get.c              ) sources for the functions in the library,
dvander@2762
   648
  pcre_globals.c          )   and some internal functions that they use
dvander@2762
   649
  pcre_info.c             )
dvander@2762
   650
  pcre_maketables.c       )
dvander@2762
   651
  pcre_newline.c          )
dvander@2762
   652
  pcre_ord2utf8.c         )
dvander@2762
   653
  pcre_refcount.c         )
dvander@2762
   654
  pcre_study.c            )
dvander@2762
   655
  pcre_tables.c           )
dvander@2762
   656
  pcre_try_flipped.c      )
dvander@2762
   657
  pcre_ucd.c              )
dvander@2762
   658
  pcre_valid_utf8.c       )
dvander@2762
   659
  pcre_version.c          )
dvander@2762
   660
  pcre_xclass.c           )
dvander@2762
   661
  pcre_printint.src       ) debugging function that is #included in pcretest,
dvander@2762
   662
                          )   and can also be #included in pcre_compile()
dvander@2762
   663
  pcre.h.in               template for pcre.h when built by "configure"
dvander@2762
   664
  pcreposix.h             header for the external POSIX wrapper API
dvander@2762
   665
  pcre_internal.h         header for internal use
dvander@2762
   666
  ucp.h                   header for Unicode property handling
dvander@2762
   667
dvander@2762
   668
  config.h.in             template for config.h, which is built by "configure"
dvander@2762
   669
dvander@2762
   670
  pcrecpp.h               public header file for the C++ wrapper
dvander@2762
   671
  pcrecpparg.h.in         template for another C++ header file
dvander@2762
   672
  pcre_scanner.h          public header file for C++ scanner functions
dvander@2762
   673
  pcrecpp.cc              )
dvander@2762
   674
  pcre_scanner.cc         ) source for the C++ wrapper library
dvander@2762
   675
dvander@2762
   676
  pcre_stringpiece.h.in   template for pcre_stringpiece.h, the header for the
dvander@2762
   677
                            C++ stringpiece functions
dvander@2762
   678
  pcre_stringpiece.cc     source for the C++ stringpiece functions
dvander@2762
   679
dvander@2762
   680
(B) Source files for programs that use PCRE:
dvander@2762
   681
dvander@2762
   682
  pcredemo.c              simple demonstration of coding calls to PCRE
dvander@2762
   683
  pcregrep.c              source of a grep utility that uses PCRE
dvander@2762
   684
  pcretest.c              comprehensive test program
dvander@2762
   685
dvander@2762
   686
(C) Auxiliary files:
dvander@2762
   687
dvander@2762
   688
  132html                 script to turn "man" pages into HTML
dvander@2762
   689
  AUTHORS                 information about the author of PCRE
dvander@2762
   690
  ChangeLog               log of changes to the code
dvander@2762
   691
  CleanTxt                script to clean nroff output for txt man pages
dvander@2762
   692
  Detrail                 script to remove trailing spaces
dvander@2762
   693
  HACKING                 some notes about the internals of PCRE
dvander@2762
   694
  INSTALL                 generic installation instructions
dvander@2762
   695
  LICENCE                 conditions for the use of PCRE
dvander@2762
   696
  COPYING                 the same, using GNU's standard name
dvander@2762
   697
  Makefile.in             ) template for Unix Makefile, which is built by
dvander@2762
   698
                          )   "configure"
dvander@2762
   699
  Makefile.am             ) the automake input that was used to create
dvander@2762
   700
                          )   Makefile.in
dvander@2762
   701
  NEWS                    important changes in this release
dvander@2762
   702
  NON-UNIX-USE            notes on building PCRE on non-Unix systems
dvander@2762
   703
  PrepareRelease          script to make preparations for "make dist"
dvander@2762
   704
  README                  this file
dvander@2762
   705
  RunTest                 a Unix shell script for running tests
dvander@2762
   706
  RunGrepTest             a Unix shell script for pcregrep tests
dvander@2762
   707
  aclocal.m4              m4 macros (generated by "aclocal")
dvander@2762
   708
  config.guess            ) files used by libtool,
dvander@2762
   709
  config.sub              )   used only when building a shared library
dvander@2762
   710
  configure               a configuring shell script (built by autoconf)
dvander@2762
   711
  configure.ac            ) the autoconf input that was used to build
dvander@2762
   712
                          )   "configure" and config.h
dvander@2762
   713
  depcomp                 ) script to find program dependencies, generated by
dvander@2762
   714
                          )   automake
dvander@2762
   715
  doc/*.3                 man page sources for the PCRE functions
dvander@2762
   716
  doc/*.1                 man page sources for pcregrep and pcretest
dvander@2762
   717
  doc/index.html.src      the base HTML page
dvander@2762
   718
  doc/html/*              HTML documentation
dvander@2762
   719
  doc/pcre.txt            plain text version of the man pages
dvander@2762
   720
  doc/pcretest.txt        plain text documentation of test program
dvander@2762
   721
  doc/perltest.txt        plain text documentation of Perl test program
dvander@2762
   722
  install-sh              a shell script for installing files
dvander@2762
   723
  libpcre.pc.in           template for libpcre.pc for pkg-config
dvander@2762
   724
  libpcrecpp.pc.in        template for libpcrecpp.pc for pkg-config
dvander@2762
   725
  ltmain.sh               file used to build a libtool script
dvander@2762
   726
  missing                 ) common stub for a few missing GNU programs while
dvander@2762
   727
                          )   installing, generated by automake
dvander@2762
   728
  mkinstalldirs           script for making install directories
dvander@2762
   729
  perltest.pl             Perl test program
dvander@2762
   730
  pcre-config.in          source of script which retains PCRE information
dvander@2762
   731
  pcrecpp_unittest.cc          )
dvander@2762
   732
  pcre_scanner_unittest.cc     ) test programs for the C++ wrapper
dvander@2762
   733
  pcre_stringpiece_unittest.cc )
dvander@2762
   734
  testdata/testinput*     test data for main library tests
dvander@2762
   735
  testdata/testoutput*    expected test results
dvander@2762
   736
  testdata/grep*          input and output for pcregrep tests
dvander@2762
   737
dvander@2762
   738
(D) Auxiliary files for cmake support
dvander@2762
   739
dvander@2762
   740
  cmake/COPYING-CMAKE-SCRIPTS
dvander@2762
   741
  cmake/FindPackageHandleStandardArgs.cmake
dvander@2762
   742
  cmake/FindReadline.cmake
dvander@2762
   743
  CMakeLists.txt
dvander@2762
   744
  config-cmake.h.in
dvander@2762
   745
dvander@2762
   746
(E) Auxiliary files for VPASCAL
dvander@2762
   747
dvander@2762
   748
  makevp.bat
dvander@2762
   749
  makevp_c.txt
dvander@2762
   750
  makevp_l.txt
dvander@2762
   751
  pcregexp.pas
dvander@2762
   752
dvander@2762
   753
(F) Auxiliary files for building PCRE "by hand"
dvander@2762
   754
dvander@2762
   755
  pcre.h.generic          ) a version of the public PCRE header file
dvander@2762
   756
                          )   for use in non-"configure" environments
dvander@2762
   757
  config.h.generic        ) a version of config.h for use in non-"configure"
dvander@2762
   758
                          )   environments
dvander@2762
   759
dvander@2762
   760
(F) Miscellaneous
dvander@2762
   761
dvander@2762
   762
  RunTest.bat            a script for running tests under Windows
dvander@2762
   763
dvander@2762
   764
Philip Hazel
dvander@2762
   765
Email local part: ph10
dvander@2762
   766
Email domain: cam.ac.uk
dvander@2762
   767
Last updated: 21 March 2009