Add data filters to gfix validation: -skip_data, -include_data, -skip_schema_data, -include_schema_data

[ChudaykinAlex]

Motivation

gfix -v -full and gfix -mend may fail to complete on a database with damaged
records in unimportant tables, whose data could be restored from other sources.
This PR allows to exclude records of such tables from validation, so the rest of
the database can still be checked and repaired.

The switches are similar to the same gbak switches.

Changes

gfix

Switch	Description
-SKIP_D(ATA)	skip records validation of tables with matching names
-INCLUDE(_DATA)	validate records of tables with matching names only
-SKIP_SCHEMA_D(ATA)	skip records validation of tables in schemas with matching names
-INCLUDE_SCHEMA_D(ATA)	validate records of tables in schemas with matching names only

• A switch accepts a regular expression in SIMILAR TO syntax, matching is case insensitive.
• The switches require -validate -full (or -mend), otherwise an error is reported.
  The requirement is checked after parsing of all switches, so the order of switches doesn't
  matter, including the order of SPB items in services API.
• The pattern is checked when the switch is parsed, so an invalid regular expression is
  reported before attaching the database. Patterns longer than 255 bytes are supported:
  gfix builds DPB with ClumpletReader::dpbList, which allows upgrade to DPB version 2.

gfix -v -full -skip_data T1 database.fdb
gfix -mend -skip_data "LOG\_%" database.fdb
gfix -v -full -include_schema_data S1 -include_data T1 database.fdb

Rules

Records of a table are skipped by the same rules as in BurpGlobals::skipRelation():
the table is filtered out either by the table name switches or by the schema name switches.
If a name matches both skip and include patterns, records are skipped.
The filters are applied to user tables only, records of system tables are always validated.

Engine

• Records of a filtered relation are not read at all, including big (fragmented) records and
  blobs, so -mend doesn't change them: VDR_records is cleared for the relation in
  Validation::walk_relation() (and restored after it), and its records are not passed to
  walk_record() / walk_blob().
• Pointer, data and index pages of the relation are still walked and checked as without -full:
  the checks of full validation (record version chains, index entries against records,
  consistency of index tree levels) are not performed for it.
• Orphan pages detection is disabled when records of some relation were actually skipped, as
  pages of its big records and blobs were not walked. A filter that skips nothing doesn't
  disable it.

API

• DPB (used together with isc_dpb_verify): isc_dpb_verify_skip_data (108),
  isc_dpb_verify_include_data (109), isc_dpb_verify_skip_schema_data (110),
  isc_dpb_verify_include_schema_data (111).
• SPB of isc_action_svc_repair: isc_spb_rpr_skip_data (53), isc_spb_rpr_include_data (54),
  isc_spb_rpr_skip_schema_data (55), isc_spb_rpr_include_schema_data (56).
  The items are string items, they are converted to UTF-8 like other string items.
• fbsvcmgr: rpr_skip_data, rpr_include_data, rpr_skip_schema_data, rpr_include_schema_data.

fbsvcmgr service_mgr action_repair dbname database.fdb rpr_validate_db rpr_full rpr_skip_data T1

Build

gfix now uses SimilarToRegex, so it's linked with re2 (builds/posix/Makefile.in,
builds/win32/msvc15/gfix.vcxproj), like gbak.

Documentation

doc/README.gfix_data_filters.md