Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ collections:
openvoxdb_8x:
output: true
permalink: '/openvoxdb/8.x/:path:output_ext'
openvoxdb_9x:
output: true
permalink: '/openvoxdb/9.x/:path:output_ext'

openbolt_latest:
output: true
Expand Down Expand Up @@ -165,6 +168,11 @@ defaults:
type: openvoxdb_8x
values:
nav: openvoxdb_8x
- scope:
path: ''
type: openvoxdb_9x
values:
nav: openvoxdb_9x

- scope:
path: ''
Expand Down
173 changes: 173 additions & 0 deletions _data/nav/openvoxdb_9x.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
---
- text: General information
items:
- text: Release notes
link: release_notes.html
- text: Overview and requirements
link: index.html
- text: Frequently asked questions
link: puppetdb-faq.html
- text: Versioning policy
link: versioning_policy.html
- text: Known issues
link: known_issues.html
- text: Community add-ons
link: community_add_ons.html
- text: Installation
items:
- text: Installing via Puppet module
link: install_via_module.html
- text: Installing from packages
link: install_from_packages.html
- text: Installing from source
link: install_from_source.html
- text: Upgrading OpenVoxDB
link: upgrade.html
- text: Connecting Puppet Servers
link: connect_puppet_server.html
- text: Connecting standalone Puppet nodes
link: connect_puppet_apply.html
- text: Configuration
items:
- text: Configuring OpenVoxDB
link: configure.html
- text: 'puppetdb.conf: Configuring a Puppet/OpenVoxDB connection'
link: puppetdb_connection.html
- text: Configuring Postgres
link: configure_postgres.html
- text: Setting up SSL for PostgreSQL
link: postgres_ssl.html
- text: Coordinating database migrations
link: migration_coordination.html
- text: Usage/admin
items:
- text: Using OpenVoxDB
link: using.html
- text: Maintaining and tuning
link: maintain_and_tune.html
- text: OpenVoxDB CLI
link: pdb_client_tools.html
- text: Exporting and anonymizing data
link: anonymization.html
- text: Scaling recommendations
link: scaling_recommendations.html
- text: Logging configuration
link: logging.html
- text: Debugging with remote REPL
link: repl.html
- text: Load testing
link: load_testing_tool.html
- text: Troubleshooting
items:
- text: General Support Guide
link: pdb_support_guide.html
- text: Session logging
link: trouble_session_logging.html
- text: PQL - Puppet Query Language
items:
- text: Tutorial
link: api/query/tutorial-pql.html
- text: Reference guide
link: api/query/v4/pql.html
- text: Examples
link: api/query/examples-pql.html
- text: API
items:
- text: Overview
link: api/overview.html
- text: Query tutorial
link: api/query/tutorial.html
- text: Curl tips
link: api/query/curl.html
- text: Query API version 4
items:
- text: Upgrading from version 3
link: api/query/v4/upgrading-from-v3.html
- text: Query structure
link: api/query/v4/query.html
- text: Entities
link: api/query/v4/entities.html
- text: AST query language
link: api/query/v4/ast.html
- text: Query paging
link: api/query/v4/paging.html
- text: Root endpoint
link: api/query/v4/overview.html
- text: Nodes endpoint
link: api/query/v4/nodes.html
- text: Environments endpoint
link: api/query/v4/environments.html
- text: Producers endpoint
link: api/query/v4/producers.html
- text: Factsets endpoint
link: api/query/v4/factsets.html
- text: Facts endpoint
link: api/query/v4/facts.html
- text: Fact-names endpoint
link: api/query/v4/fact-names.html
- text: Fact-paths endpoint
link: api/query/v4/fact-paths.html
- text: Fact-contents endpoint
link: api/query/v4/fact-contents.html
- text: Inventory endpoint
link: api/query/v4/inventory.html
- text: Catalogs endpoint
link: api/query/v4/catalogs.html
- text: Catalog inputs endpoint
link: api/query/v4/catalog_inputs.html
- text: Catalog input contents endpoint
link: api/query/v4/catalog_input_contents.html
- text: Resources endpoint
link: api/query/v4/resources.html
- text: Edges endpoint
link: api/query/v4/edges.html
- text: Reports endpoint
link: api/query/v4/reports.html
- text: Events endpoint
link: api/query/v4/events.html
- text: Event counts endpoint
link: api/query/v4/event-counts.html
- text: Aggregate event counts endpoint
link: api/query/v4/aggregate-event-counts.html
- text: Package endpoints
link: api/query/v4/packages.html
- text: Admin API version 1
items:
- text: Archive endpoint
link: api/admin/v1/archive.html
- text: Command (cmd) endpoint
link: api/admin/v1/cmd.html
- text: Summary stats endpoint
link: api/admin/v1/summary-stats.html
- text: Command API version 1
items:
- text: Commands endpoint
link: api/command/v1/commands.html
- text: Status API version 1
items:
- text: Status endpoint
link: api/status/v1/status.html
- text: Metadata API version 1
items:
- text: Version endpoint
link: api/meta/v1/version.html
- text: Server time endpoint
link: api/meta/v1/server-time.html
- text: Metrics API
items:
- text: Metrics API v2
link: api/metrics/v2/jolokia.html
- text: Wire formats
items:
- text: Catalog wire format - v9
link: api/wire_format/catalog_format_v9.html
- text: Facts wire format - v5
link: api/wire_format/facts_format_v5.html
- text: Report wire format - v8
link: api/wire_format/report_format_v8.html
- text: Deactivate node wire format - v3
link: api/wire_format/deactivate_node_format_v3.html
- text: Replace catalog inputs wire format - v1 (experimental)
link: api/wire_format/catalog_inputs_format_v1.html
- text: Configure expiration wire format - v1 (experimental)
link: api/wire_format/configure_expiration_format_v1.html
4 changes: 4 additions & 0 deletions _data/nav_map.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@
collections: openvoxdb_latest|openvoxdb_8x
base: /openvoxdb/latest/

- nav_key: openvoxdb_9x
collections: openvoxdb_9x
base: /openvoxdb/9.x/

- nav_key: openbolt_5x
collections: openbolt_latest|openbolt_5x
base: /openbolt/latest/
Expand Down
2 changes: 1 addition & 1 deletion _data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
collections: [openvox-server_latest, openvox-server_8x, openvox-server_9x]
- title: OpenVoxDB
url: /openvoxdb/latest/
collections: [openvoxdb_latest, openvoxdb_8x]
collections: [openvoxdb_latest, openvoxdb_8x, openvoxdb_9x]
- title: OpenBolt
url: /openbolt/latest/
collections: [openbolt_latest, openbolt_5x]
Expand Down
4 changes: 4 additions & 0 deletions _data/products.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,10 @@ openvoxdb:
label: OpenVoxDB
latest: 8x
versions:
- id: 9x
label: "9.x"
collection: _openvoxdb_9x
base: /openvoxdb/9.x/
- id: 8x
label: "8.x"
collection: _openvoxdb_8x
Expand Down
8 changes: 8 additions & 0 deletions docs/_openvoxdb_9x/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# OpenVoxDB documentation

If you were redirected while trying to reach the OpenVoxDB docs, the version you were trying to reach is no longer maintained.

For the most recent OpenVoxDB docs, see [docs.openvoxproject.org/openvoxdb/latest](https://docs.openvoxproject.org/openvoxdb/latest).

If you need docs for an older version of OpenVoxDB, you may be able to find it by switching to the relevant branch of this repository.

125 changes: 125 additions & 0 deletions docs/_openvoxdb_9x/anonymization.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
title: "Export, import and anonymization"
layout: default
---
[pdb_client_tools]: ./pdb_client_tools.html

# Exporting and anonymizing data

This document covers using the export, import and anonymization tools for
OpenVoxDB.

The export tool will return an archive of all of your OpenVoxDB data which can be
uploaded to another OpenVoxDB via the import tool. The export tool also has the
ability to anonymize the archive before returning it. This is particularly
useful when sharing OpenVoxDB data that contains sensitive items.

## Using the `export` command

To create an anonymized OpenVoxDB archive directly, use the `puppet db` command
from any node with the [OpenVoxDB CLI][pdb_client_tools] installed:

```console
puppet db export my-openvoxdb-export.tar.gz --anonymization moderate
```

## Using the `import` command

To import an anonymized OpenVoxDB tarball, use the `puppet db` command from
any node with the [OpenVoxDB CLI][pdb_client_tools] installed:

```console
puppet db import my-openvoxdb-export.tar.gz
```

## How does it work?

The tool walks through your entire data set, applying different rules to each of
the leaf data based on the profile you have chosen. The data structure is left
intact, and only the data contents are modified. This maintains the "shape" of
the data without exposing the underlying data you may wish to scrub.

We do this by always ensuring we replace data consistently. For example, if a
string is replaced with something random, we ensure that all instances of that
original string are replaced with the same random string throughout the data.

By keeping its original shape, the data can be anonymized based on your needs
but still hold some value to the consumer of your anonymized data.

## Anonymization profiles

You may not need to anonymize all data in every case, so we have provided a
number of profiles offering varying levels of anonymization.

The profile can be specified on the command line when the command is run. For
example, to choose the `low` profile, enter:

```console
puppet db export ./my-openvoxdb-anonymized-export.tar.gz --anonymization low
```

### Profile: full

The `full` profile will anonymize all data (including node names, resource
types, resource titles, parameter names, values, any log messages, file names,
and file lines) while retaining the data set's shape. The result is a completely
anonymized data set. Report metrics under the `resources` and `events`
categories are left intact, as these can be inferred from the rest of the data,
but names of metrics under the `time` category are anonymized as resource types.

This is useful if you are really concerned about limiting the data you expose,
but provides the least utility for the consumer depending on the activity they
are trying to test.

### Profile: moderate

The `moderate` profile attempts to be a bit smarter about what it anonymizes and
is **the recommended profile for most cases**. It sorts and anonymizes data by
data type:

* Node name: is anonymized by default.
* Resource type name: the core types that are built into Puppet are not
anonymized, including some common types from the modules: `stdlib`,
`postgresql`, `rabbitmq`, `puppetdb`, `apache`, `mrep`, `f5`, `apt`,
`registry`, `concat`, and `mysql`. Any Puppet Enterprise core type names are
also preserved. The goal here is to anonymize any custom or unknown resource
type names, as these may contain confidential information.
* Resource titles: all titles are anonymized expect for those belonging to
Filebucket, Package, Service, and Stage.
* Parameter names: are never anonymized.
* Parameter values: everything is anonymized except for the values for
`provider`, `ensure`, `noop`, `loglevel`, `audit`, and `schedule`.
* Report log messages: are always anonymized.
* File names: are always anonymized.
* File numbers: are left as they are.
* Log messages: are always anonymized.
* Metrics: metric names in the `time` category are anonymized as resource types.

### Profile: low

This profile is aimed at hiding security information specifically, but leaving
most of the data in its original state. The following categories are anonymized:

* Node name: is always anonymized.
* Parameter values: only values and messages for parameter names containing the
strings `password`, `pwd`, `secret`, `key`, or `private` are anonymized.
* Log messages: are always anonymized.

## Verifying your anonymized data

After anonymizing data with the `puppet db export` tool, we **strongly
recommend** that you analyze the anonymized data before sharing it with another
party to ensure that all sensitive data has been scrubbed.

Simply untar the export file and analyze the contents:

```console
tar -xzf my-openvoxdb-anonymized-export.tar.gz
cd puppetdb-bak
```

Inside this directory there is a directory for each content type (reports,
catalogs, and facts), and each file inside represents a node (and a report
instance for reports). The data is represented as human-readable JSON. You can
open these files and use tools such as `grep` to check the status of specific
information you wish to anonymize.
Loading