diff --git a/docs/command_reference.md b/docs/command_reference.md new file mode 100644 index 000000000..3b139dc5b --- /dev/null +++ b/docs/command_reference.md @@ -0,0 +1,561 @@ + + +# Command reference + +What each mapillary_tools command does, how to use it, and what to watch out for. +For the full list of options of the version you have installed, run `mapillary_tools COMMAND --help`. + +- [Overview](#overview) +- [Common concepts](#common-concepts) + - [Import paths](#import-paths) + - [The description file](#the-description-file) + - [Sequences and checks](#sequences-and-checks) +- [process_and_upload](#process_and_upload) +- [process](#process) +- [Geotag sources](#geotag-sources) + - [--geotag_source](#--geotag_source) + - [--video_geotag_source](#--video_geotag_source) + - [GPX and NMEA tracks](#gpx-and-nmea-tracks) + - [ExifTool](#exiftool) +- [upload](#upload) +- [sample_video](#sample_video) +- [video_process](#video_process) +- [video_process_and_upload](#video_process_and_upload) +- [authenticate](#authenticate) +- [check_upload_history](#check_upload_history) +- [zip](#zip) +- [Exit codes](#exit-codes) +- [Environment variables](#environment-variables) + +## Overview + +| Command | What it does | +| -------------------------- | -------------------------------------------------------------------------------------------------- | +| `process_and_upload` | `process`, then `upload`. The command most users need. | +| `process` | Reads the time and GPS position of images and videos, checks them, and writes the description file. | +| `upload` | Uploads the images and videos that `process` described. | +| `sample_video` | Extracts frames from videos as geotagged images. Needs FFmpeg. | +| `video_process` | `sample_video`, then `process` the frames. | +| `video_process_and_upload` | `video_process`, then `upload` the frames. | +| `authenticate` | Adds, updates or deletes user profiles, and deletes saved organization names. | +| `check_upload_history` | Tells whether processed files were already uploaded from this computer. | +| `zip` | Writes processed image sequences into zip files, to upload later. | + +Two options go before the command name: `--version`, and `--verbose` for debug logs, as in +`mapillary_tools --verbose process MY_CAPTURE_DIR`. After the command name, `--verbose` is rejected. + +## Common concepts + +### Import paths + +The commands take one or more files or directories. + +- Directories are searched recursively. `--skip_subfolders` reads only the top level. Hidden files and folders + (names starting with `.`) are skipped. +- In a directory, files that are not supported images or videos are left out without a word. +- A file named on the command line that is not a supported image or video fails with + `MapillaryUnsupportedFileError`. For an Insta360 `.insv` original, the error explains how to stitch it first. + Sidecar and companion files (`.gpx`, `.nmea`, `.xml`, `.json`, `.thm`, `.lrv`, `.srt`, `.txt`, `.DS_Store` and + the like) are skipped with a warning instead, so that `process DCIM/100GOPRO/*` works. Zip files are + uploaded as they are. +- Images are JPEG files (`.jpg`, `.jpeg`, `.jpe`; `.tif`, `.tiff`, `.pgm`, `.pnm` and `.36p` are processed as + images too) with a capture time and a GPS position (or a GPS track to place them on). Videos need GPS + of their own, or a GPX or NMEA track, or GPS that ExifTool can read. See [Geotag sources](#geotag-sources). + +### The description file + +`process` writes a JSON array with one entry per file: its capture time, position, direction and device, or an +`error` entry that says why it failed. `upload` reads it. The format is defined by the +[image description schema](../schema/image_description_schema.json); you can edit or generate the file yourself. + +- Where `process` writes it: `IMPORT_PATH/mapillary_image_description.json` when the only import path is a + directory. With several import paths, or a single file, it goes to STDOUT, with a warning. `--desc_path PATH` + sets the path, and `-` means STDOUT. +- `upload` reads it from the same default place, and needs `--desc_path` otherwise (`-` reads STDIN). +- File names in it are absolute. `upload` uploads the entries without an error whose files are under the import + paths it is given, so do not move the files between `process` and `upload`. +- `process` writes it before it fails, so a later `upload` uploads the files that did not fail. +- `upload` checks every entry without an `error` against the schema, and stops at one that does not match (exit + code 15). Entries with an `error` are not checked and not uploaded, but keep their `filetype`: `upload` + crashes on an error entry without one. + +### Sequences and checks + +`process` groups images into sequences: by folder, camera (make, model and serial number) and image size, sorted +by capture time. Images in different folders never share a sequence. A sequence is split where: + +- the time between two images exceeds `--cutoff_time` (60 s), +- the distance between two images exceeds `--cutoff_distance` (100 m), +- it reaches 1000 images, 110 GB or 6 gigapixels. + +Then each file is checked. Files that fail a check get an error entry and are not uploaded: + +| Check | Applies to | Error | +| --------------------------------------- | --------------- | ------------------------------------------------------------------------ | +| Same position and direction as the image before it (`--duplicate_distance` 0.1 m, `--duplicate_angle` 5°) | images | `MapillaryDuplicationError`, which does not fail the command | +| Position (0, 0) | images, videos | `MapillaryNullIslandError` | +| Average speed above 400 km/h | sequences, videos | `MapillaryCaptureSpeedTooFastError` | +| GPS positions that jump back and forth | sequences | `MapillaryZigZagError` (turn off with `--skip_zigzag_check`) | +| All GPS points within 10 m | videos | `MapillaryStationaryVideoError` | +| No GPS points | videos | `MapillaryVideoGPSNotFoundError` | +| GPS points too far apart in time for the uploaded video to store | videos | `MapillaryGPSPointsTooFarApartError`; a GPS timestamp is probably wrong | +| Larger than 110 GB | videos | `MapillaryFileTooLargeError` | + +## process_and_upload + +```sh +mapillary_tools process_and_upload IMPORT_PATH [IMPORT_PATH ...] [OPTIONS] +``` + +Runs [`process`](#process) and then [`upload`](#upload), and takes the options of both. + +```sh +mapillary_tools process_and_upload MY_CAPTURE_DIR MY_VIDEO_DIR/*.mp4 \ + --user_name my_profile \ + --organization_name work +``` + +Gotchas: + +- It logs in first, before processing (see [`upload`](#upload) for profiles and prompts). +- If any file fails to process, nothing is uploaded and the command exits with code 6. With + `--skip_process_errors`, the files that did not fail are uploaded. +- It writes no description file, unless you give `--desc_path`. +- Zip files in the import paths (from the [`zip`](#zip) command) are uploaded as they are. + +## process + +```sh +mapillary_tools process IMPORT_PATH [IMPORT_PATH ...] [OPTIONS] +``` + +Reads the capture time and GPS position of each image and video from the [geotag sources](#geotag-sources), +groups the images into sequences, runs the [checks](#sequences-and-checks), and writes the +[description file](#the-description-file). It uploads nothing, and it does not change your files unless you +give an `--overwrite_*` option. + +| Option | Default | Description | +| -------------------------------------- | ----------------------------- | --------------------------------------------------------------------------------------------------- | +| `--skip_subfolders` | | Read only the top level of each directory. | +| `--filetypes TYPES` | all | Comma-separated: `image`, `video`, or one type of video (`gopro`, `camm`, `blackvue`). Also `--file_types`. | +| `--skip_process_errors` | | Exit 0 when some files fail. They still get an error entry and are not uploaded. | +| `--desc_path PATH` | see above | Where to write the description file. `-` for STDOUT. | +| `--process_report_path PATH` | | Write a JSON report of the files found and the unsupported files skipped. | +| `--geotag_source SOURCE` | `native`, `exiftool_runtime` | Where to read times and positions from. Repeatable. See [Geotag sources](#geotag-sources). | +| `--geotag_source_path PATH` | | The track or XML file for the sources that read one. | +| `--video_geotag_source SOURCE` | | Geotag sources for videos only. Repeatable. | +| `--interpolation_offset_time SECONDS` | 0 | Seconds added to capture times to find positions on a GPX, NMEA or video track. | +| `--interpolation_use_gpx_start_time` | | Place the first image, or the start of a video, at the first point of the track. | +| `--offset_time SECONDS` | 0 | Seconds added to image capture times after geotagging. Does not move images. | +| `--offset_angle DEGREES` | 0 | Degrees added to image directions. | +| `--interpolate_directions` | | Point each image at the next one, instead of using the direction from EXIF. | +| `--cutoff_time`, `--cutoff_distance` | 60 s, 100 m | Where to split sequences. | +| `--duplicate_distance`, `--duplicate_angle` | 0.1 m, 5° | When an image counts as a duplicate of the one before it. `--duplicate_angle 360` ignores direction. | +| `--skip_zigzag_check` | | Do not fail sequences whose GPS zig-zags. | +| `--device_make`, `--device_model` | | Override the camera make and model. | +| `--overwrite_all_EXIF_tags` | | Write the processed time, position, direction and orientation into the EXIF of the images. `--overwrite_EXIF_time_tag`, `--overwrite_EXIF_gps_tag`, `--overwrite_EXIF_direction_tag` and `--overwrite_EXIF_orientation_tag` write one each. | +| `--num_processes N` | number of CPUs | Worker processes. 0 or less turns multiprocessing off. | + +```sh +# Images and videos with their own GPS +mapillary_tools process MY_CAPTURE_DIR + +# Images geotagged from a GPX track, with the camera clock 8 hours ahead of the GPS +mapillary_tools process MY_IMAGE_DIR \ + --geotag_source gpx --geotag_source_path track.gpx \ + --interpolation_offset_time -28800 + +# Write a report of what was found and skipped +mapillary_tools process MY_CAPTURE_DIR --process_report_path report.json +``` + +The process report (version 1) has `discovered_file_count`, `processed_file_count` (the files with a supported +extension, counted before processing: including those that fail, and those that `--filetypes` leaves out), +`skipped_file_count`, `skipped_files` with the name, category, reason code and extension of each unsupported file, +and `warnings`: one entry for each warning about a file that was still geotagged, with its `filename`, `type`, +`message` and `vars` (such as the durations of a `MapillaryGPXAlignmentWarning`). An unsupported file named on the +command line is always reported. Only `process` and `process_and_upload` list warnings. The report is written before +processing starts, and again with the warnings once geotagging has run. It holds absolute file names, so treat it +as private. + +Gotchas: + +- The command exits with code 6 if any file fails, after writing the description file. +- A directory with both images and videos processes both. Use `--filetypes image` or `--filetypes video` to + pick one. +- Processing a video folder after [`video_process`](#video_process) also finds the frames it wrote into + `mapillary_sampled_video_frames`, so `process_and_upload` would upload both the videos and their frames. Pass + the videos themselves, or move the frames away. +- `--offset_time` shifts the capture times after geotagging, and leaves positions alone. + `--interpolation_offset_time` shifts them before they are placed on a track, so it moves the images too. Both + change the capture time that is uploaded: to fix a camera clock against a GPX, use + `--interpolation_offset_time` alone. +- The `--overwrite_*` options modify your original images. +- Images whose only date is a placeholder, or before 1970, fail. `native` also rejects EXIF GPS minutes or + seconds above 60, but with ExifTool installed, the default `exiftool_runtime` fallback then geotags such an + image at a wrong position, without a warning. Check images that `native` rejects. + +## Geotag sources + +### --geotag_source + +Sources are tried in order. A file that a source finds no GPS for, places outside its track, or cannot read +because ExifTool or an XML file is missing, goes to the next source. If none geotags it, it gets the error of the +last one that tried, with two exceptions: if a GPX or NMEA track missed the file, that miss is reported +(`MapillaryOutsideGPXTrackError`, which says by how much); and the rejection of a video's own GPS is kept over a +later "GPX file not found". Other errors end the search: +a track file that is not valid, or has no time that can be read, fails the files it was to geotag, even if a +later source could have. A video whose own GPS is rejected as empty or noisy goes on only to `gpx` and `nmea` sources. +Repeat the option or separate sources with commas. The default is `native`, then `exiftool_runtime`. + +| Source | Images | Videos | +| ------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------ | +| `native` | EXIF | The GPS of the video: GoPro (GPMF), CAMM, BlackVue, and Novatek-based dashcams | +| `exif` | EXIF | Cannot read videos | +| `exiftool_runtime` (`exiftool`) | Runs ExifTool on the image | Runs ExifTool on the video | +| `exiftool_xml` | Reads the output of `exiftool -X`, by default `%g.xml` beside the file | The same | +| `gpx` | Places images on the GPX track by capture time (needs `--geotag_source_path`) | Places the video on the GPX track (by default `%g.gpx` beside it) | +| `nmea` | The same, from an NMEA log | The same, by default `%g.nmea` | +| `gopro`, `camm`, `blackvue` | Places frames that `sample_video` took from a video on the GPS track of that video, whatever its type (needs `--geotag_source_path` to the video or its folder) | Cannot read videos | + +`--geotag_source_path PATH` gives the file to every source that reads one. A source can also be given as a JSON +object, to set its own options: + +```sh +mapillary_tools process MY_CAPTURE_DIR \ + --geotag_source '{"source": "gpx", "source_path": "track.gpx", "filetypes": ["image"]}' \ + --geotag_source native +``` + +The keys are `source`, `filetypes`, `source_path`, `pattern` (see below), `interpolation_offset_time` and +`interpolation_use_gpx_start_time`. + +Gotchas: + +- A file that no source is given for fails. For example, `--geotag_source exif` on a folder with videos fails the + videos, and a source limited with `"filetypes": ["image"]` leaves the videos without one. +- `gpx` and `nmea` for images need a single track file, given with `--geotag_source_path`. Without it, the + command stops (exit code 2) when an image reaches the source. + +### --video_geotag_source + +Geotag sources for videos only. They are tried before the `--geotag_source` sources, which then geotag the images +and the videos left. If `--geotag_source` is not given, images (including the frames that `video_process` samples) +are still geotagged by the default sources. + +| Source | Reads | Default pattern | +| ------------------------------------- | ------------------------------------------------------------- | --------------- | +| `native` (`video`) | The GPS of the video, in any format that `native` reads | none | +| `gopro`, `camm`, `blackvue` | The GPS of the video, only in videos of that type | none | +| `gpx` | A GPX file | `%g.gpx` | +| `nmea` | An NMEA log | `%g.nmea` | +| `exiftool_xml` | An XML file written by `exiftool -X` | `%g.xml` | +| `exiftool_runtime` | Runs ExifTool on the video, or on the file the pattern names | `%f` | + +The pattern names the file to read, starting from the video file name: `%f` is the full name (`foo.mp4`), `%g` +the name without extension (`foo`), and `%e` the extension (`.mp4`). A relative pattern is relative to the folder +of the video. Whether a pattern is case-sensitive depends on the file system: `%g.gpx` finds `foo.GPX` on Windows +and, by default, on macOS, but not on Linux. + +```sh +# foo.mp4 is geotagged from foo.gpx beside it +mapillary_tools process VIDEO_DIR --video_geotag_source gpx + +# Stitched Insta360 videos: try a GPX first, then the GPS of the original .insv in the parent folder +mapillary_tools process VIDEO_DIR \ + --video_geotag_source gpx \ + --video_geotag_source '{"source": "exiftool_runtime", "pattern": "../%g.insv"}' +``` + +Gotchas: + +- `gopro`, `camm` and `blackvue` fail videos of another type, and say so. +- `--geotag_source_path` overrides the patterns of `gpx` and `nmea`, also those given in JSON: every video is + then placed on that one track. + +### GPX and NMEA tracks + +Images are placed on the track by their capture time; videos by the times of their frames. + +- Only GPX track points (``) are read: routes and waypoints are not. Points without a `