Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 21 additions & 8 deletions docs/configuration/options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,12 @@ You can limit allowed source URLs with the following variable:
Disabling the limit on the SVG file size can lead to memory exhaustion and DoS attacks.
:::

* [`IMGPROXY_TIFF_UNLIMITED`]: when `true`, disables the limit on the TIFF file size (50MB). Default: `false`

:::warning
Disabling the limit on the TIFF file size can lead to memory exhaustion and DoS attacks.
:::

* [`IMGPROXY_SANITIZE_SVG`]: when `true`, imgproxy will remove scripts from SVG images to prevent XSS attacks. Defaut: `true`

When using imgproxy in a development environment, it can be useful to ignore SSL verification:
Expand Down Expand Up @@ -383,6 +389,13 @@ imgproxy Pro can extract specific video frames to create thumbnails. This featur
Though using `IMGPROXY_VIDEO_THUMBNAIL_PROBE_SIZE` and `IMGPROXY_VIDEO_THUMBNAIL_MAX_ANALYZE_DURATION` can lower the memory footprint of video thumbnail generation, they should be used in production only when you know what you're doing.
:::

Check out the [Video thumbnails](../features/video_thumbnails.mdx) guide to learn more.

imgproxy Pro can also use VA-API to hardware-accelerate video frame decoding (and, optionally, scaling) on Intel and AMD GPUs. See the [Hardware acceleration](../features/hardware_acceleration.mdx) page for setup instructions.

* [`IMGPROXY_VIDEO_THUMBNAIL_VAAPI_DEVICE`]: ((pro)) the path to a DRM render node (e.g., `/dev/dri/renderD128`) to use for VA-API hardware-accelerated video decoding. When empty, VA-API is disabled. Default: blank
* [`IMGPROXY_VIDEO_THUMBNAIL_VAAPI_SCALE`]: ((pro)) when `true`, also uses VA-API hardware scaling in the ffmpeg filter graph instead of only hardware decoding. Default: `false`

## RAW formats

imgproxy Pro can process digital camera RAW images. Since loading RAW images is pretty resource intensive, their processing is disabled by default. You can enable it with the following variable:
Expand Down Expand Up @@ -452,10 +465,10 @@ imgproxy can detect objects on the image and use them to perform smart cropping,
* `one_best`: in this mode, imgproxy will focus on the object with the highest score based on its area, confidence, and class weight.
* `one_best_centermost`: the same as `one_best,` but imgproxy will add the object's proximity to the image center to its score.

* [`IMGPROXY_OBJECT_DETECTION_INFERENCE_NUM_STREAMS`]: ((pro)) when set, overrides the [IMGPROXY_ML_NUM_STREAMS](#IMGPROXY_ML_NUM_STREAMS) value for the object detection model.
* [`IMGPROXY_OBJECT_DETECTION_INFERENCE_NUM_THREADS`]: ((pro)) when set, overrides the [IMGPROXY_ML_INFERENCE_NUM_THREADS](#IMGPROXY_ML_INFERENCE_NUM_THREADS) value for the object detection model.
* [`IMGPROXY_OBJECT_DETECTION_EXECUTION_MODE`]: ((pro)) when set, overrides the [IMGPROXY_ML_EXECUTION_MODE](#IMGPROXY_ML_EXECUTION_MODE) value for the object detection model.
* [`IMGPROXY_OBJECT_DETECTION_CPU_PINNING`]: ((pro)) when set, overrides the [IMGPROXY_ML_CPU_PINNING](#IMGPROXY_ML_CPU_PINNING) value for the object detection model.
* [`IMGPROXY_OBJECT_DETECTION_ML_INFERENCE_NUM_STREAMS`]: ((pro)) when set, overrides the [IMGPROXY_ML_NUM_STREAMS](#IMGPROXY_ML_NUM_STREAMS) value for the object detection model.
* [`IMGPROXY_OBJECT_DETECTION_ML_INFERENCE_NUM_THREADS`]: ((pro)) when set, overrides the [IMGPROXY_ML_INFERENCE_NUM_THREADS](#IMGPROXY_ML_INFERENCE_NUM_THREADS) value for the object detection model.
* [`IMGPROXY_OBJECT_DETECTION_ML_EXECUTION_MODE`]: ((pro)) when set, overrides the [IMGPROXY_ML_EXECUTION_MODE](#IMGPROXY_ML_EXECUTION_MODE) value for the object detection model.
* [`IMGPROXY_OBJECT_DETECTION_ML_CPU_PINNING`]: ((pro)) when set, overrides the [IMGPROXY_ML_CPU_PINNING](#IMGPROXY_ML_CPU_PINNING) value for the object detection model.

Read the [Object Detection guide](../features/object_detection.mdx) for more info.

Expand All @@ -476,10 +489,10 @@ imgproxy can classify images by assigning them to predefined categories based on
* `nhwc`: _(default)_ channels last (TensorFlow default)
* `nchw`: channels first (PyTorch default)

* [`IMGPROXY_CLASSIFICATION_INFERENCE_NUM_STREAMS`]: ((pro)) when set, overrides the [IMGPROXY_ML_NUM_STREAMS](#IMGPROXY_ML_NUM_STREAMS) value for the classification model.
* [`IMGPROXY_CLASSIFICATION_INFERENCE_NUM_THREADS`]: ((pro)) when set, overrides the [IMGPROXY_ML_INFERENCE_NUM_THREADS](#IMGPROXY_ML_INFERENCE_NUM_THREADS) value for the classification model.
* [`IMGPROXY_CLASSIFICATION_EXECUTION_MODE`]: ((pro)) when set, overrides the [IMGPROXY_ML_EXECUTION_MODE](#IMGPROXY_ML_EXECUTION_MODE) value for the classification model.
* [`IMGPROXY_CLASSIFICATION_CPU_PINNING`]: ((pro)) when set, overrides the [IMGPROXY_ML_CPU_PINNING](#IMGPROXY_ML_CPU_PINNING) value for the classification model.
* [`IMGPROXY_CLASSIFICATION_ML_INFERENCE_NUM_STREAMS`]: ((pro)) when set, overrides the [IMGPROXY_ML_NUM_STREAMS](#IMGPROXY_ML_NUM_STREAMS) value for the classification model.
* [`IMGPROXY_CLASSIFICATION_ML_INFERENCE_NUM_THREADS`]: ((pro)) when set, overrides the [IMGPROXY_ML_INFERENCE_NUM_THREADS](#IMGPROXY_ML_INFERENCE_NUM_THREADS) value for the classification model.
* [`IMGPROXY_CLASSIFICATION_ML_EXECUTION_MODE`]: ((pro)) when set, overrides the [IMGPROXY_ML_EXECUTION_MODE](#IMGPROXY_ML_EXECUTION_MODE) value for the classification model.
* [`IMGPROXY_CLASSIFICATION_ML_CPU_PINNING`]: ((pro)) when set, overrides the [IMGPROXY_ML_CPU_PINNING](#IMGPROXY_ML_CPU_PINNING) value for the classification model.

Read the [Classification guide](../features/classification.mdx) for more info.

Expand Down
152 changes: 152 additions & 0 deletions docs/features/hardware_acceleration.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
---
title: Hardware acceleration
description: Learn about how to use VA-API hardware acceleration to speed up video thumbnail generation
---

# Hardware acceleration ((pro))

imgproxy Pro can use [VA-API](https://en.wikipedia.org/wiki/Video_Acceleration_API) to
hardware-accelerate video frame decoding (and, optionally, scaling) when
[generating video thumbnails](../configuration/options.mdx#video-thumbnails). VA-API is supported on both Intel and AMD GPUs
through the Linux DRM render-node interface, offloading decode/scale work from the CPU to the GPU
and reducing CPU usage in deployments that process a lot of video.

:::note
This page covers standard VA-API acceleration on Intel and AMD GPUs. It doesn't apply to
purpose-built media-accelerator hardware such as Amazon EC2 [VT1 instances](https://aws.amazon.com/ec2/instance-types/vt1/)
(Xilinx Alveo U30 accelerators). Those rely on the Xilinx Video SDK, which ships its own
pre-compiled FFmpeg build with proprietary Xilinx plugins instead of VA-API — imgproxy's VA-API
support can't make use of that hardware.
:::

## How it works

imgproxy Pro's official Docker images already ship an `ffmpeg` build compiled with VA-API support.
To actually use hardware acceleration at runtime, you need three things:

1. A vendor VA-API userspace driver installed in the image (imgproxy's official image doesn't
include one, since it's vendor-specific and has a significant impact on the resulting image size — see
[Building a custom image](#building-a-custom-image) below).
2. The host's DRM render node (`/dev/dri/renderD128` or similar) exposed to the container, with
permissions allowing imgproxy to open it.
3. The [`IMGPROXY_VIDEO_THUMBNAIL_VAAPI_DEVICE`](../configuration/options.mdx#IMGPROXY_VIDEO_THUMBNAIL_VAAPI_DEVICE)
(and, optionally, [`IMGPROXY_VIDEO_THUMBNAIL_VAAPI_SCALE`](../configuration/options.mdx#IMGPROXY_VIDEO_THUMBNAIL_VAAPI_SCALE))
config variables are set.

## Building a custom image

You need to build a custom image based on the official one that adds the driver for your GPU vendor.

### Intel GPUs

```dockerfile
FROM darthsim/imgproxy-pro:v4 # or v4-ml

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
intel-media-va-driver-non-free \
&& rm -rf /var/lib/apt/lists/*
```

### AMD GPUs

```dockerfile
FROM darthsim/imgproxy-pro:v4 # or v4-ml

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
mesa-va-drivers \
&& rm -rf /var/lib/apt/lists/*
```

In both cases, no rebuild of `ffmpeg`/`libva` is needed — VA-API support is already built into the
official image; only the vendor driver package is missing.

Build the image:

```bash
docker build . -t imgproxy-pro-vaapi
```

## Host requirements

Before running a hardware-accelerated container, the host itself needs a working VA-API render
node:

1. **A GPU with its kernel driver loaded.** On Linux, AMD GPUs use the `amdgpu` driver. Intel GPUs
use either `i915` (the long-standing driver, still the default for Arc Alchemist, Meteor Lake,
and older integrated GPUs) or `xe` (Intel's newer driver, the default since Lunar Lake and Arc
Battlemage, and optional/experimental on earlier Xe-based GPUs). Both ship with the kernel and
load automatically when the GPU is detected — you don't need to install anything for this on
bare metal. Verify with:

```bash
lsmod | grep -E 'i915|xe|amdgpu'
ls /dev/dri
```

You should see device nodes such as `/dev/dri/renderD128` (the render node) and
`/dev/dri/card0` (the primary/display node).

2. **You only need the render node, not `card0`.** ffmpeg's VA-API decode/scale path only opens
the render node (`/dev/dri/renderD128` or similar). `/dev/dri/card0` is used for
mode-setting/display and is unrelated to hardware-accelerated video decoding — there's no need
to expose it to the container.

3. **Group permissions.** The render node is owned by the host's `render` group (its gid varies
by distro/host). Add the user that runs the container to that group:

```bash
sudo usermod -aG render $USER
```

4. **Running under a hypervisor or cloud VM.** If imgproxy runs inside a VM rather than on bare
metal, the VM also needs the GPU exposed to it (PCI passthrough, SR-IOV, or a vendor virtual-GPU
mechanism) before any of the above applies — that's hypervisor/cloud-provider specific and
outside the scope of this page.

## Running the container

Expose the host's DRM render node to the container and make sure imgproxy's process can open it.
If imgproxy's Docker image runs as a non-root user, you typically need to add that user's group to
the host's `render` (or, on some distros, `video`) group that owns the device node:

```bash
# Find the gid of the host group that owns the render node
getent group render

# Run the container with the device exposed and the matching group added
docker run \
--device=/dev/dri \
--group-add <render-gid> \
-e IMGPROXY_ENABLE_VIDEO_THUMBNAILS=true \
-e IMGPROXY_VIDEO_THUMBNAIL_VAAPI_DEVICE=/dev/dri/renderD128 \
-e IMGPROXY_VIDEO_THUMBNAIL_VAAPI_SCALE=true \
imgproxy-pro-vaapi:latest
```

:::tip
If you're running Docker in **rootless** mode, the container's user namespace doesn't automatically
see the host's `render` group — you need to map it explicitly via `/etc/subgid` and restart the
rootless daemon, then pass the *namespace-mapped* gid (not the raw host gid) to `--group-add`:

```bash
# Find the host gid of the render group (the 3rd field, e.g. "993")
getent group render

# Map that gid into the rootless user's subordinate gid range
echo "your-user:993:1" | sudo tee -a /etc/subgid
systemctl --user restart docker
```
:::

## Verifying / troubleshooting

If `IMGPROXY_VIDEO_THUMBNAIL_VAAPI_DEVICE` is set but imgproxy can't open the device, it will fail
fast on startup with an error along the lines of "failed to ensure availability of the hardware
acceleration device". If you see this:

* Double check the device path matches an actual render node on the host (`ls /dev/dri`).
* Double check the container's `--device=/dev/dri` mount and that the container process has
permission to open the render node (see [Running the container](#running-the-container) above).
* Make sure the vendor VA-API driver package is installed in the image and matches your GPU.
131 changes: 131 additions & 0 deletions docs/features/video_thumbnails.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
title: Video thumbnails
description: Learn about how imgproxy Pro generates thumbnails, sprites, and animated previews from video sources
---

# Video thumbnails ((pro))

imgproxy Pro can use a video as a source and extract still frames: a single
thumbnail, a tiled sprite sheet, or an animated preview. imgproxy downloads only as
much of the video as it needs to reach the requested frame(s), instead of requiring
the whole file upfront.

Since even partial video downloads and decoding are considerably more expensive than
processing a regular image, video thumbnail generation is **turned off by default**.
Enable it with [`IMGPROXY_ENABLE_VIDEO_THUMBNAILS`](../configuration/options.mdx#IMGPROXY_ENABLE_VIDEO_THUMBNAILS).

:::info
imgproxy Pro can also use VA-API to hardware-accelerate video frame decoding. See the
[Hardware acceleration](hardware_acceleration.mdx) page.
:::

## How it works

imgproxy doesn't rely on a fixed file-extension allowlist to decide whether a source
is a video. It first checks the source against all supported image formats and, only if
none match, hands the data to FFmpeg's container prober as a last resort.

The only video source format currently supported is MP4 (H.264). See the
[format support table](../image_formats_support.mdx) for the current source/result matrix.

The video is fetched through imgproxy's normal streaming/ranged HTTP fetcher rather
than downloaded in full, which is why `Content-Length` isn't pre-validated for video
sources the way it is for images — imgproxy reads only as far into the stream as it
needs to.

[`IMGPROXY_MAX_SRC_RESOLUTION`](../configuration/options.mdx#IMGPROXY_MAX_SRC_RESOLUTION)
applies to the decoded video frame's dimensions exactly as it does to still images.

imgproxy favors returning something over failing outright: if a requested timestamp
falls beyond the video's actual duration, imgproxy returns the last successfully
decoded frame instead of an error. Sprites and animations behave the same way — if the
source stream ends before all the requested frames were collected, imgproxy returns
whatever frames it managed to decode.

## Single-frame thumbnails

Use the [video_thumbnail_second](../usage/processing.mdx#video-thumbnail-second)
(`vts`) processing option, or the
[IMGPROXY_VIDEO_THUMBNAIL_SECOND](../configuration/options.mdx#IMGPROXY_VIDEO_THUMBNAIL_SECOND)
config default, to pick which second of the video should be used for the thumbnail:

```imgproxy_url
.../resize:fill:300:200/video_thumbnail_second:30/plain/https://example.com/video.mp4@jpg
```

By default, imgproxy decodes forward to the exact requested second. Setting
[video_thumbnail_keyframes](../usage/processing.mdx#video-thumbnail-keyframes) (`vtk`),
or the [IMGPROXY_VIDEO_THUMBNAIL_KEYFRAMES](../configuration/options.mdx#IMGPROXY_VIDEO_THUMBNAIL_KEYFRAMES)
config, makes imgproxy seek to the nearest preceding keyframe and use the first frame
it decodes from there. This is much faster on videos with long gaps between keyframes, but the returned
frame's timestamp may not exactly match the one you requested.

As mentioned in [How it works](#how-it-works), if the requested second is past the end
of the video, imgproxy returns the last frame it could decode instead of an error.

## Tiled sprite sheets

The [video_thumbnail_tile](../usage/processing.mdx#video-thumbnail-tile) (`vtt`)
processing option generates a grid of frames sampled across the video — handy for
building scrubber-preview UIs like the ones used by video players:

```imgproxy_url
.../video_thumbnail_tile:-1:5:2:160:90/plain/https://example.com/video.mp4@jpg
```

This example generates a 5x2 grid of 160x90 tiles. When the `step` argument (the first
one) is `0` or negative, imgproxy spreads the tiles evenly across the remaining
duration of the video, calculating the actual step as
`(video_duration - start) / (columns * rows)`.

For videos with long steps between tiles, the
[IMGPROXY_VIDEO_THUMBNAIL_TILE_AUTO_KEYFRAMES](../configuration/options.mdx#IMGPROXY_VIDEO_THUMBNAIL_TILE_AUTO_KEYFRAMES)
config makes imgproxy automatically switch to keyframe-only seeking once the step
exceeds the keyframe interval, trading a bit of precision for a significant
speedup.

## Animated previews

The [video_thumbnail_animation](../usage/processing.mdx#video-thumbnail-animation)
(`vta`) processing option generates an animated preview instead of a static grid,
sampling frames across the video the same way tiles do:

```imgproxy_url
.../video_thumbnail_animation:-1:500:10:320:180/plain/https://example.com/video.mp4@gif
```

This example generates a 10-frame, 320x180 animation with a 500ms delay between
frames, spread evenly across the video's duration.

[video_thumbnail_tile](../usage/processing.mdx#video-thumbnail-tile) and
[video_thumbnail_animation](../usage/processing.mdx#video-thumbnail-animation) are
mutually exclusive — setting one clears any previously set value of the other so that
a single request can produce only a tile sheet or an animation, not both.

## Fetching video metadata

The [`/info` endpoint](../usage/getting_info.mdx)'s
[video_meta](../usage/getting_info.mdx#video-meta) (`vm`) option returns per-stream
metadata for video sources — codec, duration, bitrate, frame rate, and more — which is
useful for building a UI that picks a sensible thumbnail timestamp before requesting
it:

```imgproxy_url
.../info/video_meta:1/plain/https://example.com/video.mp4
```

## Configuration

* [`IMGPROXY_ENABLE_VIDEO_THUMBNAILS`]: when `true`, enables video thumbnail generation. Default: `false`
* [`IMGPROXY_VIDEO_THUMBNAIL_SECOND`]: the timestamp (in seconds) used for a thumbnail. Default: `1`
* [`IMGPROXY_VIDEO_THUMBNAIL_KEYFRAMES`]: when `true`, uses the nearest preceding keyframe instead of the exact requested frame. Default: `false`
* [`IMGPROXY_VIDEO_THUMBNAIL_TILE_AUTO_KEYFRAMES`]: when `true`, automatically switches tiles to keyframes-only seeking once the `step` exceeds the interval between keyframes. Default: `false`
* [`IMGPROXY_VIDEO_THUMBNAIL_PROBE_SIZE`]: the maximum number of bytes used to probe the container format. Default: `5000000`
* [`IMGPROXY_VIDEO_THUMBNAIL_MAX_ANALYZE_DURATION`]: the maximum number of milliseconds spent analyzing streams. When `0`, a heuristic is used. Default: `0`

imgproxy Pro can also use VA-API hardware acceleration to speed up video frame
decoding — see the VA-API-specific config variables on the
[Hardware acceleration](hardware_acceleration.mdx) page.

Check out the [Video thumbnails](../configuration/options.mdx#video-thumbnails)
config reference for the complete list of related config variables.
5 changes: 2 additions & 3 deletions docs/image_formats_support.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -85,12 +85,11 @@ Since MP4 requires use of a `<video>` tag instead of `<img>`, automatic conversi

## Video thumbnails ((pro)) {#video-thumbnails}

If you provide a video as a source, imgproxy takes a specific frame to create a thumbnail. To do this, imgproxy downloads only the amount of data required to reach the needed frame.
If you provide a video as a source, imgproxy Pro takes a specific frame — or a tiled sprite sheet or animated preview built from several frames — to create a thumbnail. To do this, imgproxy downloads only the amount of data required to reach the needed frame(s).

Since this still requires more data to be downloaded, video thumbnail generation is disabled by default and should be enabled with `IMGPROXY_ENABLE_VIDEO_THUMBNAILS` config option.

* `IMGPROXY_ENABLE_VIDEO_THUMBNAILS`: when true, enables video thumbnail generation. Default: `false`
* `IMGPROXY_VIDEO_THUMBNAIL_SECOND`: the timestamp of the frame (in seconds) that will be used for the thumbnail. Default: `1`.
Check out the [Video thumbnails](features/video_thumbnails.mdx) guide to learn more.

## Colorspace and HDR preservation

Expand Down
Loading
Loading