Skip to content

Add an integrated profiler to oneAPI.@profile #730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
maleadt merged 2 commits into from
Oct 23, 2026
Merged

Add an integrated profiler to oneAPI.@profile #730

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
488ca1e
Add bindings for Intel's PTI library
maleadt Oct 21, 2026
8135b85
Add an integrated profiler to `oneAPI.@profile`
maleadt Oct 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ version = "2.9.2"
AbstractFFTs = "621f4979-c628-5d54-868e-fcf4e3e8185c"
Adapt = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
CEnum = "fa961155-64e5-5f13-b03f-caf6b980ea82"
Crayons = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f"
ExprTools = "e2ba6199-217a-4e67-a87a-7c52f15ade04"
GPUArrays = "0c68f7d7-f131-5f86-a1c3-88cf8149b2d7"
GPUCompiler = "61eb1bfa-7361-4325-ad38-22787b887f55"
Expand All @@ -17,7 +18,9 @@ LLVM = "929cbde3-209d-540e-8aea-75f648917ca0"
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
NEO_jll = "700fe977-ac61-5f37-bbc8-c6c4b2b6a9fd"
PTI_jll = "00038326-6c92-5172-8bc1-19e163662d45"
PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
PrettyTables = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d"
Preferences = "21216c6a-2e73-6563-6e65-726566657250"
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Expand All @@ -27,6 +30,7 @@ SPIRV_LLVM_Translator_jll = "4a5d46fc-d8cf-5151-a261-86b458210efb"
SPIRV_Tools_jll = "6ac6d60f-d740-5983-97d7-a4482c0689f4"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
SpecialFunctions = "276daf66-3868-5448-9aa4-cd146d93841b"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
oneAPI_Level_Zero_Headers_jll = "f4bc562b-d309-54f8-9efb-476e56f0410d"
oneAPI_Level_Zero_Loader_jll = "13eca655-d68d-5b81-8367-6d99d727ab01"
oneAPI_Support_jll = "b049733a-a71d-5ed3-8eba-7d323ac00b36"
Expand All @@ -41,6 +45,7 @@ KernelAbstractionsExt = "KernelAbstractions"
AbstractFFTs = "1.5.0"
Adapt = "4"
CEnum = "0.5, 1"
Crayons = "4"
ExprTools = "0.1"
GPUArrays = "12.1"
GPUCompiler = "2.13"
Expand All @@ -50,13 +55,16 @@ KernelAbstractions = "0.10"
KernelInterface = "0.4"
LLVM = "10"
NEO_jll = "=26.35.39758"
PTI_jll = "1.2"
PrecompileTools = "1"
PrettyTables = "3"
Preferences = "1"
SPIRVIntrinsics = "1.4"
SPIRV_LLVM_Backend_jll = "23"
SPIRV_LLVM_Translator_jll = "23"
SPIRV_Tools_jll = "2025.4.0"
SpecialFunctions = "1.3, 2"
Statistics = "1"
julia = "1.10"
oneAPI_Level_Zero_Loader_jll = "1.32"
oneAPI_Support_jll = "0.11.0"
Expand Down
84 changes: 70 additions & 14 deletions docs/src/usage/profiling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,75 @@
# Profiling

To find out where a GPU application spends its time, run it under an external profiler,
such as Intel's [unitrace](https://github.com/intel/pti-gpu/tree/master/tools/unitrace)
or VTune. These record the Level Zero API calls the application makes and the time each
kernel and memory copy takes to execute on the device.
To find out where a GPU application spends its time, use oneAPI.jl's integrated profiler,
or an external profiler such as Intel's
[unitrace](https://github.com/intel/pti-gpu/tree/master/tools/unitrace) or VTune.

For simple timing measurements, see [Benchmarking](@ref) in the Performance Guide.

## Restricting collection
## Integrated profiler

Profiling a whole Julia session also captures compilation and other warm-up work. To only
collect a region of interest, start the profiler paused and wrap that region in
`oneAPI.@profile external=true`:
`oneAPI.@profile` runs code and reports the Level Zero API calls the host made, and the
kernels, memory copies and fills that executed on the GPU. It uses Intel's Profiling Tools
Interfaces for GPU (PTI), so it also covers work submitted by libraries such as oneMKL.
Run the code once beforehand, so that compilation does not end up in the profile:

```julia
julia> a = oneArray(rand(Float32, 1024, 1024)); b = similar(a);

julia> c = a * a; copyto!(b, c); # warm up

julia> oneAPI.@profile begin
c = a * a
copyto!(b, c)
end
Profiled over 5.1 ms.

Host-side activity: 33 Level Zero API calls taking 1.22 ms (24.02% of wall-clock)
┌──────────┬────────────┬───────┬──────────────────────────────────────┬───────────────────────────────────┐
│ Time (%) │ Total time │ Calls │ Time distribution │ Name │
├──────────┼────────────┼───────┼──────────────────────────────────────┼───────────────────────────────────┤
│ 13.38% │ 681.99 µs │ 4 │ 170.5 µs ± 340.57 ( 0.11 ‥ 681.35) │ zeCommandQueueSynchronize │
│ 5.61% │ 286.14 µs │ 1 │ │ zeContextMakeMemoryResident │
│ 2.50% │ 127.69 µs │ 1 │ │ zeCommandQueueExecuteCommandLists │
│ 0.85% │ 43.27 µs │ 1 │ │ zeMemAllocDevice │
│ 0.81% │ 41.14 µs │ 1 │ │ zeCommandListAppendMemoryCopy │
│ 0.40% │ 20.49 µs │ 1 │ │ zeCommandListReset │
│ 0.27% │ 13.74 µs │ 1 │ │ zeCommandListAppendLaunchKernel │
│ 0.08% │ 3.88 µs │ 1 │ │ zeDeviceGetProperties │
│ 0.06% │ 2.91 µs │ 15 │ 194.0 ns ± 410.03 ( 30.0 ‥ 1580.0) │ zeKernelSetArgumentValue │
│ 0.03% │ 1.72 µs │ 1 │ │ zeEventHostReset │
│ 0.01% │ 740.0 ns │ 1 │ │ zeCommandListClose │
│ 0.01% │ 420.0 ns │ 3 │ 140.0 ns ± 140.0 ( 40.0 ‥ 300.0) │ zeMemGetAllocProperties │
│ 0.00% │ 200.0 ns │ 1 │ │ zeKernelSetGroupSize │
│ 0.00% │ 120.0 ns │ 1 │ │ zeFenceReset │
└──────────┴────────────┴───────┴──────────────────────────────────────┴───────────────────────────────────┘

Device-side activity: GPU was busy for 551.25 µs (10.81% of wall-clock)
┌──────────┬────────────┬───────┬────────────────────────────────────┐
│ Time (%) │ Total time │ Calls │ Name │
├──────────┼────────────┼───────┼────────────────────────────────────┤
│ 9.80% │ 499.48 µs │ 1 │ sgemm_nocopy_nn_16x16_8x4 │
│ 1.02% │ 51.77 µs │ 1 │ zeCommandListAppendMemoryCopy(D2D) │
└──────────┴────────────┴───────┴────────────────────────────────────┘
```

With `trace=true`, the profiler lists every API call and GPU operation in chronological
order instead. Their ID column links each GPU operation to the API call that submitted it.
To profile code that runs quickly, `oneAPI.@bprofile` runs it repeatedly for a second (or
`time` seconds) and reports the aggregated results.

The integrated profiler is only available on x86_64 Linux.

```@docs
oneAPI.@profile
oneAPI.@bprofile
```

## External profilers

External profilers offer more detail, such as hardware metrics. Profiling a whole Julia
session also captures compilation and other warm-up work, so to only collect a region of
interest, start the profiler paused and wrap that region in `oneAPI.@profile external=true`:

```julia
using oneAPI
Expand All @@ -26,11 +84,9 @@ oneAPI.@profile external=true begin
end
```

```@docs
oneAPI.@profile
```
When Julia runs under an external profiler, `oneAPI.@profile` defaults to `external=true`.

## Launching the profiler
### Launching the profiler

oneAPI.jl uses the Level Zero loader and the Intel GPU driver from its own artifacts. An
external profiler that loads into the process before Julia starts may pick up another copy
Expand All @@ -55,7 +111,7 @@ eval "$(julia --project -e '
This is not needed when oneAPI.jl is configured to use the system libraries (see
[Using System Libraries (Advanced)](@ref)).

### unitrace
#### unitrace

unitrace is part of Intel's [PTI-GPU](https://github.com/intel/pti-gpu) tools. Start it
with `--start-paused` so that only the code in `oneAPI.@profile external=true` is
Expand All @@ -69,7 +125,7 @@ unitrace --start-paused --device-timing --chrome-kernel-logging \
`--device-timing` prints a summary of kernel and memory copy times when the application
exits; the Chrome trace can be opened in [Perfetto](https://ui.perfetto.dev).

### VTune
#### VTune

VTune finds the ITT collector through the `INTEL_LIBITTNOTIFY64` environment variable,
which it sets for applications it launches. Start the collection paused:
Expand Down
73 changes: 73 additions & 0 deletions lib/pti/PTI.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
module PTI

using ..APIUtils

using CEnum: @cenum

using PTI_jll

# PTI_jll does not load its libraries: libpti_view starts a thread once used, and libpti
# hooks into the Level Zero loader. They are only loaded by the first call into them.
is_available() = PTI_jll.is_available()

include("libpti.jl")

export PTIError

struct PTIError <: Exception
code::pti_result
end

Base.showerror(io::IO, err::PTIError) =
print(io, "PTIError: ", unsafe_string(ptiResultTypeToString(err.code)),
" (code $(Int(err.code)))")


## buffer management

# PTI requests and returns buffers from its own threads, including while the process exits,
# where running Julia code would hang. libpti_buffers implements the callbacks in C, and
# queues the completed buffers for `foreach_record` to process.
const callbacks_set = Ref(false)

function set_callbacks()
callbacks_set[] && return
ptiViewSetCallbacks(cglobal((:pti_buffers_request, libpti_buffers)),
cglobal((:pti_buffers_complete, libpti_buffers)))
callbacks_set[] = true
return
end

# Call `f` on a pointer to every record in the completed buffers, which are released
# afterwards. PTI does not document how long the strings that records point to stay valid,
# so `f` should copy anything it needs.
function foreach_record(f)
buf = Ref{Ptr{UInt8}}()
used = Ref{Csize_t}()
record = Ref{Ptr{pti_view_record_base}}()
while ccall((:pti_buffers_pop, libpti_buffers), Cint,
(Ptr{Ptr{UInt8}}, Ptr{Csize_t}), buf, used) == 1
try
record[] = C_NULL
while true
res = unchecked_ptiViewGetNextRecord(buf[], used[], record)
res == PTI_STATUS_END_OF_BUFFER && break
res == PTI_SUCCESS || throw_api_error(res)
f(record[])
end
finally
Libc.free(buf[])
end
end
end

# Number of buffers whose records were lost because they could not be allocated.
lost_buffers() = ccall((:pti_buffers_lost, libpti_buffers), UInt64, ())

function api_name(group, id)
name = Ref{Ptr{Cchar}}()
res = unchecked_ptiViewGetApiIdName(group, id, name)
res == PTI_SUCCESS ? unsafe_string(name[]) : "unknown API $id"
end

end
Loading
Loading