Rust extension given invalid name by meson-python

Link to reproducible example: https://github.com/medley56/python_rust

When using meson-python to package a Rust extension, I get the following error when running pip install .

../meson.build:10:14: ERROR: Rust crate python_rust_lib.cpython-312-darwin type dylib does not allow spaces, periods or dashes in the library name due to a limitation of rustc. Replace them with underscores, for example

The project works when using maturin as a build backend so I'm reasonably sure it's not a problem with the actual extension. I believe the problem is in meson-python or in my configuration of meson-python, causing the backend to build the rust extension with an invalid name.

Platform: MacOS Sequioia 15.3

Documentation I've consulted:
Tutorial: https://mesonbuild.com/meson-python/tutorials/introduction.html#tutorial
meson docs for extension_module: https://mesonbuild.com/Python-module.html#extension_module
PyO3 docs (uses maturin): https://pyo3.rs/v0.23.4/module.html

@medley56

Thanks for the bug report and reproducer @medley56! We should make that work - I think the reason you're running into it is that you're the first one to try. Rust support in Meson is pretty recent, and none of the maintainers of meson-python are regular Rust users I believe.

One thing that jumps out when browsing the code is that this line in meson.build:

py = import('python').find_installation()

should be:

py = import('python').find_installation(pure: false)

That may not be the problem though, I'll try to test as soon as possible.

Let's call this a feature request rather than a bug. Maturin does the following (from https://www.maturin.rs/bindings#pyo3): "maturin automatically detects pyo3 bindings when it's added as a dependency in Cargo.toml."

Neither Meson nor meson-python does anything like this. Using py.extension() on a Rust source file needs to do either something like what Maturin does (adding a pyo3 dependency and invoking the right machinery), or the user must do this explicitly in meson.build instead of feeding a single .rs file to py.extension.

I did a bit of digging in the Meson issue tracker, but can't find any relevant discussions - it may be that no one has worked on this yet.

In meson-python itself we should probably start by adding some test cases for packaging projects that are known to work with Meson as wheels, just to make sure that that all works (it should).

Thanks for the attention to this! Looking forward to being able to use meson to build a Rust extension. It sounds like you have an idea for a workaround:

or the user must do this explicitly in meson.build instead of feeding a single .rs file to py.extension

Is that something simple/useful or would you rather just call this unsupported until a better solution can be implemented? For my part, I'm happy to just use maturin for now.

It looks like if the crate is declared as cdylib, then we need to specify the rust_abi in the extension module.

rust_ext = py.extension_module(
    'python_rust_lib',  # Output name
    'src/lib.rs',  # Rust source
    install: true,
    rust_abi: 'c'
)

However, it appears that building the extension module still doesn't follow the crate dependencies and install those (it can't find pyo3 when compiling). Do you need to manually declare a dependency on other crates?

Meson can have a rust target depend on a crate: https://mesonbuild.com/Wrap-dependency-system-manual.html#cargo-wraps

dependencies: [dependency('foo-rs')]

and a cargo subproject dependency will automatically parse Cargo.toml and depend on its dependencies. But you still need to define your direct dependencies, just like you do with cargo (but with cargo, you define them in Cargo.toml instead of meson.build).

FTR: I don't really have experience with meson's rust support, that is being primarily driven by maintainers from the Gnome and Mesa ecosystems. I don't know offhand about anyone doing python module experiments with it, the experiments I've seen have involved replacing C/C++ libraries and executables. In theory, pyo3 extensions are "just" libraries that happen to utilize libpython, so it should be more or less the same modulo usage of the pyo3 crate itself.

@xclaesse

I've tried to put together an example project building with Meson a Python module using PyO3. The issue with the cdylib being given a Rust invalid name can be worked around compiling a Rust static lib and then linking it into a python module whose sources are an empty C file (at least Meson does not complain about this, I'm not sure if it actually works when linking). The second ingredient is a Cargo.lock that lists PyO3 and all transitive dependencies. Then Meson is able to resolve dependency('pyo3-0.25-rs'), generate wrap files for all required crates and start building. However, the build does not go very far: one of the PyO3 dependencies uses some proc_macro dependency and AFAICT Meson is not yet able to make sense of that. I don't know much about Rust (yet) and I don't know where to go from here.

I think refusing the name for the cdylib is a Meson bug and someone more familiar with Rust should probably have a look at this. I had a quick look at how maturin compiles Python extensions and AFAICT it directly instructs cargo to compile a cdylib with a name equivalent to the one the Meson's python module uses.

Paging @xclaesse, @dcbaker, and @bonzini for a consult.

@nrposner

I'm a PyO3 maintainer and am very interested in this, as it will eventually unblock adding Rust code to scientific python projects.

I discovered this issue while talking with @nrposner on the PyO3 discord, they also expressed interest in helping out.

@davidhewitt might also be able to answer questions from the Rust build side.

This is looking a little further ahead, I think eventually the main PyO3 repo could grow a meson-python example to go along with the maturin and setuptools-rust examples we have already.

I hope that Meson 1.11 will be able to parse Cargo.toml from the toplevel project directory and use that to configure all the dependent crates (as long as they don't have a build.rs, at least, otherwise it won't be totally automatic).

Based on that, which is currently drafted at mesonbuild/meson#15223, one should be able to add the right keyword arguments to py.extension_module(), and it will link to pyo3.

rust_abi: 'c' however could be added by extension_module already now, since a Rust-ABI Python extension module makes no sense.

@bonzini

Thanks for the interest. I don't think there is anything that needs to be done on the meson-python side of things. However, as pointed out by @bonzini there may be tweaks required in the Meson's python module. PyO3 seems to have quite a few crate dependencies. However, I believe, it should be possible to write a simple Python extension in Rust calling directly into the Python C API. If someone could provide that, it would be easy to add it as a test for meson-python and check what needs adjusting.

Note that there are some things which could be investigated in parallel to make things easier. Not familiar with the pyo3 codebase at all, but a common issue with rust crates in meson is:

rust crates that use C/C++ DFOO=bar" macros, using the rust-native spelling env!`.

Rustc does have a -D option (they call it --set-env) but it is nightly-only and there seem to be no plans to actually stabilize it. Even the nightly flag is basically new. The original design of rustc required the build system generator invoking rustc (cargo, or ninja in the case of meson) to set environment variables instead, which ninja doesn't support. IIRC there was some drama about how C's implementation of source code configuration via tuneable definitions occurs before lexing and therefore command line arguments are bad, and the idea has never fully left the devs.

In general it's better to generate the source files instead, which meson can handle via structured_sources()

PyO3 doesn't have many dependencies but it has a large build.rs. the main issue there will be to figure out how much of it duplicates the Meson Python module, and fill in the blanks.

Probably there would be a meson.build file that replaces build.rs when building PyO3 as a subproject, but that's getting a bit ahead of ourselves.

Thanks for the effort! The hdf5 libraries can be very picky.

It looks like you can specify the HDF5_DIR environment variable to the root of the conda environment and set some other things.

https://github.com/metno/hdf5-rust/blob/3983f3d4a2c878f731225bba2e7fcd5aa7cbe332/README.md#conda

RUSTFLAGS="-C link-args=-Wl,-rpath,$HDF5_DIR/lib".

For old versions of HDF5 conda packages on macOS, it may also be necessary to set DYLD_FALLBACK_LIBRARY_PATH="$HDF5_DIR/lib"

But I haven't tried this myself so I don't know if it works.

@bonzini

none of the examples I have been able to find build rust extensions in subdirectories, and I have tried (and failed) to implement this in my project

At this point in time, compiling a Python extension module written in Rust requires a version of Meson for which only a release candidate has been released, thus I would not suggest to rely on it for anything that is not testing the feature. If the Rust implementation uses PyO3 you need to translate the build.rs for the pyo3-ffi Rust module into an equivalent moson.build (as is was being discussed in earlier comments to this issue). An early attempt of doing so has been made by @bonzini but it does not yet work correctly on Windows. The same applies for all other Rust dependencies that require a build.rs. Judging from the exchange above, your project has at least one. Therefore, there is a non negligible amount of work to do. I recommend using maturine as build backend if your project is purely Rust.

Nevertheless, I'll try to verify how to make what you are trying to work.

@bonzini

I started experimenting with this on a ~real use-case, see scipy/scipy#25922. Would appreciate if you could check it out @bonzini @dnicolodi ! In particular, would be good to get rid of the meson.build patches for --cfg=Py_3_14 etc. that that PR contains right now.

In particular, would be good to get rid of the meson.build patches for --cfg=Py_3_14 etc. that that PR contains right now.

This is not possible, unless upstream PyO3 ships meson.build files equivalent to the existing build.rs. I spent some time writing a complete translation of the PyO3 build system to Meson, however, I haven't got it working on Windows, and I run out of time.

I do not understand way you decided to ship meson.build additions for the pyo3-ffi and pyo3 crates the way you do. Meson can overlay these on top of the downloaded sources via the subprojects/packagefiles/ mechanism. AFAIK these are shipped as part of the source distribution, if you want to ship subprojects in the tarball. In other words, I don't understand the need for your tools/apply_rust_patches.py script.

Meson can overlay these on top of the downloaded sources via the subprojects/packagefiles/ mechanism. AFAIK these are shipped as part of the source distribution, if you want to ship subprojects in the tarball. In other words, I don't understand the need for your tools/apply_rust_patches.py script.

I can probably switch to that instead — I wanted to avoid making the diff too large so I blanket gitignored changes to the subprojects dir.

I can probably switch to that instead — I wanted to avoid making the diff too large so I blanket gitignored changes to the subprojects dir.

Why would you add the content of the subprojects directory to the git repo? I would just add subprojects/packagefiles/ and let Meson download the sources when meson setup is called, pinning the versions via Cargo.lock.

I can probably switch to that instead — I wanted to avoid making the diff too large so I blanket gitignored changes to the subprojects dir.

Why would you add the content of the subprojects directory to the git repo? I would just add subprojects/packagefiles/ and let Meson download the sources when meson setup is called, pinning the versions via Cargo.lock.

done now :)

I got PyO3 building with Meson also on Windows, see https://github.com/dnicolodi/rust-pyo3-meson The meson.build requires some cleanups and for now it works only with PyO3 0.29.0. It requires some adjustments to work with the latest 0.29.2 version.

none of the examples I have been able to find build rust extensions in subdirectories, and I have tried (and failed) to implement this in my project

At this point in time, compiling a Python extension module written in Rust requires a version of Meson for which only a release candidate has been released, thus I would not suggest to rely on it for anything that is not testing the feature. If the Rust implementation uses PyO3 you need to translate the build.rs for the pyo3-ffi Rust module into an equivalent moson.build (as is was being discussed in earlier comments to this issue). An early attempt of doing so has been made by @bonzini but it does not yet work correctly on Windows. The same applies for all other Rust dependencies that require a build.rs. Judging from the exchange above, your project has at least one. Therefore, there is a non negligible amount of work to do. I recommend using maturine as build backend if your project is purely Rust.

Nevertheless, I'll try to verify how to make what you are trying to work.

Thanks for giving me the time of day. I've been really busy with work the past two weeks, so sorry for not getting back to you quicker.

I've written a lot of C extensions to my Python modules using setuptools and meson. The repository I linked only has rust, and has no C, so you're right that it would be simpler to support wheels of my csrk-py in maturin for that project. However, I was using it as a place to test meson integration because I had a contributor to one of my mixed Python/C projects who wants to develop Rust extensions, and I wanted to find a way to accept those contributions.

That was why we developed this crazy scheme of manually linking the Rust extensions, and I tested it out in the csrk-py repository before trying to replicate it in the mixed C / Python repository.

The other thing I could do is push a second set of packages to Pypi with the modules that have Rust extensions, and import those in the module that has the C extensions, but it sounds like that could easily become messy and unmaintainable.

It's possible that Maturin and a second set of wheels might result in a more straightforward installation method despite the burdens of maintaining a second package, because for my C extensions I am supporting a limited API which creates manylinux wheels from python 3.9 through 3.14, and keeping the wheels simple with additional languages may be impossible.

Restricting my other package to C extensions might ultimately be necessary (especially as long as I need to support 3.9), but when we get a fresh grad student who wants to write Rust extensions I'd like to be able to find a way to accept those contributions and guide the student if it's possible.

Rust extension modules are working with Meson 1.12, and as expected, this does not require any modification to meson-python, see #874 and https://github.com/dnicolodi/rust-pyo3-meson These examples use the Meson rust module and Cargo.toml but the same result can probably be achieved in other ways.

Using PyO3 crates requires translating the relevant parts of their build system to equivalent meson.build. That is included in #874 and https://github.com/dnicolodi/rust-pyo3-meson. Having that distributed with PyO3 is tracked in PyO3/pyo3#6321.

As there is nothing to be done in meson-python to support Rust extension modules, I'm closing this issue.

Navigation Menu

Repository navigation

Rust extension given invalid name by meson-python #721

64 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Rust extension given invalid name by meson-python #721

Description

Activity

rgommers commented on Jan 30, 2025

rgommers commented on Jan 30, 2025

rgommers commented on Jan 30, 2025

medley56 commented on Jan 30, 2025

greglucas commented on Jan 30, 2025

eli-schwartz commented on Jan 30, 2025

dnicolodi commented on Jun 16, 2025

ngoldbaum commented on Nov 12, 2025

bonzini commented on Nov 12, 2025

dnicolodi commented on Nov 13, 2025

eli-schwartz commented on Nov 13, 2025

bonzini commented on Nov 13, 2025

64 remaining items

xevra commented on Jul 24, 2026

dnicolodi commented on Jul 31, 2026

lucascolley commented on Aug 13, 2026

dnicolodi commented on Aug 14, 2026

lucascolley commented on Aug 14, 2026

dnicolodi commented on Aug 14, 2026

lucascolley commented on Aug 14, 2026

dnicolodi commented on Aug 14, 2026

lucascolley commented on Aug 15, 2026

dnicolodi commented on Aug 15, 2026

lucascolley commented on Aug 15, 2026

dnicolodi commented on Aug 15, 2026

xevra commented on Aug 15, 2026

dnicolodi commented on Aug 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions