Add dfx_udfs: the library that cannot be installed as a bundle

Second of three libraries for #1719, and the one carrying the mixed-workflow
case: it exposes no `__datafusion_session_components__`, so callers register
its three functions and install its two codecs by hand.

That is not an artificial handicap. `SessionExtensionComponents` carries codec
fields only, so a function library has nowhere to put its functions — the
rename in 5a1bfeb noted that UDF and provider fields will join later. Until
they do, this is what a function library actually looks like, and the example
should show what that costs rather than pretend every dependency has caught
up. A test asserts the shape rather than describing it: `with_extensions`
rejects this object, naming the hook it lacks.

The functions are `dfx_net_revenue` (the TPC-H revenue expression),
`dfx_weighted_avg`, and `dfx_revenue_rank`. The aggregate is written out rather
than delegating to a built-in because its state is the point: two running sums,
which is what lets DataFusion compute a partial aggregate per partition and
merge the results. An aggregate that could only be evaluated over its whole
input at once would give a different answer once split, which is exactly what a
worker does to it. A test pins that by checking the plan really is
`mode=Partial` and the answer is still right.

Both codecs are name-only — `try_encode_*` writes nothing and `try_decode_*`
rebuilds from `name`. Three worker tests, each a separate interpreter, pin what
that buys, and they disagree with each other in the useful way:

  codec installed, nothing registered   -> works, decode_calls == 1
  functions registered, no codec        -> works, decode_calls == 0
  neither                               -> fails, naming dfx_net_revenue

So installing the codec is an *alternative* to registering the functions, not
an addition to it. The middle case is the trap worth knowing: on the driver,
where the functions are registered, the registry is tried first and the codec
is never consulted — so a codec that was broken or missing looks fine right up
until a worker needs it.

`try_decode_*` checks the name before the buffer, in that order. An empty
encoding leaves `fun_definition` unset and carries no codec id, so it is the
one path where a payload is offered to every installed codec in turn; a codec
that trusted `buf` first would answer for names it does not own. There are two
codecs because there are two plan layers and a library cannot know which one
its callers will serialize — an engine shipping physical plans exercises only
the physical one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: timsaucer

Cargo.lock

@@ -33,6 +33,7 @@ members = [
   "examples/datafusion-ffi-example",
   "examples/datafusion-ffi-query-planner-example",
   "examples/distributed/storage-library",
+  "examples/distributed/udf-library",
 ]
 resolver = "3"
 

Cargo.toml

@@ -0,0 +1,51 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+[package]
+name = "dfx-udfs"
+version.workspace = true
+edition.workspace = true
+rust-version.workspace = true
+license.workspace = true
+description = "Example extension library: user defined functions plus the codecs that make them portable"
+homepage.workspace = true
+repository.workspace = true
+publish = false
+
+[dependencies]
+arrow = { workspace = true }
+arrow-schema = { workspace = true }
+datafusion = { workspace = true }
+datafusion-common = { workspace = true, default-features = false }
+datafusion-expr = { workspace = true }
+datafusion-ffi = { workspace = true }
+datafusion-functions-window = { workspace = true }
+datafusion-proto = { workspace = true }
+datafusion-python-util.workspace = true
+pyo3 = { workspace = true, features = [
+  "extension-module",
+  "abi3",
+  "abi3-py310",
+] }
+pyo3-log = { workspace = true }
+
+[build-dependencies]
+pyo3-build-config = { workspace = true }
+
+[lib]
+name = "dfx_udfs"
+crate-type = ["cdylib", "rlib"]

examples/distributed/udf-library/Cargo.toml

@@ -0,0 +1,20 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+fn main() {
+    pyo3_build_config::add_extension_module_link_args();
+}

examples/distributed/udf-library/build.rs

@@ -0,0 +1,32 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+[build-system]
+requires = ["maturin>=1.6,<2.0"]
+build-backend = "maturin"
+
+[project]
+name = "dfx_udfs"
+requires-python = ">=3.10"
+classifiers = [
+  "Programming Language :: Rust",
+  "Programming Language :: Python :: Implementation :: CPython",
+]
+dynamic = ["version"]
+
+[tool.maturin]
+features = ["pyo3/extension-module"]