Add Python dict support using VUnit dict_t

README.md

@@ -12,7 +12,7 @@ the selected simulator by a foreign language interface the package builds itself
 called through VHPIDIRECT (NVC, GHDL) or the FLI (Questa/ModelSim), or a VHPI application built with
 the simulator's own compiler driver (Riviera-PRO/Active-HDL). Values cross the interface with their
 VHDL types: `integer`, `real`, `string`, `boolean`, `std_ulogic`, `unsigned`/`signed`, the vector
-types, and `integer_array_t` as a NumPy array.
+types, `integer_array_t` as a NumPy array, and VUnit's `dict_t` as a Python `dict`.
 
 ## Installation
 

docs/user_guide.rst

@@ -271,6 +271,7 @@ bridge, ``eval`` has more result types:
 * ``eval_std_ulogic_vector``, returning an unconstrained ``std_ulogic_vector``.
 * ``eval_integer_array``, returning ``integer_array_t``, see
   :ref:`python_bridge:integer_array`.
+* ``eval_dict``, returning ``dict_t``, see :ref:`python_bridge:dict`.
 
 ``eval_boolean`` and ``eval_std_ulogic`` are aliased ``eval`` like the other
 result types. ``eval_std_ulogic_vector`` and ``eval_integer_array`` are not,
@@ -330,6 +331,7 @@ VHDL                       Python
 ``unsigned``               ``int``, passed as ``arg_unsigned``/``kwarg_unsigned``
 ``signed``                 ``int``, passed as ``arg_signed``/``kwarg_signed``
 ``integer_array_t``        NumPy array, not on Riviera-PRO/Active-HDL
+``dict_t``                 ``dict`` with ``str`` keys, see :ref:`python_bridge:dict`
 ========================== ========================================================
 
 An ``unsigned`` or ``signed`` value of any width becomes an exact Python
@@ -367,7 +369,7 @@ passed as a number or as the string of its characters:
 
 On NVC, GHDL and Questa, ``call`` returns the same types as ``eval``:
 ``call_boolean``, ``call_std_ulogic``, ``call_std_ulogic_vector``,
-``call_integer_array``, ``call_string``, ``call_real_vector`` and
+``call_integer_array``, ``call_dict``, ``call_string``, ``call_real_vector`` and
 ``call_integer_vector_ptr``, plus the procedures ``call_std_ulogic_vector``,
 ``call_signed`` and ``call_unsigned`` taking the result as an ``out``
 parameter. All of the functions but ``call_std_ulogic_vector`` are aliased
@@ -490,6 +492,8 @@ Type mapping
 * ``signed``/``unsigned`` ↔ ``int`` (procedure results only, Python bridge).
 * ``integer_array_t`` ↔ ``numpy.ndarray`` (Python bridge), see
   :ref:`python_bridge:integer_array`.
+* ``dict_t`` ↔ ``dict`` (results need the Python bridge), see
+  :ref:`python_bridge:dict`.
 
 The types marked as needing the Python bridge are available on NVC, GHDL and
 Questa, but not on Riviera-PRO/Active-HDL.
@@ -524,6 +528,44 @@ the failure.
 Output of ``print`` is written to the simulator output and flushed after
 every operation.
 
+.. _python_bridge:dict:
+
+dict_t and dict
+---------------
+
+A ``dict_t`` from VUnit's ``dict_pkg`` (string keys, values of mixed types) is
+a Python ``dict``. As an argument, ``arg(dict)`` and ``kwarg("name", dict)``
+write the ``dict`` as the Python literal ``{"key": value, ...}``, which works
+on all simulators. The values are ``integer``, ``real``, ``string``,
+``boolean``, ``std_ulogic``, ``integer_vector``, ``real_vector``,
+``integer_vector_ptr_t`` and nested ``dict_t``. Any other value type is
+reported like any other argument that cannot be converted. The keys are
+visited in the order of ``get_key``, which VUnit does not specify.
+
+.. code-block:: vhdl
+
+    constant cfg : dict_t := new_dict;
+    set_integer(cfg, "taps", 8);
+    set_real(cfg, "gain", 0.5);
+    call("model.configure", arg(cfg));
+
+``eval_dict`` and ``call_dict`` (aliased ``eval`` and ``call``) convert a
+Python ``dict`` with ``str`` keys to a new ``dict_t`` that the caller owns and
+deallocates. The values are converted strictly: ``int`` to ``integer``,
+``float`` to ``real``, ``str`` to ``string``, ``bool`` to ``boolean``, a
+``list`` of ``int`` to ``integer_vector_ptr_t`` and a ``dict`` to a nested
+``dict_t``, the last two stored with ``set_integer_vector_ptr_t_ref`` and
+``set_dict_t_ref``. ``None``, an ``int`` outside the VHDL integer range, a key
+that is not a ``str`` or any other value is an error, and ``eval_dict`` then
+returns an empty ``dict_t`` after reporting it. ``eval_dict`` and ``call_dict``
+are implemented by the Python bridge: Riviera-PRO/Active-HDL reports that they require NVC, GHDL or
+Questa.
+
+.. code-block:: vhdl
+
+    constant result : dict_t := eval("{'name': 'fir', 'taps': [1, 2, 1], 'gain': 0.5}");
+    check_equal(get_string(result, "name"), "fir");
+
 .. _python_bridge:integer_array:
 
 integer_array_t and NumPy
@@ -629,8 +671,8 @@ simulator installation change, so a run script needs nothing beyond
 
 This application differs from the Python bridge in a few ways: only the
 default session exists, the operations implemented by the bridge
-(``integer_array_t`` values, the ``boolean``, ``std_ulogic``, vector and
-``integer_array_t`` results, ``exec_file``) report that they require NVC, GHDL
+(``integer_array_t`` values, the ``boolean``, ``std_ulogic``, vector,
+``integer_array_t`` and ``dict_t`` results, ``exec_file``) report that they require NVC, GHDL
 or Questa, a Python error stops the simulation with the message printed by the
 application rather than through the logger of the session, and ``real`` values
 outside the single precision float range are rejected.

src/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd

@@ -121,6 +121,7 @@ package python_ffi_pkg is
   constant p_kind_integer_array : integer := 8;
   constant p_kind_integer_vector : integer := 9;
   constant p_kind_real_vector : integer := 10;
+  constant p_kind_dict : integer := 11;
 
   -- Names of the operations, used in the error messages
   impure function p_exec_operation(session : python_session_t := default_session) return string;

src/vunit_python_bridge/hdl/src/python_pkg.vhd

@@ -13,6 +13,9 @@ use vunit_lib.run_pkg.all;
 use vunit_lib.runner_pkg.all;
 use vunit_lib.integer_vector_ptr_pkg.all;
 use vunit_lib.string_ops.all;
+use vunit_lib.dict_pkg.all;
+use vunit_lib.dict_2008p_pkg.all;
+use vunit_lib.data_types_private_pkg.all;
 
 use std.textio.all;
 
@@ -78,6 +81,11 @@ package python_pkg is
   --
   -- H and L are read as 1 and 0. Any other metavalue is an error.
   --
+  -- A dict_t value becomes a Python dict, {"key": value, ...}, whose values
+  -- are written the same way: integer, real, string, boolean, std_ulogic,
+  -- integer_vector, real_vector, integer_vector_ptr_t and dict_t values.
+  -- Any other value type is an error.
+  --
   -- An integer_array_t value is transferred to Python by the Python bridge,
   -- which is only available for NVC, GHDL and Questa, and is referred to by
   -- the expression, which means that it can be used in several calls.
@@ -93,6 +101,8 @@ package python_pkg is
   impure function kwarg_signed(kw : string; value : signed) return arg_t;
   impure function arg(value : integer_array_t) return arg_t;
   impure function kwarg(kw : string; value : integer_array_t) return arg_t;
+  impure function arg(value : dict_t) return arg_t;
+  impure function kwarg(kw : string; value : dict_t) return arg_t;
 
   -- The Python expression calling identifier with the given arguments, for
   -- example to embed a call in a larger exec or eval string:
@@ -164,6 +174,10 @@ package python_pkg is
   -----------------------------------------------------------------------------
   -- Results of eval: boolean, std_ulogic, vectors and arrays
   -----------------------------------------------------------------------------
+  -- The result of eval_dict and call_dict is a new dict_t that the caller owns
+  -- and deallocates. Its values are integer, real, string, boolean,
+  -- integer_vector_ptr_t (a list of int) and dict_t (a nested dict).
+  --
   -- std_ulogic_vector and integer_array_t results are only available under
   -- their explicit names, not as eval overloads, which keeps
   -- check_equal(eval("17"), 17) and length(eval("[1, 2]")) unambiguous.
@@ -187,6 +201,11 @@ package python_pkg is
     expr : string; session : python_session_t := default_session
   ) return integer_array_t;
 
+  impure function eval_dict(
+    expr : string; session : python_session_t := default_session
+  ) return dict_t;
+  alias eval is eval_dict[string, python_session_t return dict_t];
+
   procedure eval_std_ulogic_vector(
     expr : string; result : out std_ulogic_vector; session : python_session_t := default_session
   );
@@ -247,6 +266,13 @@ package python_pkg is
   alias call is call_integer_array[
     string, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, python_session_t return integer_array_t];
 
+  impure function call_dict(
+    identifier : string; arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg;
+    session : python_session_t := default_session
+  ) return dict_t;
+  alias call is call_dict[
+    string, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, python_session_t return dict_t];
+
   procedure call_std_ulogic_vector(
     identifier : string; result : out std_ulogic_vector;
     arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg;
@@ -601,6 +627,44 @@ package body python_pkg is
     return "__vunit__.staged(" & integer'image(staged_id) & ")";
   end;
 
+  impure function p_arg_value(value : dict_t; operation : string) return string;
+
+  -- The Python source text of the value stored for a key of a dict
+  impure function p_dict_item(value : dict_t; key, operation : string) return string is
+  begin
+    case get_value_type(value, key) is
+      when vhdl_integer => return to_string(get_integer(value, key));
+      when vhdl_real => return to_string(get_real(value, key), "%.16e");
+      when vhdl_string => return p_quoted(get_string(value, key));
+      when vhdl_boolean => return arg(get_boolean(value, key)).value;
+      when ieee_std_ulogic => return p_arg_value(get_std_ulogic(value, key), operation);
+      when vhdl_integer_vector => return to_py_list_str(get_integer_vector(value, key));
+      when vhdl_real_vector => return to_py_list_str(get_real_vector(value, key));
+      when vunit_integer_vector_ptr_t => return p_arg_value(get_integer_vector_ptr_t_ref(value, key));
+      when vunit_dict_t => return p_arg_value(get_dict_t_ref(value, key), operation);
+      when others =>
+        return p_failed_value(
+          operation & " cannot convert the " & to_string(get_value_type(value, key)) &
+          " value of the dict key """ & key & """"
+        );
+    end case;
+  end;
+
+  impure function p_arg_value(value : dict_t; operation : string) return string is
+    variable result : line;
+  begin
+    swrite(result, "{");
+    for idx in 0 to num_keys(value) - 1 loop
+      if idx > 0 then
+        swrite(result, ", ");
+      end if;
+      swrite(result, p_quoted(get_key(value, idx)) & ": " & p_dict_item(value, get_key(value, idx), operation));
+    end loop;
+    swrite(result, "}");
+
+    return result.all;
+  end;
+
   function arg(value : real_vector) return arg_t is
   begin
     return (p_positional_arg, p_arg_value(value));
@@ -661,6 +725,16 @@ package body python_pkg is
     return (kw, p_arg_value(value, "kwarg"));
   end;
 
+  impure function arg(value : dict_t) return arg_t is
+  begin
+    return (p_positional_arg, p_arg_value(value, "arg"));
+  end;
+
+  impure function kwarg(kw : string; value : dict_t) return arg_t is
+  begin
+    return (kw, p_arg_value(value, "kwarg"));
+  end;
+
   -----------------------------------------------------------------------------
   -- Argument groups
   -----------------------------------------------------------------------------
@@ -875,6 +949,102 @@ package body python_pkg is
     return result;
   end;
 
+  -- The integers of a comma separated list
+  function p_split_integers(text : string) return integer_vector is
+    alias items : string(1 to text'length) is text;
+    variable count : natural := 0;
+    variable first : natural := 1;
+    variable result : integer_vector(0 to text'length);
+  begin
+    if items'length = 0 then
+      return result(1 to 0);
+    end if;
+    for idx in items'range loop
+      if items(idx) = ',' then
+        result(count) := integer'value(items(first to idx - 1));
+        count := count + 1;
+        first := idx + 1;
+      end if;
+    end loop;
+    result(count) := integer'value(items(first to items'length));
+    return result(0 to count);
+  end;
+
+  impure function p_to_integer_vector_ptr(text : string) return integer_vector_ptr_t is
+    constant items : integer_vector := p_split_integers(text);
+    constant result : integer_vector_ptr_t := new_integer_vector_ptr(items'length);
+  begin
+    for idx in items'range loop
+      set(result, idx - items'left, items(idx));
+    end loop;
+    return result;
+  end;
+
+  -- The real of "hi,lo,exponent,sign", which is sign * (hi * 2**26 + lo) * 2**exponent. The 53 bit
+  -- integer is exact and the scaling by a power of two is split in two to avoid underflowing 2**exponent.
+  function p_to_real(text : string) return real is
+    constant parts : integer_vector := p_split_integers(text);
+    constant mantissa : real := real(parts(0)) * 67108864.0 + real(parts(1));
+    constant half : integer := parts(2) / 2;
+  begin
+    return real(parts(3)) * mantissa * 2.0 ** half * 2.0 ** (parts(2) - half);
+  end;
+
+  -- The dict_t of the records the dict_t result of the bridge is made of, see
+  -- _dict_entries of runtime.py
+  impure function p_to_dict(text : string) return dict_t is
+    alias records : string(1 to text'length) is text;
+    constant result : dict_t := new_dict;
+    variable pos : natural := 1;
+    variable tag : character;
+    variable key_start, key_length, payload_start, payload_length : natural;
+
+    -- The length in front of a colon, leaving pos after the colon
+    procedure read_length(length : out natural) is
+      variable colon : natural := pos;
+    begin
+      while records(colon) /= ':' loop
+        colon := colon + 1;
+      end loop;
+      length := natural'value(records(pos to colon - 1));
+      pos := colon + 1;
+    end;
+
+    procedure add(key, payload : string) is
+      alias value : string(1 to payload'length) is payload;
+      variable ints : integer_vector_ptr_t;
+      variable nested : dict_t;
+    begin
+      case tag is
+        when 'i' => set_integer(result, key, integer'value(value));
+        when 'r' => set_real(result, key, p_to_real(value));
+        when 'b' => set_boolean(result, key, value = "1");
+        when 's' => set_string(result, key, value);
+        when 'd' =>
+          nested := p_to_dict(value);
+          set_dict_t_ref(result, key, nested);
+        when others =>
+          ints := p_to_integer_vector_ptr(value);
+          set_integer_vector_ptr_t_ref(result, key, ints);
+      end case;
+    end;
+  begin
+    while pos <= records'length loop
+      tag := records(pos);
+      pos := pos + 1;
+      read_length(key_length);
+      key_start := pos;
+      pos := pos + key_length;
+      read_length(payload_length);
+      payload_start := pos;
+      pos := pos + payload_length;
+      add(records(key_start to key_start + key_length - 1),
+          records(payload_start to payload_start + payload_length - 1));
+    end loop;
+
+    return result;
+  end;
+
   -----------------------------------------------------------------------------
   -- exec
   -----------------------------------------------------------------------------
@@ -952,6 +1122,16 @@ package body python_pkg is
     return null_integer_array;
   end;
 
+  impure function eval_dict(
+    expr : string; session : python_session_t := default_session
+  ) return dict_t is
+  begin
+    if p_eval(expr, p_kind_dict, -1, p_eval_operation(expr, session), session) then
+      return p_to_dict(p_result_string);
+    end if;
+    return new_dict;
+  end;
+
   procedure eval_std_ulogic_vector(
     expr : string; result : out std_ulogic_vector; session : python_session_t := default_session
   ) is
@@ -1052,6 +1232,16 @@ package body python_pkg is
     );
   end;
 
+  impure function call_dict(
+    identifier : string; arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg;
+    session : python_session_t := default_session
+  ) return dict_t is
+  begin
+    return eval_dict(
+      to_call_str(identifier, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10), session
+    );
+  end;
+
   procedure call_std_ulogic_vector(
     identifier : string; result : out std_ulogic_vector;
     arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg;

src/vunit_python_bridge/hdl/src/python_pkg_vhpi.vhd

@@ -105,6 +105,7 @@ package python_ffi_pkg is
   constant p_kind_integer_array : integer := 8;
   constant p_kind_integer_vector : integer := 9;
   constant p_kind_real_vector : integer := 10;
+  constant p_kind_dict : integer := 11;
 
   -- Name of the operation, used in the error messages
   impure function p_eval_operation(expr : string; session : python_session_t := default_session) return string;