OpenMage test harness, SOAP connect fix, checkout docs

.gitignore

@@ -5,3 +5,4 @@ dist/
 *.egg-info/
 __pycache__/
 .pytest_cache/
+dev/openmage/.env

CHANGELOG.rst

@@ -18,11 +18,22 @@ First release of the maintained fork of fulfilio/python-magento.
 * Add a test suite (run against a local stand-in for the Magento XML-RPC
   API) and GitHub Actions CI on Python 3.9 to 3.14. The tests confirm every
   documented API (``client.product`` etc.) is available (fulfilio#15).
-* README: fix typos and a broken indent in the examples, explain how to check that a
-  connection and credentials work (fulfilio#25), document the
+* README: fix typos and a broken indent in the examples, explain how to
+  check that a connection and credentials work (fulfilio#25), document the
   ``product.create`` data format (fulfilio#27), and explain that
   ``order.search`` needs the Fulfil.IO connector extension and is not in
   the stock Magento 1 API (fulfilio#26).
+* Add opt-in integration tests against a real OpenMage LTS 20.18 in Docker
+  (``dev/openmage``). They are skipped unless ``OPENMAGE_URL`` is set.
+* Fix ``protocol='soap'``: connecting failed with ``TypeNotFound: (Array,
+  http://schemas.xmlsoap.org/soap/encoding/)`` on Magento 1.4.2 and later
+  and on OpenMage, whose WSDL does not import the SOAP encoding schema.
+* README: document the cart/checkout call sequence, the valid
+  ``cart_customer.set`` modes (fulfilio#28), that ``cart_shipping.method``
+  needs a rate code from ``cart_shipping.list`` (fulfilio#29), and how to
+  place orders without the new-order email (fulfilio#32). All three were
+  checked against OpenMage; none is a library bug.
+* README: SOAP only works for calls without arguments; use XML-RPC.
 
 3.1 (2019-02-22)
 ----------------

README.rst

@@ -131,6 +131,65 @@ the attribute set is missing from ``data``, or a value has the wrong
 format. Look at an existing product with ``client.product.info(sku)`` to
 see the attribute names and formats your store expects.
 
+Placing an order through the cart API
+-------------------------------------
+
+The cart calls have to run in this order, all with the same store view.
+Without ``store_view`` the cart is created in the admin store (ID 0), and
+the order ends up there too.
+
+.. code-block:: python
+
+    store = 'default'   # store view code or ID
+    address = {
+        'firstname': 'Jane', 'lastname': 'Doe', 'street': '1 Main Street',
+        'city': 'Austin', 'region_id': '57', 'postcode': '73301',
+        'country_id': 'US', 'telephone': '5125550100',
+    }
+
+    quote = client.cart.create(store)
+    client.cart_product.add(quote, [{'sku': 'MUG-1', 'qty': 1}], store)
+    client.cart_customer.set(quote, {
+        'mode': 'guest',
+        'firstname': 'Jane', 'lastname': 'Doe',
+        'email': 'jane@example.com',
+    }, store)
+    client.cart_customer.addresses(quote, [
+        dict(address, mode='billing'),
+        dict(address, mode='shipping'),
+    ], store)
+
+    # Pick a rate code from the list, e.g. 'flatrate_flatrate'
+    rates = client.cart_shipping.list(quote, store)
+    client.cart_shipping.method(quote, rates[0]['code'], store)
+
+    client.cart_payment.method(quote, {'method': 'checkmo'}, store)
+    order_increment_id = client.cart.order(quote, store)
+
+Common errors:
+
+* **Setting the customer fails** (fulfilio#28). ``mode`` must be
+  ``'guest'``, ``'register'`` (also needs ``password``) or ``'customer'``
+  (needs ``customer_id`` of an existing customer). A missing mode gives
+  ``Fault 1045``; any other value, such as ``'admin'``, crashes inside
+  Magento (``Fault 1`` or an HTTP 500 on PHP 8).
+* **Fault 1062: 'Shipping method is not available'** (fulfilio#29).
+  ``cart_shipping.method`` takes the rate ``code`` string from
+  ``cart_shipping.list`` (``'flatrate_flatrate'``), not the carrier name,
+  title or a dictionary. ``Fault 1061`` means the shipping address has not
+  been set yet.
+
+Placing orders without sending the order email
+----------------------------------------------
+
+``cart.order`` cannot skip the new-order email (fulfilio#32): Magento
+always sends it when *System > Configuration > Sales Emails > Order >
+Enabled* is on for the store view. To place orders silently, create a
+store view for API orders, set that option to *No* for it, and pass that
+store view's code to every ``cart.*`` call. Note that the *Send Email*
+button on the admin order page checks the same setting, so it will not
+send emails for orders in that store view either.
+
 Searching orders
 ----------------
 
@@ -258,10 +317,35 @@ instead:
 
     client = magento.API(url, apiuser, apipass, protocol='soap')
 
+SOAP support is limited: suds cannot encode list or dictionary arguments
+for Magento's SOAP v1 API, so only calls without arguments (such as
+``client.magento.info()``) work over SOAP. Use XML-RPC for everything
+else.
+
 A minimal ``protocol='rest'`` client for the Magento 2 REST API is also
 included, but it only supports GET requests and needs ``requests``
 installed (the ``rest`` extra). For Magento 2 work, PyMagento is the better choice.
 
+Integration tests
+-----------------
+
+``uv run pytest -q`` runs offline against a stand-in for the XML-RPC API,
+and skips the tests in ``tests/integration``. Those run against a real
+OpenMage LTS that ``dev/openmage/bootstrap.sh`` sets up in Docker (PHP 8.3
+with Apache, plus MariaDB). The first run downloads and installs OpenMage
+and creates a test-only API user, a product, a customer, and the flat rate
+and check/money order methods; later runs reuse the install. The test
+credentials are in ``dev/openmage/.env.example``, which the script copies to
+``.env``.
+
+.. code-block:: sh
+
+    dev/openmage/bootstrap.sh
+    OPENMAGE_URL=http://localhost:18080/ uv run pytest -q -m openmage
+    docker compose -f dev/openmage/docker-compose.yml stop    # keeps the install
+    docker compose -f dev/openmage/docker-compose.yml start   # resume later
+    dev/openmage/bootstrap.sh --reset                          # wipe and reinstall
+
 License
 -------
 

dev/openmage/.env.example

@@ -0,0 +1,19 @@
+# TEST-ONLY credentials for the local throwaway OpenMage container.
+# Never reuse these anywhere real. bootstrap.sh copies this file to .env
+# (gitignored) on first run; edit .env, not this file, to change them.
+
+OPENMAGE_VERSION=20.18.0
+OPENMAGE_PORT=18080
+
+# SOAP/XML-RPC web services user created by bootstrap.sh (all resources).
+OPENMAGE_API_USER=pytest
+OPENMAGE_API_KEY=pytest-api-key-123
+
+# Magento admin login (http://localhost:18080/index.php/admin).
+OPENMAGE_ADMIN_USER=admin
+OPENMAGE_ADMIN_PASSWORD=admin-test-password-123
+
+DB_NAME=openmage
+DB_USER=openmage
+DB_PASSWORD=openmage
+DB_ROOT_PASSWORD=openmage-root

dev/openmage/Dockerfile

@@ -0,0 +1,47 @@
+# Minimal PHP + Apache image for a throwaway OpenMage LTS test install.
+#
+# The OpenMage source itself is not baked in: bootstrap.sh downloads the pinned
+# release into the "openmage-src" volume on first run, so the image stays small
+# and the install survives image rebuilds.
+FROM php:8.3-apache
+
+# git/patch/unzip are needed by composer (cweagans/composer-patches applies
+# OpenMage's vendor patches with git or patch).
+RUN apt-get update \
+    && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
+        git patch unzip \
+        libfreetype6-dev libjpeg62-turbo-dev libpng-dev libicu-dev libxml2-dev libzip-dev \
+    && docker-php-ext-configure gd --with-freetype --with-jpeg \
+    && docker-php-ext-install -j"$(nproc)" ftp gd intl mysqli pdo_mysql soap zip \
+    && rm -rf /var/lib/apt/lists/* /tmp/pear
+
+ARG COMPOSER_VERSION=2.10.3
+RUN curl -fsSL "https://getcomposer.org/download/${COMPOSER_VERSION}/composer.phar" -o /usr/local/bin/composer \
+    && chmod +x /usr/local/bin/composer
+
+RUN { \
+        echo 'memory_limit = 768M'; \
+        echo 'max_execution_time = 300'; \
+        echo 'date.timezone = UTC'; \
+        echo 'display_errors = Off'; \
+        echo 'log_errors = On'; \
+        echo 'opcache.validate_timestamps = 1'; \
+        echo 'opcache.revalidate_freq = 2'; \
+    } > /usr/local/etc/php/conf.d/zz-openmage.ini
+
+# Apache listens on the same port inside the container as on the host. The
+# SOAP v1 server fetches its own WSDL from the store base URL
+# (http://localhost:$OPENMAGE_PORT/...), so that URL has to resolve from inside
+# the container as well as from the host.
+RUN a2enmod rewrite headers \
+    && sed -ri 's/^Listen 80$/Listen ${OPENMAGE_PORT}/' /etc/apache2/ports.conf \
+    && sed -ri 's/<VirtualHost \*:80>/<VirtualHost *:${OPENMAGE_PORT}>/' /etc/apache2/sites-available/000-default.conf \
+    && printf '<Directory /var/www/html>\n    AllowOverride All\n</Directory>\n' > /etc/apache2/conf-enabled/openmage.conf
+
+ENV OPENMAGE_PORT=18080 \
+    COMPOSER_HOME=/tmp/composer \
+    COMPOSER_ALLOW_SUPERUSER=1
+
+# Harness scripts are copied in rather than bind-mounted so the setup works
+# with Docker VMs that do not share the checkout's path.
+COPY scripts/ /opt/harness/

dev/openmage/bootstrap.sh

@@ -0,0 +1,97 @@
+#!/usr/bin/env bash
+# Bring up a local OpenMage LTS install for the integration tests.
+#
+#   ./bootstrap.sh          start (and on first run download, install, seed)
+#   ./bootstrap.sh --reset  wipe the containers AND volumes, then start fresh
+#
+# Every step checks what is already there, so re-running on an existing install
+# only starts the containers and re-applies the seed data; it never reinstalls.
+# Stop with "docker compose stop" (keeps the install), not "down -v".
+set -euo pipefail
+cd "$(dirname "${BASH_SOURCE[0]}")"
+
+if [[ ! -f .env ]]; then
+    cp .env.example .env
+    echo "Created .env from .env.example"
+fi
+set -a
+# shellcheck disable=SC1091
+source .env
+set +a
+
+OPENMAGE_VERSION=${OPENMAGE_VERSION:-20.18.0}
+OPENMAGE_PORT=${OPENMAGE_PORT:-18080}
+BASE_URL="http://localhost:${OPENMAGE_PORT}/"
+
+if [[ "${1:-}" == "--reset" ]]; then
+    echo "Removing containers and volumes..."
+    docker compose down --volumes --remove-orphans
+fi
+
+echo "Starting containers..."
+docker compose up -d --build --wait
+
+# Run a command in the web container as the Apache user.
+web() { docker compose exec -T -u www-data -w /var/www/html web "$@"; }
+
+if ! web test -f app/Mage.php || ! web test -f vendor/autoload.php; then
+    echo "Fetching OpenMage ${OPENMAGE_VERSION}..."
+    web bash /opt/harness/fetch-source.sh "${OPENMAGE_VERSION}"
+else
+    echo "OpenMage source already present: $(web php -r 'require "app/Mage.php"; echo Mage::getOpenMageVersion();')"
+fi
+
+if ! web test -f app/etc/local.xml; then
+    echo "Installing OpenMage..."
+    web php install.php \
+        --license_agreement_accepted yes \
+        --locale en_US \
+        --timezone UTC \
+        --default_currency USD \
+        --db_host db \
+        --db_name "${DB_NAME}" \
+        --db_user "${DB_USER}" \
+        --db_pass "${DB_PASSWORD}" \
+        --url "${BASE_URL}" \
+        --secure_base_url "${BASE_URL}" \
+        --use_rewrites yes \
+        --use_secure no \
+        --use_secure_admin no \
+        --skip_url_validation yes \
+        --admin_firstname Test \
+        --admin_lastname Admin \
+        --admin_email admin@example.com \
+        --admin_username "${OPENMAGE_ADMIN_USER}" \
+        --admin_password "${OPENMAGE_ADMIN_PASSWORD}"
+else
+    echo "OpenMage already installed (app/etc/local.xml exists)."
+fi
+
+echo "Seeding API user, catalogue, customer and checkout config..."
+docker compose exec -T -u www-data -w /var/www/html \
+    -e OPENMAGE_API_USER="${OPENMAGE_API_USER}" \
+    -e OPENMAGE_API_KEY="${OPENMAGE_API_KEY}" \
+    web php /opt/harness/seed.php
+
+echo "Checking the XML-RPC endpoint..."
+for _ in $(seq 1 20); do
+    if curl -fsS -o /dev/null -H 'Content-Type: text/xml' \
+        --data '<?xml version="1.0"?><methodCall><methodName>startSession</methodName></methodCall>' \
+        "${BASE_URL}index.php/api/xmlrpc"; then
+        break
+    fi
+    sleep 1
+done
+
+cat <<EOF
+
+OpenMage is up.
+  Storefront : ${BASE_URL}
+  Admin      : ${BASE_URL}index.php/admin  (${OPENMAGE_ADMIN_USER} / see .env)
+  API user   : ${OPENMAGE_API_USER} / see .env
+
+Run the integration tests from the repository root:
+  OPENMAGE_URL=${BASE_URL} uv run pytest -q -m openmage
+
+Stop (keeps the install):  docker compose -f dev/openmage/docker-compose.yml stop
+EOF

dev/openmage/docker-compose.yml

@@ -0,0 +1,43 @@
+# Throwaway OpenMage LTS install for the opt-in integration tests.
+# Use ./bootstrap.sh rather than calling "docker compose up" directly the first
+# time: it also downloads OpenMage, installs it and seeds the test data.
+name: python-magento-openmage
+
+services:
+  db:
+    image: mariadb:10.11
+    environment:
+      MARIADB_DATABASE: ${DB_NAME:-openmage}
+      MARIADB_USER: ${DB_USER:-openmage}
+      MARIADB_PASSWORD: ${DB_PASSWORD:-openmage}
+      MARIADB_ROOT_PASSWORD: ${DB_ROOT_PASSWORD:-openmage-root}
+    command:
+      - --innodb-buffer-pool-size=128M
+      - --max-connections=50
+      - --performance-schema=OFF
+    volumes:
+      - db:/var/lib/mysql
+    healthcheck:
+      test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
+      interval: 5s
+      timeout: 5s
+      retries: 30
+    restart: "no"
+
+  web:
+    build: .
+    image: python-magento-openmage-web:php8.3
+    depends_on:
+      db:
+        condition: service_healthy
+    environment:
+      OPENMAGE_PORT: ${OPENMAGE_PORT:-18080}
+    ports:
+      - "127.0.0.1:${OPENMAGE_PORT:-18080}:${OPENMAGE_PORT:-18080}"
+    volumes:
+      - src:/var/www/html
+    restart: "no"
+
+volumes:
+  db:
+  src:

dev/openmage/scripts/fetch-source.sh

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+# Runs inside the web container (as www-data, in /var/www/html).
+# Downloads the pinned OpenMage release and installs its composer dependencies.
+# Resumable: skips the download when the source is already unpacked.
+set -euo pipefail
+version="$1"
+cd /var/www/html
+
+if [[ ! -f app/Mage.php ]]; then
+    url="https://codeload.github.com/OpenMage/magento-lts/tar.gz/refs/tags/v${version}"
+    echo "Downloading ${url}"
+    curl -fsSL "$url" -o /tmp/openmage.tar.gz
+    tar -xzf /tmp/openmage.tar.gz --strip-components=1 -C /var/www/html
+    rm -f /tmp/openmage.tar.gz
+fi
+
+if [[ ! -f vendor/autoload.php ]]; then
+    composer install --no-dev --no-interaction --no-progress --prefer-dist
+fi
+
+mkdir -p var/cache var/log var/locks var/session media