diff --git a/.gitignore b/.gitignore index 9d27c4ccb..9550e48c0 100644 --- a/.gitignore +++ b/.gitignore @@ -12,3 +12,4 @@ cabal.project.local~ *.tix .coverage +__pycache__/ diff --git a/protocol/simplex-messaging.md b/protocol/simplex-messaging.md index be7bd448f..f894f1047 100644 --- a/protocol/simplex-messaging.md +++ b/protocol/simplex-messaging.md @@ -86,7 +86,7 @@ It's designed with the focus on communication security and integrity, under the It is designed as a low level protocol for other application protocols to solve the problem of secure and private message transmission, making [MITM attack][1] very difficult at any part of the message transmission system. -This document describes SMP protocol version 20. Versions 1-5 are discontinued. The version history: +This document describes SMP protocol version 22. Versions 1-5 are discontinued. The version history: - v1: binary protocol encoding - v2: message flags (used to control notifications) @@ -108,6 +108,7 @@ This document describes SMP protocol version 20. Versions 1-5 are discontinued. - v19: service subscriptions to messages (SUBS, NSUBS, SOKS, ENDS, ALLS commands) - v20: public namespaces resolver (RSLV command, RNAME response) — direct or forwarded via PFWD - v21: server public information in handshake +- v22: `RNAME` says whether a name can be registered, not only what it resolves to ## Introduction @@ -1457,46 +1458,169 @@ while still returning a `NameRecord` matching the encoding below. #### Resolve name command -The `RSLV` command carries the canonical fully-qualified name directly as the -payload (not JSON): +From v22 the `RSLV` command carries a query; below v22 it carries the name +directly, as it always did (not JSON): ```abnf -rslv = %s"RSLV" SP domain ; domain = canonical name as non-space bytes, consuming the remainder of the transmission +rslv = %s"RSLV" SP (query / domain) ; query from v22, domain below it +query = tld label sub +tld = %s"s" / %s"t" / %s"w" ; .simplex / .testing / a web name +label = %s"N" shortString ; the second-level label as text + / %s"H" 32*32 OCTET ; its keccak-256 +sub = length *shortString ; subname labels, parent to child +domain = 1*253 OCTET ; the name as text ``` `domain` is the UTF-8 canonical fully-qualified name with the TLD always explicit (e.g. `privacy.simplex`, `test.testing`, `example.com`), bounded to 253 bytes. +**Hashed labels.** The query's second-level label is either the label itself or +the keccak-256 of it, tagged, so the two are told apart by the tag and never by +the shape of the value. + +Only the second-level label may be hashed: subname labels are needed as text to +reach the record, and a web TLD has no registry to key on. `sub..simplex` +reaches the node `sub.name.simplex` does. + +From v22 a client MUST send the hash. Older routers can only read the name, so a +client on an older session sends the name. A router answering a hashed query +does not know the label's length, so it cannot check a minimum-length policy +either: the client does that, from the pricing it is sent. + +The hash reaches the backing resolver as `[` + 64 lowercase hex + `]`, ENS's +encoding for a label whose text is unknown, because that is what its HTTP API +takes. That form appears nowhere in SMP. + +A hashed query still answers with the name. The registrar records the plaintext +label when a name is registered, keyed by the hash of that label, so a router can +look up what the hash stands for without ever being told. The router is not +trusted for it: a client MUST check that the record names the name it asked +about, and reject the answer otherwise. A name registered without that record +answers `unknown`, which fails that check. What stays impossible is learning a +name that is *not* registered: there is nothing recorded to look up, so a name +someone is merely considering never becomes known. + **Server-side validation.** The names router parses `domain` as a fully-qualified name (TLD required — bare labels are rejected) and forwards it to the configured backing resolver, which is the source of truth for which on-chain registry maps to each TLD. -The names router responds with either an `RNAME` response carrying the resolved -record, or an `ERR NAME` error whose subcode a client iterating across several -configured servers can act on distinctly: +The names router responds with either an `RNAME` response saying what it knows +about the name, or an `ERR NAME` error whose subcode a client iterating across +several configured servers can act on distinctly: | Response | Condition | Client action | |---|---|---| -| `RNAME` | record resolved | use it | -| `ERR NAME NOT_FOUND` | name not registered, unknown TLD, or malformed name | authoritative "no such name" — stop | +| `RNAME` | the router read the registry | use it | +| `ERR NAME NOT_FOUND` | the router could not read any answer for the name; below v22 also every name that does not resolve | stop, and do not read it as registrable | | `ERR NAME NO_RESOLVER` | this router has no resolver (names role not enabled) | skip this server, try the next | -| `ERR NAME RESOLVER ` | transient failure: backing resolver error (upstream 5xx, transport, timeout, decode) | transient — retry or surface, do not treat as "not found" | +| `ERR NAME RESOLVER ` | the router cannot state an answer completely: no registrar or price oracle for the TLD, an unreachable chain, a transport failure, a timeout, a registration it could not date or resolve | surface ``; retry only if it reads as transient | A client SHOULD NOT broadcast a `name` to further servers after a name-capable router has answered (`NOT_FOUND` or `RESOLVER`), since that router has already seen the lookup key; `NO_RESOLVER` discloses nothing about the name beyond the fact that this router cannot resolve, so iterating past it is safe. -#### Name record response +#### Name response -The `RNAME` response carries a JSON-encoded record as the payload: +Resolving a name and asking whether it can be registered are one lookup in the +registry, and `RNAME` answers both: a client offering to register a name that +turns out to be taken can show what took it. ```abnf -rname = %s"RNAME" SP json-bytes ; json-bytes consumes the remainder of the transmission +rname = %s"RNAME" SP registration +registration = %s"N" optTime optTime reserved SP json-bytes ; registered + / %s"A" optTime pricing ; available + / %s"R" reason ; reserved +optTime = %s"0" / (%s"1" 8*8 OCTET) ; Int64, big-endian, unix seconds +reserved = %s"0" / (%s"1" reason) ; absent = not held back +pricing = tiers basePrice minLabelLength +tiers = length *(2*2 OCTET 8*8 OCTET) ; label length -> US cents per year +basePrice = 8*8 OCTET ; US cents per year for every other length +minLabelLength = 2*2 OCTET ; characters +reason = %s"internal" / %s"trademark" / %s"community" / word +word = 1*32(%x21-7E) ; a reason this version has no word for ``` +On `N` the two `optTime` fields are the expiry and the end of the grace period, +in that order, and `json-bytes` is the record, consuming the remainder of the +transmission. On `A` the `optTime` is when a post-grace surcharge decays to +nothing. The reason words are the same on the wire, in the backing resolver's +JSON and in a client's own API. + +Money is US cents; the registry denominates in USD, never in ETH, and the +backing resolver converts before the value reaches the protocol. Times are +seconds since the Unix epoch. Lengths are characters. + +| Answer | Meaning | +|---|---| +| `N` | registered: held by someone until the expiry, renewable by its owner alone until the end of grace. It always carries a record: where the owner set none, every field is unset and the resolver address is zero | +| `A` | available: held by nobody and registrable now, at `pricing` | +| `R` | reserved: held back by the registry and not registered | + +`A` alone means registrable: a name the registry holds back answers `R` +instead, so a client has no flags to combine. A reservation on a name that *is* +registered is carried in the `reserved` field, and is why that name will not +free up when it expires. + +There is no separate answer for an auction. A name past its grace period answers +`A` with the ordinary price and the time its surcharge expires. The surcharge +itself is not carried: it decays continuously, so it cannot be quoted as a +purchase price. A client shows the ordinary price and counts down to when it +applies. + +A router MUST NOT quote a price for a reserved name: it is not for sale at the +registry's price, and quoting one would be an offer the registry will not +honour. That is why `R` has no pricing field. + +The record is carried while a name is registered and through its grace period, +and stops once the name is registrable by anyone. Keeping it through grace lets +whoever opens the name tell its owner that it is about to lapse; keeping it +longer would show a record whose owner no longer holds the name. How long a +client goes on opening an expiring name is its own decision. + +**Computing the price.** In US cents, for a duration in seconds: + +``` +price len duration = tier len * duration / 31536000 +tier len = the entry for len in tiers, or basePrice when len is not in tiers +``` + +The registry's minimum registration is 28 days, a contract constant rather than +a per-deployment value, so it is specified here rather than sent. `tiers` omits +any length below `minLabelLength`, those being unregistrable. `minLabelLength` +is sent because a hashed query carries no length: the router cannot check it, so +the client must, and a price quoted for a label the registry will refuse is +worse than no quote at all. + +Below v22, `RNAME` carries the bare record and nothing else, and every answer +without one is `ERR NAME NOT_FOUND`, as it was before this version. A name in +its grace period therefore resolves for those clients too, without the expiry +they have no field to carry. In the other direction a v22 client reads such an +answer as `N` with no expiry, grace or reservation, which is the only reason +those three fields are optional. + +From v22 a client MUST NOT read `ERR NAME NOT_FOUND` as "registrable": only `A` +says that. `NOT_FOUND` means the router has nothing to say about the name, which +includes a backing resolver whose answer it could not read. + +A router that cannot state an answer completely MUST say so as `ERR NAME +RESOLVER ` rather than answer partially. That covers a TLD with no +registrar or no price oracle configured, an unreachable chain, a timeout, a +registration it could not date, a registered name it could not resolve, and any +status word it does not recognise. Neither a registration nor availability may +be guessed: one would assert a registration nobody read, the other would offer a +name that may be held. + +A client MUST read a `reason` it does not know as unknown and still treat the +name as reserved: a later version may reserve names for reasons this one cannot +name, and losing the reservation over that would offer a name that cannot be +registered. The word itself travels unchanged, so a later client can act on it +and a current one can show or log it, which is why the set is open rather than +an enumeration. A router sends at most one token of printable ASCII, since the +field ends at a space. + `json-bytes` MUST be a UTF-8 JSON object with the following schema: | Field | JSON type | Constraints | @@ -1521,10 +1645,10 @@ an empty string, not JSON `null` and not an absent key. Link fields empty array `[]` when unset. Coin fields (`eth`, `btc`, `xmr`, `dot`) use JSON `null` as the "unset" sentinel and MAY also be absent from the object entirely. -The backing resolver filters records that are expired or otherwise unavailable -(the names router then returns `ERR NAME NOT_FOUND` to the client), so the wire -format carries no expiry field. Testnet-vs-mainnet status is derived from the -queried TLD rather than an in-record flag. +The record carries no expiry field of its own: `N` carries it alongside. +The backing resolver stops resolving a name once it is registrable by anyone, so +a record only ever accompanies `N`. Testnet-vs-mainnet status is derived +from the queried TLD rather than an in-record flag. Receivers MUST tolerate extra unknown fields (forward-compatibility for future field additions). Adding a required field is a breaking change requiring an diff --git a/scripts/resolver/README.md b/scripts/resolver/README.md index 97126c7be..9d8f0785a 100644 --- a/scripts/resolver/README.md +++ b/scripts/resolver/README.md @@ -123,9 +123,13 @@ uv run scripts/resolver/service/snrc-resolve.py # defaults to local reth + main "simplexChannel": [], "eth": null, "btc": "bc1q…", "xmr": "4ANz…", "dot": "139G…", "owner": "0xd83b…", "resolver": "0x80fa…", - "status": "registered", // registered | grace | expired | unregistered | reserved | noResolver | unknown + "status": "registered", // registered | grace | auction | expired | unregistered | reserved | noResolver | unknown "expires": 1780000000, // Unix seconds; when the registration ends - "graceEnds": 1787776000 // expires + GRACE_PERIOD; last moment the owner can renew + "graceEnds": 1787776000, // expires + GRACE_PERIOD; last moment the owner can renew + "auctionEnds": null, // when the premium reaches zero; only on `auction` + "premium": null, // decimal string, attoUSD; only on `auction` + "reasonCode": null, // only on `reserved` + "reason": null // only on `reserved` } ``` @@ -147,9 +151,10 @@ name already knows when it expires. Both timestamps are Unix seconds, and |---|---| | `registered` | live; `expires` is when that ends | | `grace` | lapsed, but only the previous owner may renew it, until `graceEnds` | -| `expired` | lapsed and past grace — anyone may register it now | +| `auction` | past grace, so anyone may register it — but at a premium, until `auctionEnds` | +| `expired` | lapsed, past grace, and past the auction — anyone may register it at the ordinary price | | `unregistered` | never registered, and free to take | -| `reserved` | not registered, and held back — registration will be refused; the body carries a `reason` | +| `reserved` | not registered, and held back — registration will be refused; the body carries `reasonCode` and `reason` | | `noResolver` | registered, but points nowhere | | `unknown` | no `SNRC_REGISTRAR_` configured, so status could not be read | @@ -165,6 +170,49 @@ released*. A subname reports the status of the 2LD above it, which is only as good as the name it sits under. +### The post-grace auction + +When grace ends anyone may register the name, but the price oracle adds a +premium that halves each day until it reaches zero. A name in that window +reports `auction` instead of `expired`, with `premium` (attoUSD as a decimal +string, since no JSON number holds a 256-bit integer) and `auctionEnds`. + +`premium` is the surcharge alone: it depends only on when the registration +lapsed, so a labelhash query gets it, but the base price depends on the label's +length, which a hash does not carry. The client adds that. + +The oracle comes from the controller's `prices()`, so no extra configuration is +needed. Its window is read from the chain; zero days switches the auction off. +Deployment constants - the grace period, the oracle and its curve - are cached +for `CONSTANTS_TTL` (5 minutes), so a retune shows up within that. Per-name +values and the decaying premium are read on every query. + +**Upgrade this service before the routers that query it.** An older resolver +reports a name in its auction as plain `expired`, which routers read as +"available at the ordinary price" while the registrar charges the premium. It +also fails to decode a bracket label under a subname (`sub.[].tld`), which +routers from v22 send. The same wrong quote happens when the auction cannot be +read at all, so set `SNRC_CONTROLLER_` wherever `SNRC_REGISTRAR_` is. + +### Why a name is reserved + +`reserved` carries `reasonCode`, the controller's reason, and `reason`, an +English sentence for a human reading this API. Clients should branch on +`reasonCode` and word it themselves, in the user's language. + +| `reasonCode` | Meaning | +|---|---| +| `unspecified` | reserved, with no reason recorded on chain | +| `trademark` | reserved to protect a trademark | +| `publicInterest` | reserved in the public interest | +| `offensive` | reserved as an offensive name | +| `internal` | reserved for SimpleX | +| `premium` | reserved as a premium name | +| `unknown` | a reason added to the contract after this resolver; still reserved | + +A controller from before reasons existed stores a boolean; its `true` reads as +`unspecified`, so nothing needs migrating. + ### Querying by labelhash A client asking whether a name is free is usually about to register it, and @@ -181,10 +229,15 @@ returns the same record. The registrar keys `nameExpires` and `reservedNames` on the labelhash too, so the status fields do not need the label either. The resolver learns the name only by guessing the label and hashing it. -Read the answer from `status`. A name is free only when the body says -`unregistered`, which comes with a 404. Every other status means somebody holds -the name or held it recently. Watch out for `noResolver`: it is also a 404, but -the name is taken. +Only the second-level label is a registry key, and it is decoded wherever it +sits: `sub.[].testing` reaches the node `sub.name.testing` does. Subname +labels stay text; a bracket label left of the 2LD is an ordinary label. Routers +from v22 send every 2LD this way, so a registrable name normally never reaches +this service. + +Read the answer from `status`. A name is free on `unregistered` (404), and on +`expired` or `auction` (410) — `auction` costs a premium on top. Every other +status means somebody holds the name. Watch `noResolver`: also a 404, but taken. The hash must be keccak-256. `openssl dgst -sha3-256` and `sha3sum` compute SHA3-256, a different function that returns 64 valid-looking hex characters @@ -222,13 +275,16 @@ which is free to change. ``` The codes are `tldNotConfigured`, `notFullyQualified`, `unregistered`, -`reserved`, `grace`, `expired`, `noResolver`, `noSuchRoute` and +`reserved`, `grace`, `auction`, `expired`, `noResolver`, `noSuchRoute` and `upstreamError`. When the registration is what went wrong, `error` and `status` hold the same value, so one field is enough to read. `upstreamError` says only which exception type the RPC call raised. The text goes to the resolver's log instead, because `SNRC_RPC` can carry a provider key -and urlopen puts the URL it failed on into the message. +and urlopen puts the URL it failed on into the message. It is also the answer +when a registrar, controller or oracle address has no contract behind it: the +empty reply is refused rather than read as zero, which would make every name +look free. ### Status codes @@ -237,7 +293,7 @@ and urlopen puts the URL it failed on into the message. | 200 | resolved (`status` is `registered`, or `unknown` when no registrar is configured) | | 400 | TLD not configured, or not a fully-qualified name | | 404 | `unregistered`, `reserved` or `noResolver` — the `status` field says which | -| 410 | registration lapsed — `status` says whether the owner can still renew (`grace`) or anyone may take it (`expired`) | +| 410 | registration lapsed — `status` says whether the owner can still renew (`grace`), anyone may take it at a premium (`auction`), or anyone may take it at the ordinary price (`expired`) | | 502 | upstream RPC error / reth not synced | ### Configuring addresses diff --git a/scripts/resolver/service/snrc-resolve.py b/scripts/resolver/service/snrc-resolve.py index cdadc1f48..8ffa6a1e5 100755 --- a/scripts/resolver/service/snrc-resolve.py +++ b/scripts/resolver/service/snrc-resolve.py @@ -41,7 +41,8 @@ (default: empty — TLD not yet deployed) SNRC_REGISTRAR_ BaseRegistrar (ERC-721) for the TLD; expiry and status (default: mainnet for .testing, empty for .simplex) - SNRC_CONTROLLER_ SimplexController (proxy) for the TLD; `reserved` status + SNRC_CONTROLLER_ SimplexController (proxy) for the TLD; reservations, + and through its `prices()` oracle what registering costs (default: mainnet for .testing, empty for .simplex) SNRC_PORT Listen port (default: 8000) SNRC_BIND Bind address (default: 0.0.0.0) @@ -66,6 +67,7 @@ import json import os import sys +import time from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from urllib.parse import unquote, urlparse from urllib.request import Request, urlopen @@ -103,8 +105,16 @@ "simplex": os.environ.get("SNRC_CONTROLLER_SIMPLEX", ""), # not deployed yet } -# `reservedNames` stores the fact only, never a reason. -RESERVED_REASON = "reserved for a brand or public interest" +# `reservedNames` holds a SimplexController.Reason; 0 means not reserved. A +# controller from before the enum stores a bool, whose `true` decodes as 1, +# which is why 1 reads as "internal". +RESERVED_REASONS = { + 1: ("internal", "reserved for SimpleX"), + 2: ("trademark", "reserved to protect a trademark"), + 3: ("community", "reserved for the community"), +} +# a Reason added to the contract after this resolver: still reserved, unworded +UNKNOWN_REASON = ("unknown", "reserved") # SLIP-44 coin types (https://github.com/satoshilabs/slips/blob/master/slip-0044.md) COIN_ETH = 60 @@ -114,6 +124,8 @@ ZERO_ADDR = "0x0000000000000000000000000000000000000000" +# The registry prices in attoUSD (1e-18 USD); the protocol carries US cents. + # ---------- RPC + ABI helpers (mirrors ens-lookup.py shape) ---------- @@ -160,12 +172,16 @@ def is_encoded_labelhash(label: str) -> bool: def node_of(name: str) -> bytes: - """namehash, accepting an encoded labelhash in place of a 2LD's label. In a - subname a bracket label is hashed as written, not decoded.""" + """namehash, decoding the 2LD's label as a labelhash wherever it sits, so + `[hash].tld` and `sub.[hash].tld` reach the nodes their names do. A bracket + label anywhere else is hashed as written.""" labels = name.split(".") - if len(labels) == 2 and is_encoded_labelhash(labels[0]): - return keccak(namehash(labels[1]) + bytes.fromhex(labels[0][1:-1])) - return namehash(name) + if len(labels) < 2 or not is_encoded_labelhash(labels[-2]): + return namehash(name) + node = keccak(namehash(labels[-1]) + bytes.fromhex(labels[-2][1:-1])) + for label in reversed(labels[:-2]): + node = keccak(node + keccak(label.encode())) + return node # ---------- Registration status ---------- @@ -177,9 +193,26 @@ def chain_now() -> int: return decode_uint(block["timestamp"]) +# Deployment constants - the grace period, the oracle and its curve - change only +# when the owner retunes a contract, so they are read once per TTL rather than on +# every query. Per-name values and the decaying premium are never cached. +CONSTANTS_TTL = 300 +_constants: dict = {} + + +def cached(key, read): + """`read()` at most once per CONSTANTS_TTL for `key`.""" + hit = _constants.get(key) + if hit and time.time() - hit[0] < CONSTANTS_TTL: + return hit[1] + value = read() + _constants[key] = (time.time(), value) + return value + + def grace_period(registrar: str) -> int: """A deployment can configure a different window, so it is read on chain.""" - return decode_uint(eth_call(registrar, selector("GRACE_PERIOD()"))) + return cached(("grace", registrar), lambda: decode_uint(eth_call(registrar, selector("GRACE_PERIOD()")))) def expiry_status(expires: int, grace: int, now: int) -> str: @@ -194,12 +227,97 @@ def expiry_status(expires: int, grace: int, now: int) -> str: return "expired" -def is_reserved(tld: str, token: int) -> bool: +def reservation_reason(tld: str, token: int) -> int: + """The SimplexController.Reason held for the name, 0 when not reserved.""" controller = CONTROLLERS.get(tld) if not controller: - return False + return 0 raw = eth_call(controller, selector("reservedNames(bytes32)") + encode_uint(token)) - return decode_uint(raw) != 0 + return decode_uint(raw) + + +def pricing_params(tld: str): + """What it costs to register a name under this TLD, in US cents, or None + when no controller or price oracle is configured.""" + return cached(("pricing", tld), lambda: read_pricing_params(tld)) + + +def read_pricing_params(tld: str): + controller = CONTROLLERS.get(tld) + if not controller: + return None + oracle = decode_address(eth_call(controller, selector("prices()"))) + if oracle == ZERO_ADDR: + return None + try: + return read_oracle_prices(controller, oracle) + except RuntimeError: + # An oracle that does not expose its curve cannot be quoted from. The + # name is still registrable; the price is simply not ours to state. + return None + + +SECONDS_PER_YEAR = 31536000 +ATTO_PER_CENT = 10**16 + + +def read_oracle_prices(controller: str, oracle: str): + """SimplexPriceOracle keeps the curve in US cents per year, the unit the SMP + protocol carries. An ENS-shaped oracle prices in attoUSD per second and + charges a premium on lapsed names that it does not expose, so a quote from + it is only safe for a name that was never registered.""" + try: + base, tiers = decode_prices(eth_call(oracle, selector("prices()"))) + premium_unknown = False + except RuntimeError: + base, tiers = decode_letter_prices(oracle) + premium_unknown = True + min_len = decode_uint(eth_call(controller, selector("minCharLength()"))) + return { + # lengths the registry refuses are left out rather than priced at zero + "rentPrices": {n: c for n, c in tiers.items() if n >= min_len}, + "basePrice": base, + "minLabelLength": min_len, + "_premiumUnknown": premium_unknown, + } + + +def decode_letter_prices(oracle: str): + """`price1Letter()`..`price6Letter()`, in attoUSD per second. Quotes round + up, so one is never below what the registry charges. An oracle built before + the six-letter tier stops at five, and charges its highest tier for anything + longer, which is what basePrice means here.""" + tiers = {} + for n in range(1, 7): + try: + rate = decode_uint(eth_call(oracle, selector(f"price{n}Letter()"))) + except RuntimeError: + if n <= 5: + raise + break + tiers[n] = ceil_div(rate * SECONDS_PER_YEAR, ATTO_PER_CENT) + return tiers.pop(max(tiers)), tiers + + +def ceil_div(a: int, b: int) -> int: + return -(-a // b) + + +def decode_prices(hex_data: str): + """`prices()` returns the base price and the lengths priced differently.""" + raw = bytes.fromhex(hex_data[2:] if hex_data.startswith("0x") else hex_data) + # a short answer is not a curve: decoding it would quote every name as free + if len(raw) < 96: + raise RuntimeError("prices(): short response") + base = int.from_bytes(raw[:32], "big") + at = int.from_bytes(raw[32:64], "big") + count = int.from_bytes(raw[at:at + 32], "big") + tiers = {} + for i in range(count): + item = at + 32 + i * 64 + length = int.from_bytes(raw[item:item + 32], "big") + tiers[length] = int.from_bytes(raw[item + 32:item + 64], "big") + return base, tiers def name_status(name: str): @@ -207,32 +325,49 @@ def name_status(name: str): tld = labels[-1] registrar = REGISTRARS.get(tld) if not registrar or len(labels) < 2: - return {"status": "unknown", "expires": None, "graceEnds": None} + return { + "status": "unknown", + "expires": None, + "graceEnds": None, + "reasonCode": None, + "reason": None, + "auctionUntil": None, + } # nameExpires and reservedNames are keyed on uint256(keccak(label)). - # Decoded for a 2LD only, the same rule node_of applies to the node. - label = labels[-2] - if len(labels) == 2 and is_encoded_labelhash(label): - token = int(label[1:-1], 16) - else: - token = int.from_bytes(keccak(label.encode()), "big") + # Only the 2LD's label is a registry key, wherever it sits - the same rule + # node_of applies to the node. + token = label_token(labels[-2]) expires = decode_uint( eth_call(registrar, selector("nameExpires(uint256)") + encode_uint(token)) ) if expires == 0: - status, grace = "unregistered", 0 + status, grace, now = "unregistered", 0, 0 else: grace = grace_period(registrar) - status = expiry_status(expires, grace, chain_now()) + now = chain_now() + status = expiry_status(expires, grace, now) - if status in ("unregistered", "expired") and is_reserved(tld, token): - status = "reserved" + # A reservation is orthogonal to the registration: a registered name can be + # held back too. + code = reservation_reason(tld, token) + reason = RESERVED_REASONS.get(code, UNKNOWN_REASON) if code else None - return { + out = { "status": status, "expires": expires or None, "graceEnds": (expires + grace) if expires else None, + "reasonCode": reason[0] if reason else None, + "reason": reason[1] if reason else None, + "auctionUntil": None, } + if status in ("unregistered", "expired"): + pricing = pricing_params(tld) + # a lapsed name may carry a premium this resolver cannot read, and a + # quote without it would be below what the registry charges + if pricing and not (status == "expired" and pricing["_premiumUnknown"]): + out.update({k: v for k, v in pricing.items() if not k.startswith("_")}) + return out def selector(signature: str) -> str: @@ -240,7 +375,10 @@ def selector(signature: str) -> str: def eth_call(to: str, data: str) -> str: - return rpc("eth_call", [{"to": to, "data": data}, "latest"]) + result = rpc("eth_call", [{"to": to, "data": data}, "latest"]) + if result == "0x": + raise RuntimeError(f"empty return from {to}: no contract at that address?") + return result def decode_address(hex_data: str) -> str: @@ -255,6 +393,33 @@ def decode_bytes(hex_data: str) -> bytes: return raw[64:64 + length] +def registered_label(registrar: str, token: int) -> str: + """The plaintext label the registrar recorded at registration, keyed by the + hash of that label. A name registered without registerWithLabel has none, + and answers "unknown" instead.""" + raw = decode_bytes(eth_call(registrar, selector("labelOf(uint256)") + encode_uint(token))) + return raw.decode("utf-8", errors="replace") if raw else "unknown" + + +def canonical_name(name: str) -> str: + """The name to answer with: a hashed query does not carry one, so the + registrar's record of the label fills it in.""" + labels = name.split(".") + registrar = REGISTRARS.get(labels[-1]) + if not registrar or len(labels) < 2 or not is_encoded_labelhash(labels[-2]): + return name + label = registered_label(registrar, label_token(labels[-2])) + return ".".join(labels[:-2] + [label, labels[-1]]) + + +def label_token(label: str) -> int: + """The registry key for a second-level label, whether it arrived as text or + already hashed.""" + if is_encoded_labelhash(label): + return int(label[1:-1], 16) + return int.from_bytes(keccak(label.encode()), "big") + + def decode_uint(hex_data: str) -> int: raw = hex_data[2:] if hex_data.startswith("0x") else hex_data return int(raw[-64:], 16) if raw else 0 @@ -539,46 +704,41 @@ def resolve(name: str): # Before the resolver lookup, so a lapsed name is not reported as noResolver. reg = name_status(name) - if reg["status"] in ("unregistered", "reserved"): + if reg["status"] in ("unregistered", "expired"): + # A name in grace is not here: its record still resolves. body = { "name": name, - "status": reg["status"], - "expires": reg["expires"], - "graceEnds": reg["graceEnds"], + **reg, "error": reg["status"], "message": ( - "this name is reserved and cannot be registered" - if reg["status"] == "reserved" - else "this name has never been registered" - ), - } - if reg["status"] == "reserved": - body["reason"] = RESERVED_REASON - return 404, body - if reg["status"] in ("grace", "expired"): - return 410, { - "name": name, - "status": reg["status"], - "expires": reg["expires"], - "graceEnds": reg["graceEnds"], - "error": reg["status"], - "message": ( - "this registration expired and can be renewed by its owner" - if reg["status"] == "grace" + "this name has never been registered" + if reg["status"] == "unregistered" else "this registration expired and is open to anyone" ), } + return (404 if reg["status"] == "unregistered" else 410), body resolver_raw = eth_call(registry, selector("resolver(bytes32)") + node_hex) resolver_addr = decode_address(resolver_raw) if resolver_addr == ZERO_ADDR: - return 404, { - "name": name, - "status": "noResolver", - "expires": reg["expires"], - "graceEnds": reg["graceEnds"], - "error": "noResolver", - "message": "no resolver set for this name", + # A registered name always resolves: with no resolver set the record is + # still returned with every field unset, so "taken until " stays + # answerable. + owner = decode_address(eth_call(registry, selector("owner(bytes32)") + node_hex)) + return 200, { + "name": canonical_name(name), + "nickname": "", + "website": "", + "location": "", + "simplexContact": [], + "simplexChannel": [], + "eth": None, + "btc": None, + "xmr": None, + "dot": None, + "owner": owner, + "resolver": ZERO_ADDR, + **reg, } owner_raw = eth_call(registry, selector("owner(bytes32)") + node_hex) @@ -603,7 +763,7 @@ def resolve(name: str): # use the ENSIP-5 dot convention (e.g. "simplex.contact") — only the # resolver's JSON surface camelCases them. return 200, { - "name": name, + "name": canonical_name(name), "nickname": nickname, "website": texts.get("url", ""), "location": texts.get("location", ""), @@ -615,9 +775,7 @@ def resolve(name: str): "dot": addr_multicoin(resolver_addr, node, COIN_DOT), "owner": owner, "resolver": resolver_addr, - "status": reg["status"], - "expires": reg["expires"], - "graceEnds": reg["graceEnds"], + **reg, } diff --git a/scripts/resolver/service/test_snrc_resolve.py b/scripts/resolver/service/test_snrc_resolve.py index 16c82f610..fdc25b9a5 100644 --- a/scripts/resolver/service/test_snrc_resolve.py +++ b/scripts/resolver/service/test_snrc_resolve.py @@ -97,6 +97,7 @@ def setUp(self): snrc.REGISTRARS = {"testing": self.REGISTRAR} snrc.CONTROLLERS = {"testing": ""} snrc.chain_now = lambda: int(time.time()) + snrc._constants.clear() def tearDown(self): snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now = self._saved @@ -135,7 +136,9 @@ def test_hash_and_label_reach_the_same_node(self): def test_a_plain_name_is_unaffected(self): self.assertEqual(snrc.node_of("alice.testing"), snrc.namehash("alice.testing")) - def test_an_encoded_subname_is_not_the_name_it_would_decode_to(self): + def test_a_bracket_subname_label_stays_literal(self): + """Only the 2LD is a key, so a bracket label left of it is hashed as + written.""" self.assertNotEqual( snrc.node_of( "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" @@ -143,13 +146,24 @@ def test_an_encoded_subname_is_not_the_name_it_would_decode_to(self): ), snrc.namehash("alice.alice.testing"), ) - self.assertNotEqual( + + def test_a_hashed_2ld_under_a_subname_reaches_the_same_node(self): + """`sub.[hash].tld` must reach the node `sub.name.tld` does.""" + self.assertEqual( snrc.node_of( - "alice." + "sub." "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" ".testing" ), - snrc.namehash("alice.alice.testing"), + snrc.namehash("sub.alice.testing"), + ) + self.assertEqual( + snrc.node_of( + "a.b." + "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" + ".testing" + ), + snrc.namehash("a.b.alice.testing"), ) def test_a_0x_prefixed_label_is_taken_literally(self): @@ -197,6 +211,17 @@ def eth_call(to, data): return eth_call + def _keys(self, status, expires, grace_ends): + """Every branch answers with the same keys; only some carry values.""" + return { + "status": status, + "expires": expires, + "graceEnds": grace_ends, + "reasonCode": None, + "reason": None, + "auctionUntil": None, + } + def setUp(self): self._saved = ( snrc.REGISTRARS, @@ -209,6 +234,7 @@ def setUp(self): # Expiry alone; ReservedTests covers a configured controller. snrc.CONTROLLERS = {"testing": ""} snrc.chain_now = lambda: int(time.time()) + snrc._constants.clear() def tearDown(self): ( @@ -232,11 +258,33 @@ def test_status_reads_the_chain_clock_not_the_host_clock(self): snrc.chain_now = lambda: future + 3650 * 86400 self.assertEqual(snrc.name_status("alice.testing")["status"], "expired") + def test_a_registrar_that_is_not_a_contract_is_an_error_not_a_free_name(self): + """An address with no code answers eth_call with empty data. Read as + zero, that would make every name look free.""" + snrc.eth_call = self._saved[2] # the real one, so its guard runs + snrc.rpc = lambda method, params: "0x" + with self.assertRaises(RuntimeError): + snrc.name_status("alice.testing") + + def test_the_grace_period_is_read_once_not_per_query(self): + seen = [] + + def eth_call(to, data): + seen.append(data[:10]) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + return "0x" + snrc.encode_uint(int(time.time()) - 3600) + + snrc.eth_call = eth_call + snrc.name_status("alice.testing") + snrc.name_status("alice.testing") + self.assertEqual(seen.count(snrc.selector("GRACE_PERIOD()")), 1) + def test_zero_expiry_means_never_registered(self): snrc.eth_call = self._expiry(0) self.assertEqual( snrc.name_status("alice.testing"), - {"status": "unregistered", "expires": None, "graceEnds": None}, + self._keys("unregistered", None, None), ) def test_recently_expired_is_in_grace_and_says_when_it_ends(self): @@ -244,7 +292,7 @@ def test_recently_expired_is_in_grace_and_says_when_it_ends(self): snrc.eth_call = self._expiry(past) self.assertEqual( snrc.name_status("alice.testing"), - {"status": "grace", "expires": past, "graceEnds": past + self.GRACE}, + self._keys("grace", past, past + self.GRACE), ) def test_past_the_grace_window_it_is_expired_and_claimable(self): @@ -263,7 +311,7 @@ def test_future_expiry_is_registered(self): snrc.eth_call = self._expiry(future) self.assertEqual( snrc.name_status("alice.testing"), - {"status": "registered", "expires": future, "graceEnds": future + self.GRACE}, + self._keys("registered", future, future + self.GRACE), ) def test_never_registered_is_not_confused_with_claimable(self): @@ -285,16 +333,36 @@ def eth_call(to, data): # the token asked about is keccak("alice"), not keccak("x") self.assertTrue(seen[0].endswith(snrc.keccak(b"alice").hex())) + def test_a_hashed_2ld_is_queried_by_its_hash_at_any_depth(self): + """The token must come from the hash, not from hashing the brackets.""" + seen = [] + + def eth_call(to, data): + seen.append(data) + return "0x" + snrc.encode_uint(0) + + snrc.eth_call = eth_call + hashed = "[" + snrc.keccak(b"alice").hex() + "]" + snrc.name_status("x." + hashed + ".testing") + self.assertTrue(seen[0].endswith(snrc.keccak(b"alice").hex())) + def test_unconfigured_tld_is_unknown_rather_than_unregistered(self): snrc.REGISTRARS = {"testing": ""} snrc.eth_call = lambda *a: self.fail("must not reach the chain") self.assertEqual( snrc.name_status("alice.testing"), - {"status": "unknown", "expires": None, "graceEnds": None}, + self._keys("unknown", None, None), ) def test_every_branch_returns_the_same_keys(self): - keys = {"status", "expires", "graceEnds"} + keys = { + "status", + "expires", + "graceEnds", + "reasonCode", + "reason", + "auctionUntil", + } snrc.eth_call = self._expiry(0) self.assertEqual(set(snrc.name_status("alice.testing")), keys) snrc.eth_call = self._expiry(int(time.time()) + 3600) @@ -313,6 +381,7 @@ def setUp(self): snrc.REGISTRARS = {"testing": self.REGISTRAR} snrc.CONTROLLERS = {"testing": self.CONTROLLER} snrc.chain_now = lambda: int(time.time()) + snrc._constants.clear() def tearDown(self): snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now = self._saved @@ -324,22 +393,28 @@ def eth_call(to, data): return "0x" + snrc.encode_uint(1 if reserved else 0) if data.startswith(snrc.selector("GRACE_PERIOD()")): return "0x" + snrc.encode_uint(90 * 86400) + if data.startswith(snrc.selector("prices()")): + return "0x" + snrc.encode_uint(0) # no price oracle, no auction return "0x" + snrc.encode_uint(expires) return eth_call - def test_unregistered_and_reserved_reads_reserved(self): + def test_unregistered_and_reserved_reports_the_reservation(self): snrc.eth_call = self._chain(0, True) - self.assertEqual(snrc.name_status("acme.testing")["status"], "reserved") + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "unregistered") + self.assertEqual(reg["reasonCode"], "internal") def test_unregistered_and_not_reserved_reads_unregistered(self): snrc.eth_call = self._chain(0, False) self.assertEqual(snrc.name_status("acme.testing")["status"], "unregistered") - def test_a_lapsed_reserved_name_is_reserved_not_claimable(self): + def test_a_lapsed_reserved_name_keeps_its_reservation(self): past = int(time.time()) - 91 * 86400 snrc.eth_call = self._chain(past, True) - self.assertEqual(snrc.name_status("acme.testing")["status"], "reserved") + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + self.assertEqual(reg["reasonCode"], "internal") def test_a_live_name_is_registered_even_if_reserved(self): snrc.eth_call = self._chain(int(time.time()) + 86400, True) @@ -358,7 +433,7 @@ def test_reserved_is_asked_by_labelhash_so_a_hashed_query_works(self): # keccak-256("acme") hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" snrc.eth_call = self._chain(0, True) - self.assertEqual(snrc.name_status(hashed + ".testing")["status"], "reserved") + self.assertEqual(snrc.name_status(hashed + ".testing")["reasonCode"], "internal") class ReservedReasonTests(unittest.TestCase): @@ -378,6 +453,7 @@ def setUp(self): snrc.REGISTRARS = {"testing": self.REGISTRAR} snrc.CONTROLLERS = {"testing": self.CONTROLLER} snrc.chain_now = lambda: int(time.time()) + snrc._constants.clear() def tearDown(self): ( @@ -394,16 +470,57 @@ def eth_call(to, data): return "0x" + snrc.encode_uint(1 if reserved else 0) if data.startswith(snrc.selector("GRACE_PERIOD()")): return "0x" + snrc.encode_uint(90 * 86400) + if data.startswith(snrc.selector("prices()")): + return "0x" + snrc.encode_uint(0) # no price oracle, no auction return "0x" + snrc.encode_uint(expires) return eth_call + def _reserved_as(self, code): + def eth_call(to, data): + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(code) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(90 * 86400) + if data.startswith(snrc.selector("prices()")): + return "0x" + snrc.encode_uint(0) + return "0x" + snrc.encode_uint(0) + + return eth_call + + def test_every_enum_value_has_a_code_and_a_sentence(self): + for code, (name, sentence) in snrc.RESERVED_REASONS.items(): + snrc.eth_call = self._reserved_as(code) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["reasonCode"], name) + self.assertEqual(reg["reason"], sentence) + + def test_a_trademark_reservation_says_so(self): + snrc.eth_call = self._reserved_as(2) + _, body = snrc.resolve("acme.testing") + self.assertEqual(body["reasonCode"], "trademark") + + def test_a_controller_storing_a_bool_reads_as_internal(self): + """Before the enum `reservedNames` was a bool; its `true` decodes as 1.""" + snrc.eth_call = self._reserved_as(1) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["reasonCode"], "internal") + self.assertEqual(reg["reason"], "reserved for SimpleX") + + def test_an_enum_value_this_resolver_predates_is_not_dropped(self): + """A new Reason still reserves the name, and says it is unknown rather + than claiming the chain recorded none.""" + snrc.eth_call = self._reserved_as(99) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["reasonCode"], "unknown") + self.assertEqual(reg["reason"], "reserved") + def test_a_reserved_name_carries_the_reason(self): snrc.eth_call = self._chain(0, True) status, body = snrc.resolve("acme.testing") self.assertEqual(status, 404) - self.assertEqual(body["status"], "reserved") - self.assertEqual(body["reason"], "reserved for a brand or public interest") + self.assertEqual(body["status"], "unregistered") + self.assertEqual(body["reason"], "reserved for SimpleX") def test_the_message_does_not_claim_a_trademark(self): snrc.eth_call = self._chain(0, True) @@ -415,21 +532,211 @@ def test_an_unregistered_name_has_no_reason(self): status, body = snrc.resolve("acme.testing") self.assertEqual(status, 404) self.assertEqual(body["status"], "unregistered") - self.assertNotIn("reason", body) + self.assertIsNone(body["reason"]) def test_an_expired_name_has_no_reason(self): snrc.eth_call = self._chain(1, False) status, body = snrc.resolve("acme.testing") self.assertEqual(status, 410) self.assertEqual(body["status"], "expired") - self.assertNotIn("reason", body) + self.assertIsNone(body["reason"]) def test_a_hashed_query_gets_the_reason_too(self): snrc.eth_call = self._chain(0, True) # keccak-256("acme") hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" _, body = snrc.resolve(hashed + ".testing") - self.assertEqual(body["reason"], "reserved for a brand or public interest") + self.assertEqual(body["reason"], "reserved for SimpleX") + + +class PricingTests(unittest.TestCase): + """The oracle keeps the curve in US cents per year, and a lapsed name costs + the ordinary price: this registry runs no auction.""" + + REGISTRY = "0x58fc46996d975c57883564648bda5206d1a0102b" + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + CONTROLLER = "0x281ca41311c2aa808c917c4674639d7567b75714" + ORACLE = "0x1e0c9a2b9d1a4c8f7b3e5d6a9c2f4b8e1d7a3c50" + + GRACE = 90 * 86400 + BASE = 200 + EXCEPTIONS = {1: 64000, 2: 16000, 3: 1600, 4: 800, 5: 500} + MIN_LENGTH = 3 + + def setUp(self): + self._saved = ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) + snrc.REGISTRIES = {"testing": self.REGISTRY} + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": self.CONTROLLER} + self.now = int(time.time()) + snrc.chain_now = lambda: self.now + snrc._constants.clear() + + def tearDown(self): + ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) = self._saved + + def _prices_return(self): + words = [snrc.encode_uint(self.BASE), snrc.encode_uint(0x40), + snrc.encode_uint(len(self.EXCEPTIONS))] + for length, cents in self.EXCEPTIONS.items(): + words += [snrc.encode_uint(length), snrc.encode_uint(cents)] + return "0x" + "".join(words) + + def _chain(self, expires, oracle=None, reserved=0): + oracle = self.ORACLE if oracle is None else oracle + self.oracle_calls = [] + + def eth_call(to, data): + if data.startswith(snrc.selector("nameExpires(uint256)")): + return "0x" + snrc.encode_uint(expires) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(reserved) + if data.startswith(snrc.selector("minCharLength()")): + self.assertEqual(to, self.CONTROLLER) + return "0x" + snrc.encode_uint(self.MIN_LENGTH) + if data.startswith(snrc.selector("prices()")): + if to == self.CONTROLLER: + return "0x" + snrc.encode_uint(int(oracle, 16)) + self.oracle_calls.append(data[:10]) + self.assertEqual(to, oracle) + return self._prices_return() + return self.fail("unexpected call " + data[:10]) + + return eth_call + + def _lapsed(self, days_past_grace): + """An expiry whose grace ended `days_past_grace` days ago. The extra + second clears the boundary, which counts as still in grace.""" + return self.now - self.GRACE - 1 - days_past_grace * 86400 + + def test_the_prices_are_the_oracles_cents_per_year(self): + snrc.eth_call = self._chain(self._lapsed(0)) + reg = snrc.name_status("acme.testing") + # 1 and 2 are below minCharLength + self.assertEqual(reg["rentPrices"], {3: 1600, 4: 800, 5: 500}) + self.assertEqual(reg["basePrice"], self.BASE) + self.assertEqual(reg["minLabelLength"], self.MIN_LENGTH) + + def test_a_lapsed_name_costs_the_ordinary_price(self): + snrc.eth_call = self._chain(self._lapsed(0)) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + self.assertIsNone(reg["auctionUntil"]) + + def test_a_controller_with_no_oracle_leaves_the_name_merely_expired(self): + snrc.eth_call = self._chain(self._lapsed(0), oracle=snrc.ZERO_ADDR) + self.assertEqual(snrc.name_status("acme.testing")["status"], "expired") + + def test_a_name_in_grace_never_reaches_the_oracle(self): + snrc.eth_call = self._chain(self.now - 3600) + self.assertEqual(snrc.name_status("acme.testing")["status"], "grace") + self.assertEqual(self.oracle_calls, []) + + def test_the_oracle_curve_is_read_once_not_per_query(self): + snrc.eth_call = self._chain(self._lapsed(1)) + snrc.name_status("acme.testing") + seen_first = len(self.oracle_calls) + snrc.name_status("acme.testing") + self.assertEqual(self.oracle_calls[seen_first:], []) + + def test_a_reserved_lapsed_name_keeps_its_reservation(self): + snrc.eth_call = self._chain(self._lapsed(0), reserved=2) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + self.assertEqual(reg["reasonCode"], "trademark") + + def test_resolve_reports_the_prices(self): + snrc.eth_call = self._chain(self._lapsed(1)) + status, body = snrc.resolve("acme.testing") + self.assertEqual(status, 410) + self.assertEqual(body["status"], "expired") + self.assertEqual(body["basePrice"], self.BASE) + self.assertIsNone(body["auctionUntil"]) + + def test_a_hashed_query_is_priced_too(self): + # keccak-256("acme") + hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" + snrc.eth_call = self._chain(self._lapsed(0)) + _, body = snrc.resolve(hashed + ".testing") + self.assertEqual(body["status"], "expired") + self.assertEqual(body["basePrice"], self.BASE) + + +class EnsOracleTests(unittest.TestCase): + """.testing runs an ENS-shaped oracle: it prices in attoUSD per second and + charges a premium on lapsed names that it does not expose.""" + + REGISTRY = "0x58fc46996d975c57883564648bda5206d1a0102b" + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + CONTROLLER = "0x281ca41311c2aa808c917c4674639d7567b75714" + ORACLE = "0x1e0c9a2b9d1a4c8f7b3e5d6a9c2f4b8e1d7a3c50" + GRACE = 90 * 86400 + MIN_LENGTH = 6 + + def setUp(self): + self._saved = (snrc.REGISTRIES, snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now) + snrc.REGISTRIES = {"testing": self.REGISTRY} + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": self.CONTROLLER} + self.now = int(time.time()) + snrc.chain_now = lambda: self.now + snrc._constants.clear() + + def tearDown(self): + (snrc.REGISTRIES, snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now) = self._saved + + def _chain(self, expires, letter_cents=0): + def eth_call(to, data): + if data.startswith(snrc.selector("nameExpires(uint256)")): + return "0x" + snrc.encode_uint(expires) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(0) + if data.startswith(snrc.selector("minCharLength()")): + return "0x" + snrc.encode_uint(self.MIN_LENGTH) + if data.startswith(snrc.selector("prices()")): + if to == self.CONTROLLER: + return "0x" + snrc.encode_uint(int(self.ORACLE, 16)) + raise RuntimeError("eth_call returned 0x") # no prices() on this oracle + for n in range(1, 7): + if data.startswith(snrc.selector(f"price{n}Letter()")): + rate = letter_cents * snrc.ATTO_PER_CENT // snrc.SECONDS_PER_YEAR + return "0x" + snrc.encode_uint(rate) + return self.fail("unexpected call " + data[:10]) + + return eth_call + + def test_a_never_registered_name_is_priced_from_the_letter_curve(self): + snrc.eth_call = self._chain(0) + reg = snrc.name_status("ghost.testing") + self.assertEqual(reg["status"], "unregistered") + self.assertEqual(reg["basePrice"], 0) + self.assertEqual(reg["minLabelLength"], self.MIN_LENGTH) + + def test_a_non_zero_letter_curve_converts_to_cents_per_year(self): + snrc.eth_call = self._chain(0, letter_cents=1200) + self.assertEqual(snrc.name_status("ghost.testing")["basePrice"], 1200) + + def test_a_lapsed_name_is_not_priced_because_the_premium_is_unreadable(self): + snrc.eth_call = self._chain(self.now - self.GRACE - 1) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + self.assertNotIn("basePrice", reg) class ErrorCodeTests(unittest.TestCase): @@ -448,6 +755,7 @@ def setUp(self): snrc.REGISTRARS = {"testing": self.REGISTRAR} snrc.CONTROLLERS = {"testing": ""} snrc.chain_now = lambda: int(time.time()) + snrc._constants.clear() def tearDown(self): ( @@ -478,7 +786,6 @@ def test_an_unconfigured_tld_names_the_ones_that_are(self): def test_a_registration_problem_reports_the_status_as_the_code(self): for expires, code in ( (0, "unregistered"), - (int(time.time()) - 3600, "grace"), (int(time.time()) - 91 * 86400, "expired"), ): with self.subTest(code=code): @@ -487,12 +794,20 @@ def test_a_registration_problem_reports_the_status_as_the_code(self): self.assertEqual(body["error"], code) self.assertEqual(body["status"], code) - def test_a_registered_name_pointing_nowhere_is_noResolver(self): + def test_a_name_in_grace_still_resolves(self): + snrc.eth_call = self._chain(int(time.time()) - 3600) + status, body = snrc.resolve("alice.testing") + self.assertEqual(status, 200) + self.assertEqual(body["status"], "grace") + self.assertNotIn("error", body) + + def test_a_registered_name_pointing_nowhere_resolves_with_empty_records(self): snrc.eth_call = self._chain(int(time.time()) + 86400) status, body = snrc.resolve("alice.testing") - self.assertEqual(status, 404) - self.assertEqual(body["error"], "noResolver") - self.assertEqual(body["status"], "noResolver") + self.assertEqual(status, 200) + self.assertEqual(body["status"], "registered") + self.assertEqual(body["resolver"], snrc.ZERO_ADDR) + self.assertEqual(body["simplexContact"], []) def test_every_error_body_carries_both_fields(self): snrc.eth_call = self._chain(0) diff --git a/src/Simplex/Messaging/Agent.hs b/src/Simplex/Messaging/Agent.hs index 5caaa5be3..552996d1b 100644 --- a/src/Simplex/Messaging/Agent.hs +++ b/src/Simplex/Messaging/Agent.hs @@ -228,7 +228,7 @@ import Simplex.Messaging.Protocol ErrorType (AUTH), MsgBody, MsgFlags (..), - NameRecord, + NameRegistration (..), NtfServer, ProtoServerWithAuth (..), ProtocolServer (..), @@ -461,7 +461,7 @@ getConnShortLink c = withAgentEnv c .:. getConnShortLink' c -- | Resolve a SimpleX name (PFWD RSLV). The agent owns server selection: it -- picks a names-capable server (ServerRoles.names) from the user's nameSrvs, so -- chat clients just pass the parsed domain. -resolveSimplexName :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AE NameRecord +resolveSimplexName :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AE NameRegistration resolveSimplexName c nm userId domain = withAgentEnv c $ resolveSimplexName' c nm userId domain {-# INLINE resolveSimplexName #-} @@ -1270,7 +1270,7 @@ getConnShortLink' c nm userId = \case deleteLocalInvShortLink' :: AgentClient -> ConnShortLink 'CMInvitation -> AM () deleteLocalInvShortLink' c (CSLInvitation _ srv linkId _) = withStore' c $ \db -> deleteInvShortLink db srv linkId -resolveSimplexName' :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AM NameRecord +resolveSimplexName' :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AM NameRegistration resolveSimplexName' c nm userId domain = do resolverSrv <- getNextNameServer c userId resolveName c nm userId resolverSrv domain diff --git a/src/Simplex/Messaging/Agent/Client.hs b/src/Simplex/Messaging/Agent/Client.hs index 1f25c4cb5..b1ee8c011 100644 --- a/src/Simplex/Messaging/Agent/Client.hs +++ b/src/Simplex/Messaging/Agent/Client.hs @@ -272,7 +272,7 @@ import Simplex.Messaging.Protocol NetworkError (..), MsgFlags (..), MsgId, - NameRecord, + NameRegistration (..), NtfServer, NtfServerWithAuth, ProtoServer, @@ -2022,7 +2022,7 @@ getQueueLink c nm userId server lnkId = -- resolver) and falls back to a direct send when the proxy is unavailable -- (faster but exposes the client IP). Mode selection is delegated to -- `sendOrProxySMPCommand`, which honours the network config (SPMNever etc.). -resolveName :: AgentClient -> NetworkRequestMode -> UserId -> SMPServer -> SimplexDomain -> AM NameRecord +resolveName :: AgentClient -> NetworkRequestMode -> UserId -> SMPServer -> SimplexDomain -> AM NameRegistration resolveName c nm userId server domain = snd <$> sendOrProxySMPCommand c nm userId server "" "RSLV" NoEntity resolveViaProxy resolveDirectly where diff --git a/src/Simplex/Messaging/Client.hs b/src/Simplex/Messaging/Client.hs index 6f5234558..12c67e725 100644 --- a/src/Simplex/Messaging/Client.hs +++ b/src/Simplex/Messaging/Client.hs @@ -166,7 +166,7 @@ import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON, sumTypeJSON import Simplex.Messaging.Protocol import Simplex.Messaging.Protocol.Types import Simplex.Messaging.Server.QueueStore.QueueInfo -import Simplex.Messaging.SimplexName (SimplexDomain) +import Simplex.Messaging.SimplexName (SimplexDomain, fullDomainName) import Simplex.Messaging.TMap (TMap) import qualified Simplex.Messaging.TMap as TM import Simplex.Messaging.Transport @@ -1054,27 +1054,38 @@ proxySMPMessage c nm proxiedRelay spKey sId flags msg = proxyOKSMPCommand c nm p -- through `proxySMPCommand` and pattern-matches the expected RNAME response. -- Version-gated on the destination relay (mirrors `connectSMPProxiedRelay`): -- the client never sends RSLV to a relay that predates names support. -proxyResolveName :: SMPClient -> NetworkRequestMode -> ProxiedRelay -> SimplexDomain -> ExceptT SMPClientError IO (Either ProxyClientError NameRecord) +proxyResolveName :: SMPClient -> NetworkRequestMode -> ProxiedRelay -> SimplexDomain -> ExceptT SMPClientError IO (Either ProxyClientError NameRegistration) proxyResolveName c nm proxiedRelay name - | prVersion proxiedRelay >= namesSMPVersion = - proxySMPCommand c nm proxiedRelay Nothing NoEntity (RSLV name) >>= \case - Right (RNAME nr) -> pure $ Right nr + | v >= namesSMPVersion = + proxySMPCommand c nm proxiedRelay Nothing NoEntity (RSLV (nameQuery v name)) >>= \case + Right (RNAME reg) | resolvedName name reg -> pure $ Right reg Right r -> throwE $ unexpectedResponse r Left e -> pure $ Left e | otherwise = throwE $ PCETransportError TEVersion + where + v = prVersion proxiedRelay -- | Direct (non-PFWD) name resolution. Exposes the client IP to the resolver; -- callers that want anonymity should use `proxyResolveName` via the standard -- proxy fallback in the agent. RSLV requires no entity ID or authorization --- (see `noAuthCmd` in Protocol.hs). Version-gated on the session here, not the --- encoder, so an old server never receives RSLV. -directResolveName :: SMPClient -> NetworkRequestMode -> SimplexDomain -> ExceptT SMPClientError IO NameRecord +-- (see `noAuthCmd` in Protocol.hs). Gated on the session version, below which +-- the server has no RSLV at all; the encoder gates the query format separately. +directResolveName :: SMPClient -> NetworkRequestMode -> SimplexDomain -> ExceptT SMPClientError IO NameRegistration directResolveName c nm name - | thVersion (thParams c) >= namesSMPVersion = - sendProtocolCommand c nm Nothing NoEntity (Cmd SResolver (RSLV name)) >>= \case - RNAME nr -> pure nr + | v >= namesSMPVersion = + sendProtocolCommand c nm Nothing NoEntity (Cmd SResolver (RSLV (nameQuery v name))) >>= \case + RNAME reg | resolvedName name reg -> pure reg r -> throwE $ unexpectedResponse r | otherwise = throwE $ PCETransportError TEVersion + where + v = thVersion (thParams c) + +-- | The record must name the name that was asked for: a hashed query does not +-- tell the router which name it is, so the router is not trusted for it. +resolvedName :: SimplexDomain -> NameRegistration -> Bool +resolvedName d = \case + NRRegistered {nameRecord} -> T.toLower (nrName nameRecord) == fullDomainName d + _ -> True -- | Acknowledge message delivery (server deletes the message). -- diff --git a/src/Simplex/Messaging/Protocol.hs b/src/Simplex/Messaging/Protocol.hs index a28d78fe6..3899d6b9c 100644 --- a/src/Simplex/Messaging/Protocol.hs +++ b/src/Simplex/Messaging/Protocol.hs @@ -80,6 +80,15 @@ module Simplex.Messaging.Protocol ErrorType (..), CommandError (..), ProxyError (..), + NameQuery (..), + NameQueryLabel (..), + nameQuery, + queryName, + NameRegistration (..), + NamePricing (..), + USDCents (..), + NameReservedReason (..), + oldRegistration, NameErrorType (..), BrokerErrorType (..), NetworkError (..), @@ -248,6 +257,8 @@ import Data.Kind import Data.List (foldl') import Data.List.NonEmpty (NonEmpty (..)) import qualified Data.List.NonEmpty as L +import Data.Map.Strict (Map) +import qualified Data.Map.Strict as M import Data.Maybe (isJust, isNothing) import Data.String import Data.Text (Text) @@ -270,7 +281,8 @@ import Simplex.Messaging.Parsers import Simplex.Messaging.Protocol.Types import Simplex.Messaging.Server.QueueStore.QueueInfo import Simplex.Messaging.ServiceScheme -import Simplex.Messaging.SimplexName (SimplexDomain) +import Simplex.Messaging.SystemTime (SystemSeconds) +import Simplex.Messaging.SimplexName (LabelHash, SimplexDomain (..), SimplexTLD (..), fullDomainName, labelHash, labelHashText) import Simplex.Messaging.Transport import Simplex.Messaging.Transport.Client (TransportHost, TransportHosts (..)) import Simplex.Messaging.Util (bshow, eitherToMaybe, safeDecodeUtf8, (<$?>)) @@ -603,7 +615,7 @@ data Command (p :: Party) where -- - corrId: unique correlation ID between proxy and relay, also used as a nonce to encrypt forwarded transmission RFWD :: EncFwdTransmission -> Command ProxyService -- use CorrId as CbNonce, proxy to relay -- Resolve SimpleX name. - RSLV :: SimplexDomain -> Command Resolver + RSLV :: NameQuery -> Command Resolver deriving instance Show (Command p) @@ -739,8 +751,8 @@ data BrokerMsg where OK :: BrokerMsg ERR :: ErrorType -> BrokerMsg PONG :: BrokerMsg - -- Resolved SimpleX name. - RNAME :: NameRecord -> BrokerMsg + -- What the router knows about a SimpleX name. + RNAME :: NameRegistration -> BrokerMsg deriving (Eq, Show) data RcvMessage = RcvMessage @@ -1589,6 +1601,182 @@ data ErrorType DUPLICATE_ -- not part of SMP protocol, used internally deriving (Eq, Show) +-- | What RSLV asks about. Unlike SimplexDomain, which is always text, this may +-- name a label by its hash. +data NameQuery = NameQuery + { queryTLD :: SimplexTLD, + -- | only the second-level label may be hashed: subname labels are needed as + -- text to reach the record + queryLabel :: NameQueryLabel, + -- | parent to child, as in SimplexDomain + querySub :: [Text] + } + deriving (Eq, Show) + +data NameQueryLabel + = NQName Text + | NQHash LabelHash + deriving (Eq, Show) + +instance Encoding NameQueryLabel where + smpEncode = \case + NQName t -> smpEncode ('N', t) + NQHash h -> smpEncode ('H', h) + smpP = + A.anyChar >>= \case + 'N' -> NQName <$> smpP + 'H' -> NQHash <$> smpP + _ -> fail "bad NameQueryLabel" + +-- | How the backing resolver is addressed for this query. +queryName :: NameQuery -> Text +queryName = fullDomainName . queryDomain + +-- | The query as a name: what RSLV carries below v22, and what the resolver's +-- HTTP API takes. The only place a hashed label is written as text; SMP tags +-- the choice instead. +queryDomain :: NameQuery -> SimplexDomain +queryDomain NameQuery {queryTLD, queryLabel, querySub} = + SimplexDomain {nameTLD = queryTLD, domain = label, subDomain = querySub} + where + label = case queryLabel of + NQName t -> t + NQHash h -> labelHashText h + +-- | The name a client asked about, hashed from v22. The hash only hides an +-- unregistered name: a registered one comes back with its name in the record, +-- and a short label is guessable by hashing candidates. A web TLD has no +-- registry, so it is never hashed. +nameQuery :: VersionSMP -> SimplexDomain -> NameQuery +nameQuery v SimplexDomain {nameTLD, domain, subDomain} = + NameQuery {queryTLD = nameTLD, queryLabel = label, querySub = subDomain} + where + label + | v >= nameAvailSMPVersion && nameTLD /= TLDWeb = NQHash (labelHash domain) + | otherwise = NQName domain + +instance Encoding NameQuery where + smpEncode NameQuery {queryTLD, queryLabel, querySub} = + smpEncode (queryTLD, queryLabel, EncList querySub) + smpP = do + (queryTLD, queryLabel, EncList querySub) <- smpP + pure NameQuery {queryTLD, queryLabel, querySub} + +-- | US cents, rounded up where the registry's unit does not divide evenly, so +-- a quote is never below what is charged. The exact price is settled on chain. +newtype USDCents = USDCents Int64 + deriving (Eq, Ord, Show) + deriving newtype (Encoding) + +-- | What the registry holds for a name. A name that cannot be dated or priced +-- is not a case here: the router answers ERR NAME RESOLVER instead. +data NameRegistration + = -- | Held by someone. Always carries a record: where the owner set none, + -- every field is unset and the resolver address is zero. + NRRegistered + { -- | unix seconds the registration runs out. Absent only from a v20/v21 + -- router, whose answer carried the record alone. + expires :: Maybe SystemSeconds, + -- | unix seconds, > expires: until here only the owner may renew + graceUntil :: Maybe SystemSeconds, + -- | held back as well, which is why it will not free up at expiry + reservedReason_ :: Maybe NameReservedReason, + nameRecord :: NameRecord + } + | -- | Held by nobody, and registrable now. + NRAvailable + { pricing :: NamePricing, + -- | while set, the name also costs a surcharge above `pricing` that + -- decays to nothing at this time. The surcharge itself is not carried: + -- it changes continuously, so it cannot be quoted as a price. + auctionUntil :: Maybe SystemSeconds + } + | -- | Held back by the registry and not registered. No price: it is not for + -- sale at the registry's price. + NRReserved {reservedReason :: NameReservedReason} + deriving (Eq, Show) + +instance Encoding NameRegistration where + smpEncode = \case + NRRegistered {expires, graceUntil, reservedReason_, nameRecord} -> + smpEncode ('N', expires, graceUntil, reservedReason_, ' ', Tail $ LB.toStrict $ J.encode nameRecord) + NRAvailable {pricing, auctionUntil} -> smpEncode ('A', auctionUntil, pricing) + NRReserved {reservedReason} -> smpEncode ('R', reservedReason) + smpP = + A.anyChar >>= \case + 'N' -> do + (expires, graceUntil, reservedReason_) <- smpP + nameRecord <- J.eitherDecodeStrict . unTail <$?> _smpP + pure NRRegistered {expires, graceUntil, reservedReason_, nameRecord} + 'A' -> do + auctionUntil <- smpP + pricing <- smpP + pure NRAvailable {pricing, auctionUntil} + 'R' -> NRReserved <$> smpP + _ -> fail "bad NameRegistration" + +-- | Enough to price the name locally. The client knows the label, so it knows +-- which tier applies and whether the label is long enough; the router, behind a +-- hash, knows neither. The formula is in protocol/simplex-messaging.md. +data NamePricing = NamePricing + { -- | US cents per year, for the lengths the registry prices specially + rentPrices :: Map Int USDCents, + -- | US cents per year for every other length + basePrice :: USDCents, + -- | characters; the registry refuses shorter labels + minLabelLength :: Int + } + deriving (Eq, Show) + +instance Encoding NamePricing where + smpEncode NamePricing {rentPrices, basePrice, minLabelLength} = + smpEncode (EncList $ map tier $ M.toList rentPrices, basePrice, w16 minLabelLength) + where + tier (len, price) = (w16 len, price) + w16 = fromIntegral :: Int -> Word16 + smpP = do + (EncList tiers, basePrice, minLen) <- smpP + pure NamePricing {rentPrices = tierMap tiers, basePrice, minLabelLength = fromIntegral (minLen :: Word16)} + where + tierMap :: [(Word16, USDCents)] -> Map Int USDCents + tierMap = M.fromList . map (\(len, price) -> (fromIntegral len, price)) + +-- | Why the registry holds a name back. +data NameReservedReason + = -- | held for SimpleX + NRRInternal + | NRRTrademark + | NRRCommunity + | -- | a reason added to the registry after this version: still reserved, and + -- carries its own word so a later version can name it + NRRUnknown Text + deriving (Eq, Show) + +-- | One vocabulary, shared by the wire, the backing resolver and the JSON API. +instance StrEncoding NameReservedReason where + strEncode = \case + NRRInternal -> "internal" + NRRTrademark -> "trademark" + NRRCommunity -> "community" + NRRUnknown t -> encodeUtf8 t + strP = reservedReasonOf . safeDecodeUtf8 <$> A.takeTill (== ' ') + where + reservedReasonOf = \case + "internal" -> NRRInternal + "trademark" -> NRRTrademark + "community" -> NRRCommunity + t -> NRRUnknown t + +instance Encoding NameReservedReason where + smpEncode = strEncode + smpP = strP + +-- | A v20/v21 router's answer: the name resolves, and nothing else was said. +oldRegistration :: NameRecord -> NameRegistration +oldRegistration nameRecord = + NRRegistered {expires = Nothing, graceUntil = Nothing, reservedReason_ = Nothing, nameRecord} + + -- | Name resolution error data NameErrorType = -- | the names role / resolver is not configured on this server @@ -1822,7 +2010,9 @@ instance PartyI p => ProtocolEncoding SMPVersion ErrorType (Command p) where PRXY host auth_ -> e (PRXY_, ' ', host, auth_) PFWD fwdV pubKey (EncTransmission s) -> e (PFWD_, ' ', fwdV, pubKey, Tail s) RFWD (EncFwdTransmission s) -> e (RFWD_, ' ', Tail s) - RSLV d -> e (RSLV_, ' ', d) + RSLV q + | v >= nameAvailSMPVersion -> e (RSLV_, ' ', q) + | otherwise -> e (RSLV_, ' ', queryDomain q) where e :: Encoding a => a -> ByteString e = smpEncode @@ -1929,7 +2119,9 @@ instance ProtocolEncoding SMPVersion ErrorType Cmd where CT SNotifierService NSUBS_ | v >= rcvServiceSMPVersion -> Cmd SNotifierService <$> (NSUBS <$> _smpP <*> smpP) | otherwise -> pure $ Cmd SNotifierService $ NSUBS (-1) mempty - CT SResolver RSLV_ -> Cmd SResolver . RSLV <$> _smpP <* A.takeByteString + CT SResolver RSLV_ + | v >= nameAvailSMPVersion -> Cmd SResolver . RSLV <$> _smpP <* A.takeByteString + | otherwise -> Cmd SResolver . RSLV . nameQuery v <$> _smpP <* A.takeByteString fromProtocolError = fromProtocolError @SMPVersion @ErrorType @BrokerMsg {-# INLINE fromProtocolError #-} @@ -1972,7 +2164,11 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where | v < clientNoticesSMPVersion -> BLOCKED info {notice = Nothing} _ -> err PONG -> e PONG_ - RNAME rec -> e (RNAME_, ' ', Tail $ LB.toStrict $ J.encode rec) + RNAME reg + | v >= nameAvailSMPVersion -> e (RNAME_, ' ', reg) + | otherwise -> case reg of + NRRegistered {nameRecord} -> e (RNAME_, ' ', Tail $ LB.toStrict $ J.encode nameRecord) + _ -> e (ERR_, ' ', NAME NOT_FOUND) where e :: Encoding a => a -> ByteString e = smpEncode @@ -2019,7 +2215,9 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where OK_ -> pure OK ERR_ -> ERR <$> _smpP PONG_ -> pure PONG - RNAME_ -> fmap RNAME . J.eitherDecodeStrict . unTail <$?> _smpP + RNAME_ + | v >= nameAvailSMPVersion -> RNAME <$> _smpP + | otherwise -> fmap (RNAME . oldRegistration) . J.eitherDecodeStrict . unTail <$?> _smpP where serviceRespP resp | v >= rcvServiceSMPVersion = resp <$> _smpP <*> smpP @@ -2042,7 +2240,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where PKEY {} -> noEntityMsg RRES _ -> noEntityMsg ALLS -> noEntityMsg - RNAME _ -> noEntityMsg + RNAME {} -> noEntityMsg -- other broker responses must have queue ID _ | B.null entId -> Left $ CMD NO_ENTITY @@ -2412,3 +2610,10 @@ $(J.deriveJSON defaultJSON ''BlockingInfo) -- run deriveJSON in one TH splice to allow mutual instance $(concat <$> mapM @[] (J.deriveJSON (sumTypeJSON id)) [''ProxyError, ''NameErrorType, ''ErrorType]) + +instance ToJSON NameReservedReason where + toJSON = strToJSON + toEncoding = strToJEncoding + +instance FromJSON NameReservedReason where + parseJSON = strParseJSON "NameReservedReason" diff --git a/src/Simplex/Messaging/Server.hs b/src/Simplex/Messaging/Server.hs index 16ad58ab3..fd461dfb3 100644 --- a/src/Simplex/Messaging/Server.hs +++ b/src/Simplex/Messaging/Server.hs @@ -1494,12 +1494,12 @@ client Just nenv -> pure (Just nenv) -- Runs on a forked thread so RSLV does not block other commands; -- concurrency is limited by serverResolverConcurrency in forkCmd. - resolveNameMsg :: NamesEnv -> SimplexDomain -> M s BrokerMsg + resolveNameMsg :: NamesEnv -> NameQuery -> M s BrokerMsg resolveNameMsg nenv d = do st <- asks (rslvStats . serverStats) (selector, msg) <- liftIO (resolveName nenv d) <&> \case - Right rec -> (rslvSucc, RNAME rec) + Right reg -> (rslvSucc, RNAME reg) Left e@NOT_FOUND -> (rslvNotFound, ERR $ NAME e) Left e -> (rslvResolverErrs, ERR $ NAME e) incStat (selector st) $> msg diff --git a/src/Simplex/Messaging/Server/Names.hs b/src/Simplex/Messaging/Server/Names.hs index 856339bc8..f360789cf 100644 --- a/src/Simplex/Messaging/Server/Names.hs +++ b/src/Simplex/Messaging/Server/Names.hs @@ -17,12 +17,16 @@ where import qualified Control.Exception as E import Control.Logger.Simple (logError) -import Data.Bifunctor (first) +import qualified Data.Map.Strict as M import Data.Maybe (fromMaybe) +import Data.Text (Text) import qualified Data.Text as T -import Simplex.Messaging.Protocol (NameErrorType (..), NameRecord) +import Data.Text.Encoding (encodeUtf8) +import Simplex.Messaging.Encoding.String (strDecode) +import Simplex.Messaging.Protocol (NameErrorType (..), NamePricing (..), NameQuery, NameRecord, NameRegistration (..), NameReservedReason (..), USDCents (..), oldRegistration, queryName) import Simplex.Messaging.Server.Names.HttpResolver - ( ResolverEnv, + ( NameStatusResp (..), + ResolverEnv, ResolverError (..), RpcAuth (..), closeResolverEnv, @@ -30,7 +34,7 @@ import Simplex.Messaging.Server.Names.HttpResolver newResolverEnv, resolveHttp, ) -import Simplex.Messaging.SimplexName (SimplexDomain, fullDomainName) +import Simplex.Messaging.SystemTime (RoundedSystemTime (..)) import System.Timeout (timeout) data NamesConfig = NamesConfig @@ -58,9 +62,9 @@ pingEndpoint :: NamesEnv -> IO (Either ResolverError ()) pingEndpoint NamesEnv {resolverEnv, config} = fromMaybe (Left ResolverTimeout) <$> timeout (resolverTimeoutMs config * 1000) (healthHttp resolverEnv) -resolveName :: NamesEnv -> SimplexDomain -> IO (Either NameErrorType NameRecord) -resolveName env d = do - r <- E.try (timeout (resolverTimeoutMs (config env) * 1000) (fetch env d)) +resolveName :: NamesEnv -> NameQuery -> IO (Either NameErrorType NameRegistration) +resolveName env q = do + r <- E.try (timeout (resolverTimeoutMs (config env) * 1000) (fetch env q)) case r of Right result -> pure (fromMaybe (Left (RESOLVER "timeout")) result) Left e @@ -69,15 +73,59 @@ resolveName env d = do logError $ "[NAMES] resolver fetch raised " <> T.pack (E.displayException e) pure (Left (RESOLVER "resolver error")) -fetch :: NamesEnv -> SimplexDomain -> IO (Either NameErrorType NameRecord) -fetch NamesEnv {resolverEnv} d = - first mapResolverError <$> resolveHttp resolverEnv (fullDomainName d) +fetch :: NamesEnv -> NameQuery -> IO (Either NameErrorType NameRegistration) +fetch NamesEnv {resolverEnv} q = + either (Left . mapResolverError) nameRegistration <$> resolveHttp resolverEnv (queryName q) + +-- | A resolver that reports no status is an older one, which returned a record +-- only for a live registration. +nameRegistration :: (Maybe NameRecord, Maybe NameStatusResp) -> Either NameErrorType NameRegistration +nameRegistration = \case + (rec_, Just ns) -> mapStatus rec_ ns + (Just rec, Nothing) -> Right (oldRegistration rec) + (Nothing, Nothing) -> Left NOT_FOUND + +-- | The resolver's status words. An unknown status is not an answer. +mapStatus :: Maybe NameRecord -> NameStatusResp -> Either NameErrorType NameRegistration +mapStatus rec_ ns@NameStatusResp {nsStatus, nsExpires, nsGraceEnds, nsReasonCode, nsAuctionUntil} = + case nsStatus of + "registered" -> registered + "grace" -> registered + "unregistered" -> available + "expired" -> available + s -> Left (RESOLVER (T.take 32 s)) + where + reservedReason_ = resolverReason <$> nsReasonCode + -- A registered name always has a record, and a registration this router + -- cannot date is not one it can report. + registered = case (rec_, nsExpires, nsGraceEnds) of + (Just nameRecord, Just expires, Just graceUntil) -> + Right NRRegistered {expires = Just (RoundedSystemTime expires), graceUntil = Just (RoundedSystemTime graceUntil), reservedReason_, nameRecord} + (Nothing, _, _) -> Left (RESOLVER "no record") + _ -> Left (RESOLVER "no expiry") + available = case reservedReason_ of + Just r -> Right (NRReserved r) + Nothing -> case namePricing ns of + Just pricing -> Right NRAvailable {pricing, auctionUntil = RoundedSystemTime <$> nsAuctionUntil} + Nothing -> Left (RESOLVER "no price oracle") + +-- | An unknown code still reserves the name, and travels on as itself. Cut to +-- one printable token: the wire slot it goes into ends at a space. +resolverReason :: Text -> NameReservedReason +resolverReason t = either (const (NRRUnknown t')) id (strDecode (encodeUtf8 t')) + where + t' = T.take 32 (T.takeWhile (\c -> c > ' ' && c < '\DEL') t) + +namePricing :: NameStatusResp -> Maybe NamePricing +namePricing NameStatusResp {nsRentPrices, nsBasePrice, nsMinLabelLength} = do + rentPrices <- M.map USDCents <$> nsRentPrices + basePrice <- USDCents <$> nsBasePrice + minLabelLength <- nsMinLabelLength + pure NamePricing {rentPrices, basePrice, minLabelLength} mapResolverError :: ResolverError -> NameErrorType mapResolverError = \case HttpStatusErr 404 -> NOT_FOUND - -- 410 is a lapsed registration: an answer about the name, not a resolver - -- failure, so it must not become RESOLVER. HttpStatusErr 410 -> NOT_FOUND HttpStatusErr 400 -> NOT_FOUND HttpStatusErr code -> RESOLVER ("HTTP " <> T.pack (show code)) diff --git a/src/Simplex/Messaging/Server/Names/HttpResolver.hs b/src/Simplex/Messaging/Server/Names/HttpResolver.hs index 118810a08..69d2c4e65 100644 --- a/src/Simplex/Messaging/Server/Names/HttpResolver.hs +++ b/src/Simplex/Messaging/Server/Names/HttpResolver.hs @@ -3,14 +3,17 @@ {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE StrictData #-} +{-# LANGUAGE TemplateHaskell #-} +{-# LANGUAGE TupleSections #-} -- | HTTP transport for the public-namespace resolver. -- -- The Python REST resolver (see scripts/resolver/snrc-resolve.py) exposes -- -- GET /resolve/ -> 200 with a NameRecord JSON document --- 404 / 400 for unknown names / TLDs --- 502 for upstream RPC failures +-- 404 / 410 for names that do not resolve, the body +-- saying why (reserved, lapsed, never registered) +-- 400 for unknown TLDs, 502 for upstream RPC failures -- GET /health -> 200 when the resolver process is ready -- -- Boundary properties: @@ -24,6 +27,7 @@ module Simplex.Messaging.Server.Names.HttpResolver ( RpcAuth (..), ResolverEnv, ResolverError (..), + NameStatusResp (..), newResolverEnv, closeResolverEnv, resolveHttp, @@ -33,11 +37,17 @@ where import qualified Control.Exception as E import qualified Data.Aeson as J +import Data.Aeson.Key (Key) +import qualified Data.Aeson.KeyMap as JKM +import qualified Data.Aeson.TH as JQ +import qualified Data.Aeson.Types as JT import Data.Bifunctor (first) import qualified Data.ByteArray.Encoding as BAE import Data.ByteString.Char8 (ByteString) import qualified Data.ByteString.Char8 as B import qualified Data.ByteString.Lazy as BL +import Data.Int (Int64) +import Data.Map.Strict (Map) import Data.Text (Text) import Data.Text.Encoding (encodeUtf8) import Network.HTTP.Client @@ -58,6 +68,7 @@ import Network.HTTP.Client.TLS (tlsManagerSettings) import qualified Network.HTTP.Types as HT import Network.HTTP.Types.URI (urlEncode) import Simplex.Messaging.Names.Record (NameRecord) +import Simplex.Messaging.Parsers (defaultJSON, dropPrefix) data RpcAuth = AuthBearer Text | AuthBasic Text Text @@ -75,6 +86,25 @@ data ResolverEnv = ResolverEnv maxResponseBytes :: Int } +-- | What the resolver says about a name. Only some statuses carry the fields +-- below the status. +data NameStatusResp = NameStatusResp + { nsStatus :: Text, + nsExpires :: Maybe Int64, + nsGraceEnds :: Maybe Int64, + nsReasonCode :: Maybe Text, + -- | when the post-grace surcharge decays to nothing + nsAuctionUntil :: Maybe Int64, + -- | US cents per year, by label length + nsRentPrices :: Maybe (Map Int Int64), + -- | US cents per year for every other length + nsBasePrice :: Maybe Int64, + nsMinLabelLength :: Maybe Int + } + deriving (Show) + +$(JQ.deriveFromJSON defaultJSON {J.fieldLabelModifier = dropPrefix "ns"} ''NameStatusResp) + data ResolverError = HttpFailure HttpException | HttpStatusErr Int @@ -108,24 +138,42 @@ authHeader = \case let encoded = BAE.convertToBase BAE.Base64 (encodeUtf8 u <> ":" <> encodeUtf8 p) :: ByteString in ("Authorization", "Basic " <> encoded) --- | GET /resolve/, decoding the 200 body --- directly into a NameRecord in one pass (no intermediate Aeson Value). The --- name is percent-encoded (every non-unreserved byte per RFC 3986): the --- resolver expects raw labels, so slashes/punctuation must not alter the path. -resolveHttp :: ResolverEnv -> Text -> IO (Either ResolverError NameRecord) +-- | GET /resolve/, returning the record when the +-- name resolves and what the resolver says about the name either way. The status +-- code cannot tell an unregistered name from a reserved or lapsed one, so on the +-- two codes that carry availability the body is read as well. The name is +-- percent-encoded (every non-unreserved byte per RFC 3986): the resolver expects +-- raw labels, so slashes/punctuation must not alter the path. +resolveHttp :: ResolverEnv -> Text -> IO (Either ResolverError (Maybe NameRecord, Maybe NameStatusResp)) resolveHttp env name = - (>>= first InvalidJson . J.eitherDecodeStrict . BL.toStrict) - <$> httpGet env ("/resolve/" <> B.unpack (urlEncode True (encodeUtf8 name))) + (>>= nameResp) <$> httpGet env ("/resolve/" <> B.unpack (urlEncode True (encodeUtf8 name))) + where + nameResp (status, bs) + | status < 400 = (,statusResp bs "status") . Just <$> first InvalidJson (J.eitherDecode bs) + | status == 404 || status == 410 = + maybe (Left $ HttpStatusErr status) (Right . (Nothing,) . Just) (statusResp bs "error") + | otherwise = Left (HttpStatusErr status) + +-- | What the resolver says about the name, under "status" on a 200 and "error" +-- on the codes that carry availability. Older resolvers send neither. +statusResp :: BL.ByteString -> Key -> Maybe NameStatusResp +statusResp bs k = case J.decode bs of + Just (J.Object o) -> do + v <- JKM.lookup k o + JT.parseMaybe J.parseJSON (J.Object (JKM.insert "status" v o)) + _ -> Nothing -- | GET /health; success = reachable with status < 400. The body is -- size-capped but NOT decoded — the probe only checks reachability. healthHttp :: ResolverEnv -> IO (Either ResolverError ()) -healthHttp env = (() <$) <$> httpGet env "/health" +healthHttp env = (>>= statusOk . fst) <$> httpGet env "/health" + where + statusOk status = if status >= 400 then Left (HttpStatusErr status) else Right () --- | GET , returning the response body bytes on status < 400 --- within the size cap. Redirects are disabled and Authorization is attached --- only when configured. -httpGet :: ResolverEnv -> String -> IO (Either ResolverError BL.ByteString) +-- | GET , returning the response status and body bytes within the +-- size cap. Redirects are disabled and Authorization is attached only when +-- configured. +httpGet :: ResolverEnv -> String -> IO (Either ResolverError (Int, BL.ByteString)) httpGet ResolverEnv {manager, baseUrl, authHdr, timeoutMicro, maxResponseBytes} path = do req0 <- parseRequest (baseUrl <> path) let req = @@ -136,9 +184,6 @@ httpGet ResolverEnv {manager, baseUrl, authHdr, timeoutMicro, maxResponseBytes} } result <- E.try $ withResponse req manager $ \res -> do let status = HT.statusCode (responseStatus res) - if status >= 400 - then pure (Left (HttpStatusErr status)) - else do - bs <- brReadSome (responseBody res) (maxResponseBytes + 1) - pure $ if BL.length bs > fromIntegral maxResponseBytes then Left BodyTooLarge else Right bs + bs <- brReadSome (responseBody res) (maxResponseBytes + 1) + pure $ if BL.length bs > fromIntegral maxResponseBytes then Left BodyTooLarge else Right (status, bs) pure (either (Left . HttpFailure) id result) diff --git a/src/Simplex/Messaging/Server/Prometheus.hs b/src/Simplex/Messaging/Server/Prometheus.hs index 575f699c6..201f88426 100644 --- a/src/Simplex/Messaging/Server/Prometheus.hs +++ b/src/Simplex/Messaging/Server/Prometheus.hs @@ -469,7 +469,7 @@ prometheusMetrics sm rtm ts = \# TYPE simplex_smp_names_reqs counter\n\ \simplex_smp_names_reqs " <> mshow _rslvReqs <> "\n# rslvReqs\n\ \\n\ - \# HELP simplex_smp_names_success NameRecord successfully resolved and returned.\n\ + \# HELP simplex_smp_names_success NameRecord resolved, or availability answered.\n\ \# TYPE simplex_smp_names_success counter\n\ \simplex_smp_names_success " <> mshow _rslvSucc <> "\n# rslvSucc\n\ \\n\ diff --git a/src/Simplex/Messaging/SimplexName.hs b/src/Simplex/Messaging/SimplexName.hs index 2dd0f8645..c81dbd4cb 100644 --- a/src/Simplex/Messaging/SimplexName.hs +++ b/src/Simplex/Messaging/SimplexName.hs @@ -10,14 +10,21 @@ module Simplex.Messaging.SimplexName SimplexTLD (..), SimplexNameType (..), fullDomainName, + LabelHash (..), + labelHash, + labelHashText, shortNameInfoStr, ) where import Control.Applicative (optional, (<|>)) +import Crypto.Hash (Digest, hash) +import Crypto.Hash.Algorithms (Keccak_256) import qualified Data.Aeson.TH as J import qualified Data.Attoparsec.ByteString.Char8 as A import qualified Data.Attoparsec.Text as AT +import qualified Data.ByteArray as BA +import qualified Data.ByteArray.Encoding as BAE import Data.ByteString.Char8 (ByteString) import qualified Data.ByteString.Char8 as B import Data.Char (isDigit) @@ -70,6 +77,24 @@ nameLabelP = do -- (Cyrillic а vs ASCII a hash to different on-chain records). isNameLetter c = c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' +-- | The registry's key for a label, and what +-- BaseRegistrarImplementation.labelOf takes. Always 32 bytes. +newtype LabelHash = LabelHash ByteString + deriving (Eq, Show) + +instance Encoding LabelHash where + smpEncode (LabelHash h) = h + smpP = LabelHash <$> A.take 32 + +-- | keccak-256 of the lowercased label, as the registry keys it. +labelHash :: Text -> LabelHash +labelHash label = LabelHash $ BA.convert (hash (encodeUtf8 (T.toLower label)) :: Digest Keccak_256) + +-- | ENS's encoding for a label whose text is unknown, which is what the +-- backing resolver's HTTP API takes. +labelHashText :: LabelHash -> Text +labelHashText (LabelHash h) = "[" <> decodeLatin1 (BAE.convertToBase BAE.Base16 h) <> "]" + -- | Cap the name at 253 bytes (DNS full-domain limit) boundedNonSpace :: A.Parser ByteString boundedNonSpace = do @@ -107,6 +132,18 @@ instance Encoding SimplexDomain where smpEncode = strEncode smpP = strP +instance Encoding SimplexTLD where + smpEncode = \case + TLDSimplex -> "s" + TLDTesting -> "t" + TLDWeb -> "w" + smpP = + A.anyChar >>= \case + 's' -> pure TLDSimplex + 't' -> pure TLDTesting + 'w' -> pure TLDWeb + _ -> fail "bad SimplexTLD" + fullDomainName :: SimplexDomain -> Text fullDomainName SimplexDomain {nameTLD, domain, subDomain} = T.intercalate "." (reverse subDomain ++ [domain] ++ tld') where diff --git a/src/Simplex/Messaging/SystemTime.hs b/src/Simplex/Messaging/SystemTime.hs index d53a60aa1..18d5ffb50 100644 --- a/src/Simplex/Messaging/SystemTime.hs +++ b/src/Simplex/Messaging/SystemTime.hs @@ -23,11 +23,12 @@ import Data.Time.Clock.System (SystemTime (..), getSystemTime, systemToUTCTime) import Data.Typeable (Proxy (..)) import GHC.TypeLits (KnownNat, Nat, natVal) import Simplex.Messaging.Agent.Store.DB (FromField (..), ToField (..)) +import Simplex.Messaging.Encoding (Encoding) import Simplex.Messaging.Encoding.String newtype RoundedSystemTime (t :: Nat) = RoundedSystemTime {roundedSeconds :: Int64} deriving (Eq, Ord, Show) - deriving newtype (FromJSON, ToJSON, FromField, ToField) + deriving newtype (Encoding, FromJSON, ToJSON, FromField, ToField) type SystemDate = RoundedSystemTime 86400 diff --git a/src/Simplex/Messaging/Transport.hs b/src/Simplex/Messaging/Transport.hs index a366e79f7..d2c30d25a 100644 --- a/src/Simplex/Messaging/Transport.hs +++ b/src/Simplex/Messaging/Transport.hs @@ -53,6 +53,7 @@ module Simplex.Messaging.Transport rcvServiceSMPVersion, namesSMPVersion, serverInfoSMPVersion, + nameAvailSMPVersion, simplexMQVersion, smpBlockSize, TransportConfig (..), @@ -175,6 +176,7 @@ smpBlockSize = 16384 -- 19 - service subscriptions to messages (10/20/2025) -- 20 - public namespaces resolver, RSLV command (6/20/2026) -- 21 - server public information in handshake (7/5/2026) +-- 22 - RNAME answers name availability as well as the record (7/25/2026) data SMPVersion @@ -211,6 +213,11 @@ namesSMPVersion = VersionSMP 20 serverInfoSMPVersion :: VersionSMP serverInfoSMPVersion = VersionSMP 21 +-- | RNAME carries availability. A server below this answers RSLV with the +-- record alone, and ERR NAME NOT_FOUND for a name that does not resolve. +nameAvailSMPVersion :: VersionSMP +nameAvailSMPVersion = VersionSMP 22 + minClientSMPRelayVersion :: VersionSMP minClientSMPRelayVersion = VersionSMP 14 @@ -218,20 +225,20 @@ minServerSMPRelayVersion :: VersionSMP minServerSMPRelayVersion = VersionSMP 14 currentClientSMPRelayVersion :: VersionSMP -currentClientSMPRelayVersion = VersionSMP 21 +currentClientSMPRelayVersion = VersionSMP 22 currentServerSMPRelayVersion :: VersionSMP -currentServerSMPRelayVersion = VersionSMP 21 +currentServerSMPRelayVersion = VersionSMP 22 -- Max SMP protocol version to be used in e2e encrypted connection between -- client and server, as defined by SMP proxy. Normally set below the current -- version to prevent client version fingerprinting by the destination relays --- when clients upgrade at different times. Pinned to the current version (20) --- for this release because proxied name resolution is gated on namesSMPVersion --- (20), so the one-version anti-fingerprinting buffer does not apply yet; it --- reappears once the current version advances past 20. +-- when clients upgrade at different times. Pinned to the current version (22) +-- for this release because a proxied RSLV only carries availability from +-- nameAvailSMPVersion (22), so the one-version anti-fingerprinting buffer does +-- not apply yet; it reappears once the current version advances past 22. proxiedSMPRelayVersion :: VersionSMP -proxiedSMPRelayVersion = VersionSMP 20 +proxiedSMPRelayVersion = VersionSMP 22 -- minimal supported protocol version is 14 supportedClientSMPRelayVRange :: VersionRangeSMP diff --git a/tests/AgentTests/ResolveNameTests.hs b/tests/AgentTests/ResolveNameTests.hs index f55faf75f..dd4e23c59 100644 --- a/tests/AgentTests/ResolveNameTests.hs +++ b/tests/AgentTests/ResolveNameTests.hs @@ -86,6 +86,21 @@ resolveNameTests = do it "surfaces as SMP host (NAME (RESOLVER ..))" testBackendError describe "success path" $ it "returns NameRecord" testDirectSuccess + describe "name availability" $ + it "an unregistered name answers as available" testAvailSuccess + +testAvailSuccess :: HasCallStack => IO () +testAvailSuccess = + withDirectResolver (status404, availableBody) $ \c -> do + r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" []) + case r of + Right (SMP.NRAvailable {}) -> pure () + _ -> expectationFailure $ "expected Right NRAvailable, got: " <> show r + +-- an unregistered name is only available if the resolver also priced it +availableBody :: LB.ByteString +availableBody = + "{\"error\":\"unregistered\",\"rentPrices\":{\"3\":12793,\"4\":3198},\"basePrice\":100,\"minLabelLength\":3}" testDirectNotFound :: HasCallStack => IO () testDirectNotFound = @@ -148,5 +163,5 @@ testDirectSuccess = withDirectResolver (status200, J.encode testNameRecord) $ \c -> do r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" []) case r of - Right nr -> nr `shouldBe` testNameRecord - _ -> expectationFailure $ "expected Right NameRecord, got: " <> show r + Right (SMP.NRRegistered {nameRecord}) -> nameRecord `shouldBe` testNameRecord + _ -> expectationFailure $ "expected Right NRRegistered, got: " <> show r diff --git a/tests/RSLVTests.hs b/tests/RSLVTests.hs index d453c5553..a199dbb31 100644 --- a/tests/RSLVTests.hs +++ b/tests/RSLVTests.hs @@ -15,6 +15,8 @@ import Control.Monad.Trans.Except (ExceptT, runExceptT) import qualified Data.Aeson as J import qualified Data.ByteString.Char8 as B import qualified Data.ByteString.Lazy as LB +import qualified Data.Map.Strict as M +import Data.IORef (IORef, readIORef) import Data.List.NonEmpty (NonEmpty (..)) import Data.Text (Text) import Data.Text.Encoding (encodeUtf8) @@ -33,7 +35,12 @@ import Simplex.Messaging.Protocol Command (..), CorrId (..), ErrorType (..), + NamePricing (..), + NameRegistration (..), + NameReservedReason (..), + USDCents (..), NameErrorType (..), + NameReservedReason (..), SParty (..), Transmission, TransmissionForAuth (..), @@ -44,6 +51,7 @@ import Simplex.Messaging.Protocol ) import qualified Simplex.Messaging.Protocol as SMP import Simplex.Messaging.SimplexName (SimplexDomain) +import Simplex.Messaging.SystemTime (RoundedSystemTime (..)) import Simplex.Messaging.Transport import Simplex.Messaging.Version (mkVersionRange) import Test.Hspec hiding (fit, it) @@ -57,6 +65,11 @@ withResolverServer (st, body) runTest = NRS.withResolverServer (NRS.resolveResp st body) $ \port _ -> withSmpServerConfigOn (transport @TLS) (withNames port memCfg) testPort (const runTest) +withResolverServerReqs :: (Status, LB.ByteString) -> (IORef [[Text]] -> IO a) -> IO a +withResolverServerReqs (st, body) runTest = + NRS.withResolverServer (NRS.resolveResp st body) $ \port reqs -> + withSmpServerConfigOn (transport @TLS) (withNames port memCfg) testPort (const (runTest reqs)) + withProxyAndResolver :: (Status, LB.ByteString) -> IO a -> IO a withProxyAndResolver (st, body) runTest = NRS.withResolverServer (NRS.resolveResp st body) $ \port _ -> @@ -65,7 +78,7 @@ withProxyAndResolver (st, body) runTest = sendRslv :: Transport c => THandleSMP c 'TClient -> B.ByteString -> SimplexDomain -> IO (Transmission (Either ErrorType BrokerMsg)) sendRslv h@THandle {params} corrId d = do - let TransmissionForAuth {tToSend} = encodeTransmissionForAuth params (CorrId corrId, NoEntity, Cmd SResolver (RSLV d)) + let TransmissionForAuth {tToSend} = encodeTransmissionForAuth params (CorrId corrId, NoEntity, Cmd SResolver (RSLV (SMP.nameQuery currentClientSMPRelayVersion d))) [Right ()] <- tPut h (Right (Nothing, tToSend) :| []) r :| _ <- tGetClient h pure r @@ -83,6 +96,18 @@ rslvTests = do it "PFWD-wrapped RSLV success returns RNAME (record JSON frames over the proxy)" testRslvForwardedSuccess describe "RSLV success path (RNAME response)" $ do it "returns RNAME with NameRecord" testRslvSuccess + describe "RSLV availability (RNAME response)" $ do + it "unregistered comes back AVAILABLE" testRslvAvailable + it "auction comes back with premium" testRslvAuction + it "reserved comes back with the reason" testRslvReserved + it "PFWD-wrapped auction reaches the resolver" testRslvForwardedAuction + describe "RSLV below v22" $ do + it "still resolves a name to its record" testRslvOldClientRecord + it "still answers NAME NOT_FOUND for a name that does not resolve" testRslvOldClientNotFound + describe "hashed lookups" $ do + it "RSLV sends the 2LD as its hash" testRslvSendsTheHash + it "subname labels stay text" testSubnameKeepsItsLabels + it "a record naming a different name is rejected" testRslvWrongName testRslvBackendNotFound :: IO () testRslvBackendNotFound = @@ -127,7 +152,7 @@ testRslvVersion = Left (PCETransportError TEVersion) -> pure () _ -> expectationFailure $ "expected Left (PCETransportError TEVersion), got: " <> show r -forwardedResolveAlice :: IO (Either SMPClientError (Either ProxyClientError SMP.NameRecord)) +forwardedResolveAlice :: IO (Either SMPClientError (Either ProxyClientError SMP.NameRegistration)) forwardedResolveAlice = do g <- C.newRandom ts <- getCurrentTime @@ -150,8 +175,8 @@ testRslvForwardedSuccess :: IO () testRslvForwardedSuccess = withProxyAndResolver (status200, J.encode testNameRecord) $ forwardedResolveAlice >>= \r -> case r of - Right (Right nr) -> nr `shouldBe` testNameRecord - _ -> expectationFailure $ "expected Right (Right NameRecord), got: " <> show r + Right (Right NRRegistered {nameRecord}) -> nameRecord `shouldBe` testNameRecord + _ -> expectationFailure $ "expected Right (Right NRRegistered), got: " <> show r testRslvSuccess :: IO () testRslvSuccess = @@ -160,8 +185,132 @@ testRslvSuccess = (corrId, _entId, resp) <- sendRslv h "rs07" (domain "alice.simplex") corrId `shouldBe` CorrId "rs07" case resp of - Right (RNAME nr) -> nr `shouldBe` testNameRecord - _ -> expectationFailure $ "expected Right (RNAME ..), got: " <> show resp + Right (RNAME NRRegistered {nameRecord}) -> nameRecord `shouldBe` testNameRecord + _ -> expectationFailure $ "expected Right (RNAME NRRegistered), got: " <> show resp + +testRslvAvailable :: IO () +testRslvAvailable = + withResolverServer (status404, availableBody) $ + testSMPClient @TLS $ \h -> do + (corrId, _entId, resp) <- sendRslv h "na01" (domain "ghost.simplex") + corrId `shouldBe` CorrId "na01" + resp `shouldBe` Right (RNAME (NRAvailable auctionPricing Nothing)) + +testRslvAuction :: IO () +testRslvAuction = + withResolverServer (status410, auctionBody) $ + testSMPClient @TLS $ \h -> do + (_, _, resp) <- sendRslv h "na02" (domain "lapsed.simplex") + resp `shouldBe` Right (RNAME (NRAvailable auctionPricing (Just (RoundedSystemTime 1790294400)))) + +testRslvReserved :: IO () +testRslvReserved = + withResolverServer (status404, "{\"error\":\"unregistered\",\"reasonCode\":\"trademark\"}") $ + testSMPClient @TLS $ \h -> do + (_, _, resp) <- sendRslv h "na03" (domain "acme.simplex") + resp `shouldBe` Right (RNAME (NRReserved NRRTrademark)) + +-- | A client that predates v22 must see exactly what it saw before: the record +-- for a name that resolves, and NOT_FOUND for one that does not. +oldClient :: IO SMPClient +oldClient = do + g <- C.newRandom + ts <- getCurrentTime + let srv = SMPServer testHost testPort testKeyHash + -- the version just below the gate: a lower ceiling would pass even if + -- the gate were at 20 or 21 + oldCfg = defaultSMPClientConfig {serverVRange = mkVersionRange minServerSMPRelayVersion serverInfoSMPVersion} + pcE <- getProtocolClient g NRMInteractive (1, srv, Nothing) oldCfg [] Nothing ts (\_ -> pure ()) + either (fail . show) pure pcE + +testRslvOldClientRecord :: IO () +testRslvOldClientRecord = + withResolverServer (status200, J.encode testNameRecord) $ do + pc <- oldClient + r <- runExceptT' (directResolveName pc NRMInteractive (domain "alice.simplex")) + r `shouldBe` NRRegistered Nothing Nothing Nothing testNameRecord + +testRslvOldClientNotFound :: IO () +testRslvOldClientNotFound = + withResolverServer (status404, availableBody) $ do + pc <- oldClient + r <- runExceptT (directResolveName pc NRMInteractive (domain "alice.simplex")) + case r of + Left (PCEProtocolError (SMP.NAME SMP.NOT_FOUND)) -> pure () + _ -> expectationFailure $ "expected Left (PCEProtocolError (NAME NOT_FOUND)), got: " <> show r + +testRslvForwardedAuction :: IO () +testRslvForwardedAuction = + withProxyAndResolver (status410, auctionBody) $ + forwardedResolveAlice >>= \r -> case r of + Right (Right (NRAvailable _ auctionUntil)) -> auctionUntil `shouldBe` Just (RoundedSystemTime 1790294400) + _ -> expectationFailure $ "expected Right (Right NRAvailable), got: " <> show r + +pricingJson :: LB.ByteString +pricingJson = "\"rentPrices\":{\"3\":12793,\"4\":3198},\"basePrice\":100,\"minLabelLength\":3" + +availableBody :: LB.ByteString +availableBody = "{\"error\":\"unregistered\"," <> pricingJson <> "}" + +-- a name past its grace period, still inside the window where it costs a +-- surcharge above the ordinary price +auctionBody :: LB.ByteString +auctionBody = "{\"error\":\"expired\",\"auctionUntil\":1790294400," <> pricingJson <> "}" + +auctionPricing :: NamePricing +auctionPricing = + NamePricing + { rentPrices = M.fromList [(3, USDCents 12793), (4, USDCents 3198)], + basePrice = USDCents 100, + minLabelLength = 3 + } + +-- keccak-256("alice"), the registry key +aliceHash :: Text +aliceHash = "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" + +-- | The paths the client asked the resolver for. +resolvePaths :: IORef [[Text]] -> IO [[Text]] +resolvePaths reqs = filter isResolve <$> readIORef reqs + where + isResolve = \case ("resolve" : _) -> True; _ -> False + +currentClient :: IO SMPClient +currentClient = do + g <- C.newRandom + ts <- getCurrentTime + let srv = SMPServer testHost testPort testKeyHash + pcE <- getProtocolClient g NRMInteractive (1, srv, Nothing) defaultSMPClientConfig [] Nothing ts (\_ -> pure ()) + either (fail . show) pure pcE + +testRslvSendsTheHash :: IO () +testRslvSendsTheHash = + withResolverServerReqs (status200, J.encode testNameRecord) $ \reqs -> do + pc <- currentClient + r <- runExceptT' (directResolveName pc NRMInteractive (domain "alice.simplex")) + resolvePaths reqs `shouldReturn` [["resolve", aliceHash <> ".simplex"]] + -- the client never sent the name, and the record still names it + case r of + NRRegistered {nameRecord} -> SMP.nrName nameRecord `shouldBe` "alice.simplex" + _ -> expectationFailure $ "expected NRRegistered, got: " <> show r + +testSubnameKeepsItsLabels :: IO () +testSubnameKeepsItsLabels = + withResolverServerReqs (status404, availableBody) $ \reqs -> do + pc <- currentClient + _ <- runExceptT' (directResolveName pc NRMInteractive (domain "x.alice.simplex")) + resolvePaths reqs `shouldReturn` [["resolve", "x." <> aliceHash <> ".simplex"]] + +-- a hashed query does not tell the router the name, so the record's own name is +-- checked against the one that was asked for +testRslvWrongName :: IO () +testRslvWrongName = + withResolverServer (status200, J.encode testNameRecord {SMP.nrName = "mallory.simplex"}) $ do + pc <- currentClient + r <- runExceptT (directResolveName pc NRMInteractive (domain "alice.simplex")) + case r of + Left (PCEUnexpectedResponse _) -> pure () + _ -> expectationFailure $ "expected Left (PCEUnexpectedResponse ..), got: " <> show r runExceptT' :: Show e => ExceptT e IO a -> IO a runExceptT' a = runExceptT a >>= either (fail . show) pure diff --git a/tests/SMPNamesTests.hs b/tests/SMPNamesTests.hs index 16a332d5f..8c1795cfd 100644 --- a/tests/SMPNamesTests.hs +++ b/tests/SMPNamesTests.hs @@ -11,13 +11,14 @@ import qualified Data.ByteString.Lazy as LB import Data.Either (isLeft, isRight) import Data.IORef (readIORef) import Data.List (sort) +import qualified Data.Map.Strict as M import qualified Data.Text as T import Data.Text.Encoding (encodeUtf8) import Network.HTTP.Types (status200, status400, status404, status410, status500, status502) import NamesResolverServer (resolveResp, testNamesConfig, withResolverServer, withResolverServerDelayed) import Simplex.Messaging.Encoding (smpDecode, smpEncode) -import Simplex.Messaging.Encoding.String (strDecode) -import Simplex.Messaging.Protocol (ErrorType (..), NameErrorType (..), NameRecord (..)) +import Simplex.Messaging.Encoding.String (strDecode, strEncode) +import Simplex.Messaging.Protocol (Command (..), ErrorType (..), NameErrorType (..), NamePricing (..), NameRecord (..), NameRegistration (..), NameReservedReason (..), ProtocolEncoding (..), USDCents (..), nameQuery, queryName) import Simplex.Messaging.Server.Main (validateUrl) import Simplex.Messaging.Server.Names ( NamesConfig (..), @@ -27,7 +28,9 @@ import Simplex.Messaging.Server.Names resolveName, ) import Simplex.Messaging.Server.Names.HttpResolver (ResolverError (..)) -import Simplex.Messaging.SimplexName (SimplexDomain (..), SimplexTLD (..)) +import Simplex.Messaging.SimplexName (SimplexDomain (..), SimplexTLD (..), fullDomainName) +import Simplex.Messaging.SystemTime (RoundedSystemTime (..)) +import Simplex.Messaging.Transport (currentClientSMPRelayVersion, nameAvailSMPVersion, namesSMPVersion, serverInfoSMPVersion) import Test.Hspec testNameRecord :: NameRecord @@ -51,8 +54,10 @@ smpNamesTests :: Spec smpNamesTests = do describe "NameRecord JSON (Protocol)" nameRecordEncodingSpec describe "ErrorType NAME wire encoding" errorWireSpec + describe "RSLV wire encoding" rslvWireSpec describe "Name parsing (SimplexDomain)" parseNameSpec describe "HTTP resolver" resolverSpec + describe "name availability" availabilitySpec describe "Resolver health probe" healthSpec describe "resolver_endpoint validation" validateUrlSpec @@ -101,8 +106,141 @@ errorWireSpec = -- RESOLVER detail may contain spaces - must survive the round-trip smpDecode (smpEncode (NAME (RESOLVER "HTTP 502"))) `shouldBe` Right (NAME (RESOLVER "HTTP 502")) +-- the query format changed at v22, so an older session must still get the name +rslvWireSpec :: Spec +rslvWireSpec = do + it "below v22 carries the name, as it did before" $ + encodeProtocol v20 (RSLV (nameQuery v20 aliceDomain')) `shouldBe` "RSLV " <> smpEncode aliceDomain' + it "from v22 carries the query" $ + encodeProtocol v22 (RSLV (nameQuery v22 aliceDomain')) `shouldBe` "RSLV " <> smpEncode (nameQuery v22 aliceDomain') + where + v20 = serverInfoSMPVersion + v22 = nameAvailSMPVersion + aliceDomain' = SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} + +availabilitySpec :: Spec +availabilitySpec = do + -- one lookup answers what the name points to, whether it can be taken, and + -- whether it is held back + it "a registered name answers with its record and dates" $ + answers status200 (recordWith "\"status\":\"registered\",\"expires\":1813853483,\"graceEnds\":1821629483") $ + NRRegistered {expires = Just (RoundedSystemTime 1813853483), graceUntil = Just (RoundedSystemTime 1821629483), reservedReason_ = Nothing, nameRecord = testNameRecord} + it "a name in grace keeps its record" $ + answers status200 (recordWith "\"status\":\"grace\",\"expires\":1785000000,\"graceEnds\":1792776000") $ + NRRegistered {expires = Just (RoundedSystemTime 1785000000), graceUntil = Just (RoundedSystemTime 1792776000), reservedReason_ = Nothing, nameRecord = testNameRecord} + it "a registered name can be held back too" $ + answers status200 (recordWith "\"status\":\"registered\",\"expires\":1813853483,\"graceEnds\":1821629483,\"reasonCode\":\"internal\"") $ + NRRegistered {expires = Just (RoundedSystemTime 1813853483), graceUntil = Just (RoundedSystemTime 1821629483), reservedReason_ = Just NRRInternal, nameRecord = testNameRecord} + it "a resolver that sends no status still answers with the record" $ + answers status200 (J.encode testNameRecord) $ + NRRegistered {expires = Nothing, graceUntil = Nothing, reservedReason_ = Nothing, nameRecord = testNameRecord} + it "an unregistered name answers with the price" $ + answers status404 (jsonBody ("{\"error\":\"unregistered\"," <> pricingJson <> "}")) $ + NRAvailable {pricing = testPricing, auctionUntil = Nothing} + it "expired is available, counting down to the ordinary price" $ + answers status410 (jsonBody ("{\"error\":\"expired\",\"auctionUntil\":1790294400," <> pricingJson <> "}")) $ + NRAvailable {pricing = testPricing, auctionUntil = Just (RoundedSystemTime 1790294400)} + it "reserved carries the reason and no price" $ + answers status404 (jsonBody ("{\"error\":\"unregistered\",\"reasonCode\":\"trademark\"," <> pricingJson <> "}")) $ + NRReserved NRRTrademark + -- losing the reservation would offer a name that cannot be registered + it "a reason from a later version still reserves the name" $ + answers status404 "{\"error\":\"unregistered\",\"reasonCode\":\"seasonal\"}" (NRReserved (NRRUnknown "seasonal")) + -- the reason re-encodes into a slot that ends at a space, so it is cut to one + -- token + it "a reason with a space is cut at the space" $ + answers status404 "{\"error\":\"unregistered\",\"reasonCode\":\"two words\"}" (NRReserved (NRRUnknown "two")) + it "an over-long reason is truncated" $ + answers status404 (jsonBody ("{\"error\":\"unregistered\",\"reasonCode\":\"" <> replicate 100 'z' <> "\"}")) $ + NRReserved (NRRUnknown (T.replicate 32 "z")) + -- a name that cannot be dated or priced is not one this router reports on + it "a registration without expiry is a resolver error" $ + refuses status200 (recordWith "\"status\":\"registered\"") (RESOLVER "no expiry") + it "a registered name without a record is a resolver error" $ + refuses status404 "{\"error\":\"registered\",\"expires\":1813853483,\"graceEnds\":1821629483}" (RESOLVER "no record") + it "no price oracle is a resolver error" $ + refuses status404 "{\"error\":\"unregistered\"}" (RESOLVER "no price oracle") + -- only 404 and 410 carry availability, so only their bodies are read as a + -- status + it "upstream failure is a resolver error" $ + refuses status502 "{\"error\":\"upstreamError\"}" (RESOLVER "HTTP 502") + it "unconfigured TLD is not found" $ + refuses status400 "{\"error\":\"tldNotConfigured\"}" NOT_FOUND + it "unreadable status is a resolver error" $ + refuses status404 "{\"error\":\"unknown\"}" (RESOLVER "unknown") + it "long status is truncated" $ + refuses status404 (jsonBody ("{\"error\":\"" <> replicate 400 'e' <> "\"}")) (RESOLVER (T.replicate 32 "e")) + -- NOT_FOUND says the router has nothing to say, never that the name is + -- registrable + it "unreadable 404 body stays NOT_FOUND" $ + refuses status404 "gateway" NOT_FOUND + it "over-cap body is a resolver error" $ + withResolverServer (resolveResp status200 (jsonBody ("{\"status\":\"registered\",\"pad\":\"" <> replicate 400 'x' <> "\"}"))) $ \port _ -> do + env <- newNamesEnv (testNamesConfig port) {resolverMaxResponseBytes = 200} + resolveName env navlDomain `shouldReturn` Left (RESOLVER "response too large") + it "every registration survives the wire" $ + mapM_ + (\a -> smpDecode (smpEncode a) `shouldBe` Right a) + [ NRRegistered {expires = Just (RoundedSystemTime 1813853483), graceUntil = Just (RoundedSystemTime 1821629483), reservedReason_ = Nothing, nameRecord = testNameRecord}, + NRRegistered {expires = Nothing, graceUntil = Nothing, reservedReason_ = Just NRRInternal, nameRecord = testNameRecord}, + NRAvailable {pricing = testPricing, auctionUntil = Nothing}, + NRAvailable {pricing = testPricing, auctionUntil = Just (RoundedSystemTime 1790294400)}, + NRReserved NRRInternal, + NRReserved NRRTrademark, + NRReserved NRRCommunity, + NRReserved (NRRUnknown "seasonal") + ] + -- one vocabulary: the same word on the wire, from the resolver, and in JSON + it "a reason reads the same in JSON as on the wire" $ do + J.encode (NRRUnknown "seasonal") `shouldBe` "\"seasonal\"" + J.encode NRRTrademark `shouldBe` "\"trademark\"" + where + jsonBody = LB.fromStrict . B.pack + -- the resolver returns the record and the registration status in one body + recordWith extra = LB.init (J.encode testNameRecord) <> "," <> extra <> "}" + answers st body a = resolverSays st body (Right a) + refuses st body e = resolverSays st body (Left e) + resolverSays st body expected = + withResolverServer (resolveResp st body) $ \port _ -> do + env <- newNamesEnv (testNamesConfig port) + resolveName env navlDomain `shouldReturn` expected + navlDomain = nameQuery namesSMPVersion SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} + +-- | The .testing oracle: US cents per year by label length. +testPricing :: NamePricing +testPricing = + NamePricing + { rentPrices = M.fromList [(3, USDCents 12793), (4, USDCents 3198)], + basePrice = USDCents 100, + minLabelLength = 3 + } + +pricingJson :: String +pricingJson = "\"rentPrices\":{\"3\":12793,\"4\":3198},\"basePrice\":100,\"minLabelLength\":3" + parseNameSpec :: Spec parseNameSpec = do + -- the hashed form is a query, not a name: it has its own type + it "a name is never a hash" $ + parseN ("[" <> T.replicate 64 "b" <> "].simplex") `shouldSatisfy` isLeft + -- keccak-256("alice"), the same constant the resolver's own tests use + it "hashes the 2LD to the registry key" $ + (queryName . nameQuery currentClientSMPRelayVersion <$> parseN "alice.simplex") + `shouldBe` Right "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501].simplex" + it "leaves subname labels as text" $ + (queryName . nameQuery currentClientSMPRelayVersion <$> parseN "x.alice.simplex") + `shouldBe` Right "x.[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501].simplex" + it "leaves a web name alone: no registry, nothing to key on" $ + (queryName . nameQuery currentClientSMPRelayVersion <$> parseN "example.com") `shouldBe` Right "example.com" + -- below v22 a router can only read the name + it "sends the name itself below v22" $ + (queryName . nameQuery namesSMPVersion <$> parseN "alice.simplex") `shouldBe` Right "alice.simplex" + it "a query survives the wire" $ + mapM_ + (\q -> smpDecode (smpEncode q) `shouldBe` Right q) + [ nameQuery currentClientSMPRelayVersion d, + nameQuery namesSMPVersion d + ] it "accepts a valid simplex-TLD name" $ case parseN "privacy.simplex" of Right d -> do @@ -138,13 +276,14 @@ parseNameSpec = do where parseN :: T.Text -> Either String SimplexDomain parseN = strDecode . encodeUtf8 + d = SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = ["x"]} resolverSpec :: Spec resolverSpec = do it "returns NameRecord on 200 OK" $ withResolverServer (resolveResp status200 (J.encode testNameRecord)) $ \port _ -> do env <- newNamesEnv (testNamesConfig port) - resolveName env aliceDomain `shouldReturn` Right testNameRecord + resolveName env aliceDomain `shouldReturn` Right (NRRegistered Nothing Nothing Nothing testNameRecord) it "returns NOT_FOUND on 404" $ withResolverServer (resolveResp status404 "{}") $ \port _ -> do @@ -200,7 +339,7 @@ resolverSpec = do readIORef reqs `shouldReturn` [["resolve", "alice.simplex"]] where - aliceDomain = SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} + aliceDomain = nameQuery namesSMPVersion SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} healthSpec :: Spec healthSpec = do