diff --git a/CLAUDE.md b/CLAUDE.md index 74e7dcf070..f5c5d257fb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -269,6 +269,8 @@ Symptoms in tests: a v4-specific assertion fails (e.g. an entitlement should-be- **The build stamp comes from a script, not a Maven plugin**: `git.properties` (what `/status` and the root endpoint's `git_commit` report) is written by `scripts/write_git_properties.sh`, invoked from `obp-api/pom.xml`'s `maven-antrun-plugin` execution `generate-git-properties` at `generate-resources`, straight into `target/classes`. It used to be `git-commit-id-maven-plugin`, which was wrong in two ways: its bundled JGit 6.7 has no `commondir` support, so `GitDirLocator.resolveWorktree()` redirects a linked worktree's gitdir to the *main* checkout's `.git` — every build run from `.claude/worktrees/*` stamped the main checkout's branch and commit — and its `PropertiesFileGenerator` skips rewriting when only `git.build.time` differs, freezing the timestamp. Add stamp fields by editing the script (keep the `git.*` key names; `StatusPage.scala` and `APIUtil.gitCommit` read them by name), and don't reintroduce a per-module generator: exactly one `git.properties` may be on the runtime classpath, otherwise which one is reported is incidental. `.github/workflows/test_worktree_build.yml` guards both failure modes. +**SQL must run on PostgreSQL 16+, not just the local Postgres**: PostgreSQL 16 refuses a bind parameter followed directly by a letter (`ERROR: trailing junk after parameter at or near "$1AND"`), which 14 and earlier accept. In doobie, `fr"…"` appends a space and `fr0"…"` doesn't, so `fr0"$value" ++ fr"AND"` renders `$1AND`: it passes every local test on Postgres 14 and fails on a server running 16. Use `fr0` only where the next fragment starts with a space, a comma or a closing parenthesis, or nothing follows; otherwise use `fr`. The same goes for raw SQL strings: always put a space after `?` / `$n`. This shipped once: every Dynamic Entity projection build failed on dcr (Postgres 16) with that error, logged as `DE projection provisioning failed`, while working locally — `ProjectionStore.scope` built `entityname = $1AND bankid = $2AND …`. Test new SQL against Postgres 16 or later. + ## CI (shard map + run tips) Perf note: integration tests are DB/HTTP-bound (~0.4 s/test) on both frameworks; the http4s win is the **pure-unit tier** (no running server, ~0.008 s/test). `ResourceDocsTest`/`SwaggerDocsTest` are the slowest per-test cost — they serialize the whole API surface, so cost grows with endpoint count. `Http4sResourceDocs` already caches the serialized output (`Caching.{getDynamic,getStatic,getAll}ResourceDocCache` + `getStaticSwaggerDocCache`, keyed via `APIUtil.createResourceDocCacheKey`), so repeat requests for the same version/params skip re-serialization. diff --git a/obp-api/src/main/scala/bootstrap/liftweb/Boot.scala b/obp-api/src/main/scala/bootstrap/liftweb/Boot.scala index 070bc35719..a68570e7c7 100644 --- a/obp-api/src/main/scala/bootstrap/liftweb/Boot.scala +++ b/obp-api/src/main/scala/bootstrap/liftweb/Boot.scala @@ -1144,6 +1144,7 @@ object ToSchemify extends MdcLoggable { code.glossaryitem.DynamicGlossaryItem, code.platformapp.PlatformApp, code.platformapp.PlatformAppRequiredScope, + code.domainapi.DomainApi, PayeeLookup, UtilityPaymentCallback, BulkPayment, diff --git a/obp-api/src/main/scala/code/api/dynamic/domainapi/DomainApiPaths.scala b/obp-api/src/main/scala/code/api/dynamic/domainapi/DomainApiPaths.scala new file mode 100644 index 0000000000..6373fbfdb6 --- /dev/null +++ b/obp-api/src/main/scala/code/api/dynamic/domainapi/DomainApiPaths.scala @@ -0,0 +1,181 @@ +/** +Open Bank Project - API +Copyright (C) 2011-2026, TESOBE GmbH. + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +Email: contact@tesobe.com +TESOBE GmbH. +Osloer Strasse 16/17 +Berlin 13359, Germany + +This product includes software developed at +TESOBE (http://www.tesobe.com/) + + */ + +package code.api.dynamic.domainapi + +import cats.effect.IO +import cats.effect.unsafe.implicits.global +import code.api.Constant.ApiPathZero +import code.api.berlin.group.ConstantsBG +import code.api.util.APIUtil.ResourceDoc +import code.domainapi.DomainApiRoute +import com.openbankproject.commons.util.{ApiShortVersions, ApiStandards, ApiVersion} +import org.json4s.JsonAST.{JObject, JValue} + +/** + * This object holds the rules that relate a Domain API's URLs to OBP's own, in one place. + * + * A Domain API publishes the Dynamic Entities and Dynamic Resource Docs of one space under a base path, + * so that `/carbon-registry/v1/activity` reaches what OBP serves at + * `/obp/v7.0.0/banks/SYS/dynamic-entities/activity`. Two things need the same mapping: the front door + * ([[Http4sDomainApi]]), which rewrites an incoming call to the OBP URL, and the Domain API's OpenAPI + * file, which rewrites each documented OBP URL to the published one. Both read the functions here, so + * the published documentation cannot describe a path the front door does not serve, or the reverse. + * + * The same holds for the one thing a Domain API changes in a response: a Dynamic Entity record response + * leaves out `bank_id`, because the base path already fixes the space. [[responseUnderDomainApi]] is + * applied both to the real response and to the documented example. + */ +object DomainApiPaths { + + /** What the front door records on a request it rewrote: which Domain API, and the path that was called. */ + case class DomainApiCall(domainApiId: String, basePath: String, calledPath: String) + + val domainApiCallKey: org.typelevel.vault.Key[DomainApiCall] = + org.typelevel.vault.Key.newKey[IO, DomainApiCall].unsafeRunSync() + + private val obp = ApiStandards.obp.toString + private val v700 = ApiVersion.v7_0_0.toString + private val dynamicEndpoint = ApiShortVersions.`dynamic-endpoint`.toString + private val dynamicResourceDocSegment = "dynamic-resource-doc" + private val dynamicEntitiesSegment = "dynamic-entities" + + /** + * The first path segments OBP serves itself. A base path may not start with one of them, so a Domain + * API can never be confused with, or hide, one of OBP's own URLs. The front door also runs last in the + * request chain, after every OBP route, as a second line of defence. + */ + def reservedFirstSegments: Set[String] = Set( + obp, ApiPathZero, "open-banking", ConstantsBG.berlinGroupVersion1.urlPrefix, ConstantsBG.berlinGroupVersion2.urlPrefix, + "my", "apps", "status", "health", "alive", "banks", "oauth", "dauth", "siwe", ".well-known", "static", + "openapi.json", "openapi.yaml" + ) + + /** + * Path segments a Dynamic Entity URL already gives a meaning to right after the space, and the names of + * the Domain API's own documentation. A Dynamic Resource Doc whose path starts with one of them would be + * hidden under a Domain API, so it counts as a clash. + */ + val reservedUnderBasePath: Set[String] = Set("my", "public", "community", "openapi.json", "openapi.yaml") + + private val Segment = "[a-z0-9]([a-z0-9.-]*[a-z0-9])?" + private val MajorVersionSegment = "v(0|[1-9][0-9]*)".r + private val SemanticVersion = "(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)".r + + /** None when the base path is acceptable, otherwise why not. */ + def basePathProblem(basePath: String): Option[String] = { + val segments = basePath.split("/", -1).toList + if (segments.length < 2 || segments.length > 5) Some("it must have two to five segments") + else if (!segments.forall(_.matches(Segment))) Some("each segment must be lowercase letters, digits, hyphens or dots") + else if (reservedFirstSegments.contains(segments.head)) Some(s"its first segment, ${segments.head}, is one OBP serves") + else if (MajorVersionSegment.unapplySeq(segments.last).isEmpty) Some("its last segment must be the major version, vN") + else None + } + + /** The N of the `vN` that ends a base path. */ + def majorOf(basePath: String): Option[Int] = + basePath.split("/").lastOption.collect { case MajorVersionSegment(n) => n.toInt } + + /** A version is MAJOR.MINOR.PATCH, and its MAJOR is the base path's. */ + def versionFits(version: String, basePath: String): Boolean = version match { + case SemanticVersion(major, _, _) => majorOf(basePath).contains(major.toInt) + case _ => false + } + + /** + * Two base paths overlap when one is the other or starts with it, segment by segment. Overlapping base + * paths are refused, so a call never has more than one Domain API it could belong to. + */ + def overlap(a: String, b: String): Boolean = { + val (as, bs) = (a.split("/").toList, b.split("/").toList) + as.startsWith(bs) || bs.startsWith(as) + } + + /** The Domain API a request path is under, and the segments after its base path. */ + def find(routes: List[DomainApiRoute], pathSegments: List[String]): Option[(DomainApiRoute, List[String])] = + routes.collectFirst { + case route if pathSegments.startsWith(route.basePathSegments) => (route, pathSegments.drop(route.basePathSegments.length)) + } + + /** The OBP path a call to a Dynamic Entity under a Domain API is served at. */ + def dynamicEntityPath(space: String, rest: List[String]): List[String] = + obp :: v700 :: "banks" :: space :: dynamicEntitiesSegment :: rest + + /** The OBP path a call to a Dynamic Resource Doc under a Domain API is served at. */ + def dynamicResourceDocPath(space: String, rest: List[String]): List[String] = + obp :: dynamicEndpoint :: "banks" :: space :: dynamicResourceDocSegment :: rest + + /** + * The path under the base path at which a documented endpoint is published, from its ResourceDoc's + * request URL: the v7.0.0 Dynamic Entity docs (`/banks/SPACE/dynamic-entities/...`) and the Dynamic + * Resource Docs (`/banks/SPACE/dynamic-resource-doc/...`). Anything else is not published (None). + */ + def publishedPath(space: String, docRequestUrl: String): Option[String] = { + // A doc's request URL may carry the prefix of the version it is served in (/obp/v7.0.0/...); the + // space starts at `banks`. + val segments = docRequestUrl.split("/").filter(_.nonEmpty).toList.dropWhile(_ != "banks") + segments match { + case "banks" :: `space` :: kind :: rest if rest.nonEmpty && (kind == dynamicEntitiesSegment || kind == dynamicResourceDocSegment) => + Some(rest.mkString("/", "/", "")) + case _ => None + } + } + + /** A path template with each placeholder (an all-capitals segment) reduced to one form, for comparison. */ + private def templateKey(path: String): String = + path.split("/").filter(_.nonEmpty).map(s => if (s.matches("[A-Z][A-Z0-9_]*")) "{}" else s).mkString("/", "/", "") + + /** + * The verb and path pairs that more than one endpoint of the space would publish, and the Dynamic + * Resource Docs whose path starts with a segment a Dynamic Entity URL or the documentation already uses. + * Each is described for the person who has to resolve it. + */ + def clashes(space: String, docs: List[ResourceDoc]): List[String] = { + val published = docs.flatMap(doc => publishedPath(space, doc.requestUrl).map(path => (doc, path))) + val duplicates = published + .groupBy { case (doc, path) => (doc.requestVerb.toUpperCase, templateKey(path)) } + .collect { case ((verb, key), entries) if entries.length > 1 => + s"$verb $key (${entries.map(_._1.partialFunctionName).sorted.mkString(", ")})" + }.toList + val hidden = published.collect { + case (doc, path) if doc.requestUrl.contains(s"/$dynamicResourceDocSegment/") && + reservedUnderBasePath.contains(path.split("/").filter(_.nonEmpty).headOption.getOrElse("")) => + s"${doc.requestVerb.toUpperCase} $path (${doc.partialFunctionName}) starts with a reserved segment" + } + (duplicates ++ hidden).sorted + } + + /** A Dynamic Entity record response as a Domain API returns it: without `bank_id`. */ + def responseUnderDomainApi(response: JObject): JObject = + JObject(response.obj.filterNot(_._1 == "bank_id")) + + /** The same rule applied to a documented example, which may be any JSON. */ + def exampleUnderDomainApi(example: Any): Any = example match { + case o: JObject => responseUnderDomainApi(o) + case other => other + } +} diff --git a/obp-api/src/main/scala/code/api/dynamic/domainapi/Http4sDomainApi.scala b/obp-api/src/main/scala/code/api/dynamic/domainapi/Http4sDomainApi.scala new file mode 100644 index 0000000000..7e2abaa898 --- /dev/null +++ b/obp-api/src/main/scala/code/api/dynamic/domainapi/Http4sDomainApi.scala @@ -0,0 +1,138 @@ +/** +Open Bank Project - API +Copyright (C) 2011-2026, TESOBE GmbH. + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +Email: contact@tesobe.com +TESOBE GmbH. +Osloer Strasse 16/17 +Berlin 13359, Germany + +This product includes software developed at +TESOBE (http://www.tesobe.com/) + + */ + +package code.api.dynamic.domainapi + +import cats.data.{Kleisli, OptionT} +import cats.effect.IO +import code.api.Constant.HostName +import code.api.ResourceDocs1_4_0.OpenAPI31JSONFactory +import code.api.ResourceDocs1_4_0.OpenAPI31JSONFactory.{InfoJson, ServerJson} +import code.api.cache.Caching +import code.api.dynamic.domainapi.DomainApiPaths.{DomainApiCall, domainApiCallKey} +import code.api.dynamic.endpoint.Http4sDynamicEndpoint +import code.api.dynamic.entity.Http4sDynamicEntity +import code.api.util.APIUtil.ResourceDoc +import code.api.util.{APIUtil, YAMLUtils} +import code.api.v1_4_0.JSONFactory1_4_0 +import code.domainapi.{DomainApiRoute, DomainApis} +import code.util.Helper.MdcLoggable +import com.openbankproject.commons.util.ApiVersion +import org.http4s._ +import org.http4s.headers.`Content-Type` +import org.json4s.JsonAST.JValue +import org.json4s.native.JsonMethods.compact + +/** + * This object is the front door of the Domain APIs: it serves the URLs under each registered base path. + * + * A call under a base path is rewritten to the OBP URL of the same endpoint (see [[DomainApiPaths]]) and + * handed to the handler that serves that URL, so authentication, Roles, Consents, rate limiting, + * row-level access, field restrictions, Dynamic Query checks and metrics all run exactly as they do for + * the OBP URL. A Domain API grants nothing. A Dynamic Entity of the space is tried first, then a Dynamic + * Resource Doc of the space; registering or updating a Domain API is refused while two of the space's + * endpoints would answer the same verb and path, so the order only matters for a clash created later. + * + * `BASE_PATH/openapi.json` and `BASE_PATH/openapi.yaml` serve the Domain API's own OpenAPI document. + * + * It is wired into Http4sApp.baseServices last, just before the JSON 404, so no OBP route can be hidden + * by a base path; base paths are also refused when their first segment is one OBP serves. + */ +object Http4sDomainApi extends MdcLoggable { + + private type HttpF[A] = OptionT[IO, A] + + private val jsonContentType = `Content-Type`(MediaType.application.json, Charset.`UTF-8`) + private val yamlContentType = `Content-Type`(new MediaType("application", "yaml"), Charset.`UTF-8`) + + private def withPath(req: Request[IO], segments: List[String]): Request[IO] = + req.withUri(req.uri.withPath(Uri.Path.unsafeFromString(segments.mkString("/", "/", "")))) + + lazy val routes: HttpRoutes[IO] = + Kleisli[HttpF, Request[IO], Response[IO]] { (req: Request[IO]) => + val segments = req.uri.path.segments.map(_.encoded).toList + DomainApiPaths.find(DomainApis.domainApiProvider.vend.routes(), segments) match { + case None => OptionT.none[IO, Response[IO]] + case Some((route, rest)) => + rest match { + case "openapi.json" :: Nil if req.method == Method.GET => + OptionT.liftF(IO(openApiJson(route)).map(body => + Response[IO](Status.Ok).withEntity(body).withContentType(jsonContentType))) + case "openapi.yaml" :: Nil if req.method == Method.GET => + OptionT.liftF(IO(openApiYaml(route)).map(body => + Response[IO](Status.Ok).withEntity(body).withContentType(yamlContentType))) + case Nil => OptionT.none[IO, Response[IO]] + case _ => + val marked = req.withAttribute(domainApiCallKey, + DomainApiCall(route.domainApiId, route.basePath, req.uri.path.renderString)) + Http4sDynamicEntity.wrappedRoutesDynamicEntityV700.run(withPath(marked, DomainApiPaths.dynamicEntityPath(route.bankId, rest))) + .orElse(Http4sDynamicEndpoint.wrappedRoutesDynamicEndpoint.run(withPath(marked, DomainApiPaths.dynamicResourceDocPath(route.bankId, rest)))) + } + } + } + + /** + * The ResourceDocs of a space's endpoints that a Domain API publishes: the v7.0.0 Dynamic Entity docs + * and the Dynamic Resource Docs of that space. + */ + def spaceDocs(space: String): List[ResourceDoc] = + APIUtil.allDynamicResourceDocsIn(ApiVersion.v7_0_0).filter(doc => APIUtil.dynamicResourceDocBelongsToSpace(doc, space)) + + /** The space's docs as the Domain API publishes them: at their published path, examples without bank_id. */ + def publishedDocs(route: DomainApiRoute): List[ResourceDoc] = + spaceDocs(route.bankId).flatMap { doc => + DomainApiPaths.publishedPath(route.bankId, doc.requestUrl).map { path => + val published = doc.copy(requestUrl = path, successResponseBody = DomainApiPaths.exampleUnderDomainApi(doc.successResponseBody)) + published.connectorMethods = doc.connectorMethods + published + } + } + + /** The Domain API's OpenAPI 3.1 document: the published docs, with the Domain API's own title, version and server. */ + def openApi(route: DomainApiRoute): JValue = { + val docsJson = JSONFactory1_4_0.createResourceDocsJson(publishedDocs(route), isVersion4OrHigher = true, locale = None).resource_docs + val document = OpenAPI31JSONFactory.createOpenAPI31Json(docsJson, route.version, HostName).copy( + info = InfoJson(title = route.title, version = route.version, description = Some(route.description).filter(_.nonEmpty)), + servers = List(ServerJson(url = s"$HostName/${route.basePath}", description = Some(route.title)))) + OpenAPI31JSONFactory.OpenAPI31JsonFormats.toJValue(document) + } + + // Cached with the dynamic resource docs, which are cleared whenever a Dynamic Entity or Dynamic Resource + // Doc changes. The key carries everything of the registration the document shows. + private def cached(route: DomainApiRoute, format: String)(build: => String): String = { + val key = s"domain-api-openapi:$format:${route.domainApiId}:${route.basePath}:${route.version}:${(route.title + route.description).hashCode}" + Caching.getDynamicResourceDocCache(key).getOrElse { + val rendered = build + Caching.setDynamicResourceDocCache(key, rendered) + rendered + } + } + + def openApiJson(route: DomainApiRoute): String = cached(route, "json")(compact(org.json4s.native.JsonMethods.render(openApi(route)))) + + def openApiYaml(route: DomainApiRoute): String = cached(route, "yaml")(YAMLUtils.jValueToYAMLSafe(openApi(route), "# Error converting to YAML")) +} diff --git a/obp-api/src/main/scala/code/api/dynamic/endpoint/Http4sDynamicEndpoint.scala b/obp-api/src/main/scala/code/api/dynamic/endpoint/Http4sDynamicEndpoint.scala index a87c085227..c59f800c16 100644 --- a/obp-api/src/main/scala/code/api/dynamic/endpoint/Http4sDynamicEndpoint.scala +++ b/obp-api/src/main/scala/code/api/dynamic/endpoint/Http4sDynamicEndpoint.scala @@ -28,6 +28,7 @@ package code.api.dynamic.endpoint import org.json4s._ import cats.data.{Kleisli, OptionT} import cats.effect.IO +import code.api.dynamic.endpoint.helper.DynamicEndpointMatch import code.api.dynamic.endpoint.helper.{DynamicEndpointHelper, DynamicEndpoints} import code.api.util.CustomJsonFormats import code.api.util.http4s.Http4sRequestAttributes.EndpointHelpers @@ -117,8 +118,19 @@ object Http4sDynamicEndpoint extends MdcLoggable { */ private def pieceC(req: Request[IO]): OptionT[IO, Response[IO]] = DynamicEndpoints.findEndpoint(req) match { - case None => OptionT.none[IO, Response[IO]] - case Some(doc) => + case DynamicEndpointMatch.NotFound => OptionT.none[IO, Response[IO]] + case DynamicEndpointMatch.Ambiguous(spaces) => + OptionT.liftF { + Http4sCallContextBuilder.fromRequest(req, apiVersionString).flatMap { cc => + ErrorResponseConverter.toHttp4sResponse( + code.api.JsonResponseException(s"${code.api.util.ErrorMessages.DynamicResourceDocUrlAmbiguous}${spaces.mkString(", ")}.", 409, cc.correlationId), cc) + .flatMap(EndpointHelpers.recordMetricFor(_)(cc)) + } + } + case DynamicEndpointMatch.Found(doc, matchedReq) => + // matchedReq is the request with the URL that names the doc's space, also when the caller used the + // older URL without it, so the handler, its path parameters and the metric all see one URL. + val req = matchedReq OptionT.liftF { Http4sCallContextBuilder.fromRequest(req, apiVersionString).flatMap { cc0 => val cc = cc0.copy(resourceDocument = Some(doc), operationId = Some(doc.operationId)) diff --git a/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicEndpoints.scala b/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicEndpoints.scala index 4ebf1c7d19..d38eff616e 100644 --- a/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicEndpoints.scala +++ b/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicEndpoints.scala @@ -44,6 +44,18 @@ import org.http4s.{Request, Response} import java.net.URLDecoder import scala.collection.immutable.List +/** + * What a request under /obp/dynamic-endpoint/ names: one runtime-compiled endpoint (with the request it + * should be run as, which may be the canonical form of the URL the caller used), several that the URL + * cannot tell apart, or none. + */ +sealed trait DynamicEndpointMatch +object DynamicEndpointMatch { + case class Found(doc: ResourceDoc, request: Request[IO]) extends DynamicEndpointMatch + case class Ambiguous(spaces: List[String]) extends DynamicEndpointMatch + case object NotFound extends DynamicEndpointMatch +} + object DynamicEndpoints { //TODO, better put all other dynamic endpoints into this list. eg: dynamicEntityEndpoints, dynamicSwaggerDocsEndpoints .... val disabledEndpointOperationIds = getDisabledEndpointOperationIds @@ -66,12 +78,35 @@ object DynamicEndpoints { * (ResourceDoc.authCheckIO) and the handler. Replaces the former Lift `dynamicEndpoint` * (PartialFunction[Req, CallContext => Box[JsonResponse]]) that ran through the Lift dispatch. */ - def findEndpoint(req: Request[IO]): Option[ResourceDoc] = { + def findEndpoint(req: Request[IO]): DynamicEndpointMatch = { val partPath = req.uri.path.segments.drop(2).map(_.encoded).toList // segments after /obp/dynamic-endpoint val verb = req.method.name - endpointGroups.iterator - .flatMap(_.docs.iterator) - .find(doc => doc.requestVerb == verb && doc.dynamicHttp4sFunction.isDefined && doc.matchesPartPath(partPath)) + def servable(doc: ResourceDoc): Boolean = doc.requestVerb == verb && doc.dynamicHttp4sFunction.isDefined + val otherGroups = endpointGroups.filterNot(_ == DynamicResourceDocsEndpointGroup) + val resourceDocs = if (endpointGroups.contains(DynamicResourceDocsEndpointGroup)) DynamicResourceDocsEndpointGroup.docs.filter(servable) else Nil + + otherGroups.iterator.flatMap(_.docs.iterator).find(doc => servable(doc) && doc.matchesPartPath(partPath)) match { + case Some(doc) => DynamicEndpointMatch.Found(doc, req) + case None => partPath match { + // The URL names its space: /banks/BANK_ID/dynamic-resource-doc/..., BANK_ID being a bank's id or SYS. + // The space segment is compared exactly first: matchesPartPath alone would read an upper-case + // space such as SYS as a path variable, and so match it against any bank id. + case "banks" :: space :: _ => + resourceDocs.find(doc => DynamicResourceDocsEndpointGroup.spaceOf(doc) == space && doc.matchesPartPath(partPath)) + .map(doc => DynamicEndpointMatch.Found(doc, req)).getOrElse(DynamicEndpointMatch.NotFound) + // The URL that names no space, kept so existing callers keep working: it is served only when exactly + // one doc, in any space, answers it, and then as if the caller had used that doc's own URL. + case first :: _ if first == DynamicResourceDocsEndpointGroup.urlPrefix => + resourceDocs.filter(doc => doc.matchesPartPath("banks" :: DynamicResourceDocsEndpointGroup.spaceOf(doc) :: partPath)) match { + case doc :: Nil => + val canonical = req.uri.path.segments.take(2).map(_.encoded).toList ++ ("banks" :: DynamicResourceDocsEndpointGroup.spaceOf(doc) :: partPath) + DynamicEndpointMatch.Found(doc, req.withUri(req.uri.withPath(org.http4s.Uri.Path.unsafeFromString(canonical.mkString("/", "/", ""))))) + case Nil => DynamicEndpointMatch.NotFound + case several => DynamicEndpointMatch.Ambiguous(several.map(DynamicResourceDocsEndpointGroup.spaceOf).distinct.sorted) + } + case _ => DynamicEndpointMatch.NotFound + } + } } def dynamicResourceDocs: List[ResourceDoc] = endpointGroups.flatMap(_.docs) diff --git a/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicResourceDocsEndpointGroup.scala b/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicResourceDocsEndpointGroup.scala index 7ea394c1a2..b6479aebb6 100644 --- a/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicResourceDocsEndpointGroup.scala +++ b/obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicResourceDocsEndpointGroup.scala @@ -38,6 +38,23 @@ import scala.collection.immutable.List object DynamicResourceDocsEndpointGroup extends EndpointGroup with code.util.Helper.MdcLoggable { override lazy val urlPrefix: String = APIUtil.getPropsValue("url.prefix.dynamic.resourceDoc", "dynamic-resource-doc") + /** The space a doc belongs to, as its URL names it: its bank's id, or SYS for the system space. */ + def spaceOf(doc: APIUtil.ResourceDoc): String = code.api.dynamic.entity.helper.DynamicEntitySpace.bankIdOrSystem(doc.createdByBankId) + + /** + * Every doc is served under the space it belongs to: /banks/BANK_ID/dynamic-resource-doc/REQUEST_URL, + * with SYS as BANK_ID for the system space, as the v7.0.0 Dynamic Entity URLs and the Dynamic Endpoints + * created from Swagger name theirs. Without the space, two banks could each have a doc at the same URL + * and which one answered would not depend on anything the caller sent. (The URL without the space is + * still answered when it is unambiguous; see DynamicEndpoints.findEndpoint.) + */ + override def docs: List[APIUtil.ResourceDoc] = resourceDocs map { doc => + val newUrl = s"/banks/${spaceOf(doc)}/$urlPrefix/${doc.requestUrl}".replaceAll("/+", "/") + val newDoc = doc.copy(requestUrl = newUrl) // copy preserves dynamicHttp4sFunction + newDoc.connectorMethods = doc.connectorMethods // copy does not keep the var; reset it, as EndpointGroup.docs does + newDoc + } + override protected def resourceDocs: List[APIUtil.ResourceDoc] = // Per-row isolation: a stored methodBody written against the deprecated Lift contract diff --git a/obp-api/src/main/scala/code/api/dynamic/entity/Http4sDynamicEntity.scala b/obp-api/src/main/scala/code/api/dynamic/entity/Http4sDynamicEntity.scala index 897375e3f2..80946b2ec8 100644 --- a/obp-api/src/main/scala/code/api/dynamic/entity/Http4sDynamicEntity.scala +++ b/obp-api/src/main/scala/code/api/dynamic/entity/Http4sDynamicEntity.scala @@ -171,8 +171,12 @@ object Http4sDynamicEntity extends MdcLoggable { private val namesEverySpaceKey: org.typelevel.vault.Key[Boolean] = org.typelevel.vault.Key.newKey[IO, Boolean].unsafeRunSync() + // Under a Domain API the base path already fixes the space, so the response leaves bank_id out. The rule + // is DomainApiPaths.responseUnderDomainApi, which the Domain API's documented examples also go through. private def wrapBankId(req: Request[IO], bankId: Option[String], result: JObject): JObject = - if (bankId.isDefined || req.attributes.lookup(namesEverySpaceKey).contains(true)) + if (req.attributes.lookup(code.api.dynamic.domainapi.DomainApiPaths.domainApiCallKey).isDefined) + code.api.dynamic.domainapi.DomainApiPaths.responseUnderDomainApi(result) + else if (bankId.isDefined || req.attributes.lookup(namesEverySpaceKey).contains(true)) (("bank_id" -> DynamicEntitySpace.bankIdOrSystem(bankId)): JObject) merge result else result diff --git a/obp-api/src/main/scala/code/api/dynamic/entity/projection/PostgresProjectionBackend.scala b/obp-api/src/main/scala/code/api/dynamic/entity/projection/PostgresProjectionBackend.scala index 7be2a02979..90eb5a29fe 100644 --- a/obp-api/src/main/scala/code/api/dynamic/entity/projection/PostgresProjectionBackend.scala +++ b/obp-api/src/main/scala/code/api/dynamic/entity/projection/PostgresProjectionBackend.scala @@ -161,7 +161,7 @@ object PostgresProjectionBackend extends DynamicEntityQueryBackend { case (true, Some(uid)) => fr"AND EXISTS (SELECT 1 FROM" ++ Fragment.const(s"${ProjectionStore.aclTable} acl") ++ fr"WHERE" ++ Fragment.const(s"acl.${ProjectionStore.aclDataIdColumn} = $childBlobAlias.${ProjectionStore.idColumn}") ++ - fr"AND" ++ Fragment.const(s"acl.${ProjectionStore.aclUserIdColumn}") ++ fr"=" ++ fr0"$uid" ++ + fr"AND" ++ Fragment.const(s"acl.${ProjectionStore.aclUserIdColumn}") ++ fr"=" ++ fr"$uid" ++ fr"AND" ++ Fragment.const(s"acl.${ProjectionStore.aclCanReadColumn} = true") ++ fr")" case _ => Fragment.empty } diff --git a/obp-api/src/main/scala/code/api/dynamic/entity/projection/ProjectionStore.scala b/obp-api/src/main/scala/code/api/dynamic/entity/projection/ProjectionStore.scala index df810515de..c2df5157b2 100644 --- a/obp-api/src/main/scala/code/api/dynamic/entity/projection/ProjectionStore.scala +++ b/obp-api/src/main/scala/code/api/dynamic/entity/projection/ProjectionStore.scala @@ -121,9 +121,10 @@ object ProjectionStore { // to no bank, never a SQL NULL, so this is a plain equality. It used to bind an Option and // compare with IS NOT DISTINCT FROM; that stopped matching the moment the sentinel replaced the // NULL, and this is the one place outside the Mapper queries that reads the column directly. - val byEntity = Fragment.const(p + entityNameColumn) ++ fr"=" ++ fr0"$entityName" - val byBank = Fragment.const(p + bankIdColumn) ++ fr"=" ++ fr0"${bankId.getOrElse(DYNAMIC_ENTITY_SYSTEM_LEVEL_BANK_ID)}" - val byPersonal = Fragment.const(p + personalColumn) ++ fr"=" ++ fr0"$isPersonalEntity" + // fr, not fr0: each value is followed by AND, and `$1AND` is refused by PostgreSQL 16+. + val byEntity = Fragment.const(p + entityNameColumn) ++ fr"=" ++ fr"$entityName" + val byBank = Fragment.const(p + bankIdColumn) ++ fr"=" ++ fr"${bankId.getOrElse(DYNAMIC_ENTITY_SYSTEM_LEVEL_BANK_ID)}" + val byPersonal = Fragment.const(p + personalColumn) ++ fr"=" ++ fr"$isPersonalEntity" val base = byEntity ++ fr"AND" ++ byBank ++ fr"AND" ++ byPersonal if (isPersonalEntity) base ++ fr"AND" ++ Fragment.const(p + userIdColumn) ++ fr"IS NOT DISTINCT FROM" ++ fr0"${userId: Option[String]}" else base diff --git a/obp-api/src/main/scala/code/api/util/ApiRole.scala b/obp-api/src/main/scala/code/api/util/ApiRole.scala index 65d2c113b4..b503df15bd 100644 --- a/obp-api/src/main/scala/code/api/util/ApiRole.scala +++ b/obp-api/src/main/scala/code/api/util/ApiRole.scala @@ -590,6 +590,20 @@ object ApiRole extends MdcLoggable{ case class CanDeletePlatformApp(requiresBankId: Boolean = false) extends ApiRole lazy val canDeletePlatformApp = CanDeletePlatformApp() + // Domain APIs: a space's dynamic endpoints published under a base path of its own. Held at a bank id, or at + // SYS for the system space, like the other Roles of the dynamic spaces. + case class CanCreateDomainApi(requiresBankId: Boolean = true) extends ApiRole + lazy val canCreateDomainApi = CanCreateDomainApi() + + case class CanGetDomainApis(requiresBankId: Boolean = true) extends ApiRole + lazy val canGetDomainApis = CanGetDomainApis() + + case class CanUpdateDomainApi(requiresBankId: Boolean = true) extends ApiRole + lazy val canUpdateDomainApi = CanUpdateDomainApi() + + case class CanDeleteDomainApi(requiresBankId: Boolean = true) extends ApiRole + lazy val canDeleteDomainApi = CanDeleteDomainApi() + // Shows which Consumers and client IP addresses are sending the most traffic to the instance // (TrafficSources). About the instance, so held at the empty bank id. It names Consumers and IP // addresses, which is why it is a Role of its own and not part of CanGetTelemetry. diff --git a/obp-api/src/main/scala/code/api/util/ErrorMessages.scala b/obp-api/src/main/scala/code/api/util/ErrorMessages.scala index f7073b3452..11e853287e 100644 --- a/obp-api/src/main/scala/code/api/util/ErrorMessages.scala +++ b/obp-api/src/main/scala/code/api/util/ErrorMessages.scala @@ -111,6 +111,13 @@ object ErrorMessages { val DynamicEntityUpdateNotSchemaCompatible = "OBP-09023: Operation is not allowed, because this DynamicEntity already has data. The definition of a populated entity can only be changed in schema-compatible ways: the entity name must stay the same, every existing property must keep its name and type, and no property may be added to 'required'. New optional properties may be added. Changing indexed, index, example, description, minLength, maxLength, the read/write role settings and hide_field_from_public_access is allowed. Delete all the data before making a structural change." val DynamicEntityFieldNotReadable = "OBP-09025: These fields cannot be used to filter or sort, because you may not read them: " val DynamicEntityRecordIdTooLong = "OBP-09024: The id of this DynamicEntity record is too long. A record id is stored in a column of 255 characters. Please supply a shorter id, or leave the id field out of the request body and one will be generated." + // Domain APIs: a space's dynamic endpoints published under a base path of its own. + val InvalidDomainApiBasePath = "OBP-09026: Invalid base_path. It must be two to five segments of lowercase letters, digits, hyphens or dots, separated by /, with no leading or trailing /, ending with the major version as vN (for example carbon-registry/v1), and its first segment must not be one OBP itself serves: " + val DomainApiBasePathAlreadyExists = "OBP-09027: Another Domain API already uses this base_path, or one that starts with it or that it starts with: " + val InvalidDomainApiVersion = "OBP-09028: Invalid version. It must be a semantic version, MAJOR.MINOR.PATCH (for example 1.0.0), whose MAJOR equals the N of the vN that ends base_path." + val DomainApiNotFound = "OBP-09029: Domain API not found in this space. Please specify a valid value for DOMAIN_API_ID." + val InvalidDomainApiTitle = "OBP-09030: Invalid title or description. title must be 1 to 255 characters and description at most 2000." + val DomainApiPathClash = "OBP-09031: Two endpoints of this space would answer the same verb and path under a Domain API, so it cannot be published until one of them is renamed or removed: " // General messages (OBP-10XXX) @@ -1019,6 +1026,7 @@ object ErrorMessages { val PaymentChallengeHasNoOnBehalfOfUser = "OBP-40064: This payment needs Strong Customer Authentication, but the user it is being made for could not be determined, " + "so there is nobody who can be asked to authorise it. A consent that names the user it acts for is required before a payment of this size can be started." val DynamicQueryInvalid = "OBP-40065: The Dynamic Query cannot be run as written. " + val DynamicResourceDocUrlAmbiguous = "OBP-40067: More than one Dynamic Resource Doc, in different spaces, answers this URL. Call it under its space, at /obp/dynamic-endpoint/banks/BANK_ID/dynamic-resource-doc/..., where BANK_ID is one of: " val DynamicQueryEntityNotReadable = "OBP-40066: This Dynamic Query reads Dynamic Entities you may not read: " // Exceptions (OBP-50XXX) val UnknownError = "OBP-50000: Unknown Error." diff --git a/obp-api/src/main/scala/code/api/util/Glossary.scala b/obp-api/src/main/scala/code/api/util/Glossary.scala index f91947c1f7..8314c265e9 100644 --- a/obp-api/src/main/scala/code/api/util/Glossary.scala +++ b/obp-api/src/main/scala/code/api/util/Glossary.scala @@ -4246,6 +4246,32 @@ object Glossary extends MdcLoggable { | |`POST /obp/v7.0.0/management/dynamic-resource-docs/explain` shows how a Dynamic Query would be answered, without reading any record, so its author can check that the SQL is sane and that access is what they expect: each read it would make, in order, with the SQL OBP builds for it (every value shown as `?`) or a description when it goes through the record provider; every Dynamic Entity it reads, with its read Role and whether the caller may read it; every read-restricted field it touches; and the refusal a caller would get. It can explain the query for the requesting User or for a caller who is not logged in, and with the parameters a caller would add. The API Manager's Explain button uses it. | +""".stripMargin) + + glossaryItems += GlossaryItem( + title = "Domain APIs", + description = + s""" +|# Domain APIs +| +|A **Domain API** publishes the Dynamic Entities and Dynamic Resource Docs (Dynamic Queries included) of one space under a base path of its own, so that an API built on OBP can be offered without OBP's own URL structure in front of it. A space is a bank, or the system space, whose bank id is `SYS`. +| +|With the base path `carbon-registry/v1` over the system space: +| +|* `/carbon-registry/v1/activity` answers what `/obp/v7.0.0/banks/SYS/dynamic-entities/activity` answers, and likewise `activity/ACTIVITY_ID`, `my/activity`, `public/activity`, `community/activity` and `activity/ACTIVITY_ID/access`; +|* `/carbon-registry/v1/registry/summary` answers what the Dynamic Resource Doc at `/obp/dynamic-endpoint/banks/SYS/dynamic-resource-doc/registry/summary` answers; +|* `/carbon-registry/v1/openapi.yaml` and `/carbon-registry/v1/openapi.json` are its OpenAPI document, with the Domain API's own title, description, version and server, and only its own endpoints. +| +|Dynamic Entity names are not renamed: only the part of the URL before them is. The endpoints that create and change definitions stay at their OBP URLs; a Domain API publishes the endpoints that serve and take data. Dynamic Endpoints made from a Swagger file are not published under a Domain API yet. +| +|**It only renames.** A call under a base path is rewritten to the OBP URL and runs exactly as a call to that URL would: the same authentication, Roles, Consents, rate limits, row-level access and field restrictions, and the same API Metrics. A Domain API grants nothing. The one difference in a response is that a Dynamic Entity record response leaves out `bank_id`, because the base path already fixes the space; the OpenAPI document's examples leave it out too. +| +|**Base path and version.** The base path is two to five segments of lowercase letters, digits, hyphens and dots, ending with the major version as `vN`, for example `carbon-registry/v1`. It may not start with a segment OBP serves itself (such as `obp` or `open-banking`), and may not overlap another Domain API's base path. The Domain API's `version` is its full semantic version, MAJOR.MINOR.PATCH, whose MAJOR is the N of the base path. A compatible change, such as a new optional field or a new Dynamic Query, edits `version` (or leaves it) and keeps every URL; a breaking change gets a new Domain API with a new base path, for example `carbon-registry/v2`, which can run alongside the old one while clients move. OBP never changes the version itself. For Dynamic Entities OBP already keeps changes compatible once an entity holds records (only optional properties may be added); a Dynamic Resource Doc can be changed in any way, so keeping its changes compatible is up to its author. +| +|**Clashes.** Under a base path the Dynamic Entities and Dynamic Resource Docs of the space share one set of paths. A Domain API is refused (${ErrorMessages.DomainApiPathClash.takeWhile(_ != ':')}) while two of them would answer the same verb and path, or while a Dynamic Resource Doc's path starts with `my`, `public`, `community`, `openapi.json` or `openapi.yaml`. If a clash is created later, the Dynamic Entity answers. +| +|**Managing.** `/obp/v7.0.0/management/banks/BANK_ID/domain-apis` creates and lists a space's Domain APIs, and `/obp/v7.0.0/management/banks/BANK_ID/domain-apis/DOMAIN_API_ID` reads, updates and deletes one. BANK_ID is a bank's id or `SYS`. The Roles are CanCreateDomainApi, CanGetDomainApis, CanUpdateDomainApi and CanDeleteDomainApi, held at that BANK_ID. +| """.stripMargin) glossaryItems += GlossaryItem( diff --git a/obp-api/src/main/scala/code/api/util/http4s/Http4sApp.scala b/obp-api/src/main/scala/code/api/util/http4s/Http4sApp.scala index 82e11da486..0bdbc3277c 100644 --- a/obp-api/src/main/scala/code/api/util/http4s/Http4sApp.scala +++ b/obp-api/src/main/scala/code/api/util/http4s/Http4sApp.scala @@ -183,6 +183,9 @@ object Http4sApp extends MdcLoggable { .orElse(code.api.DirectLoginRoutes.routes.run(req)) .orElse(code.api.SIWERoutes.routes.run(req)) .orElse(code.api.AliveCheckRoutes.routes.run(req)) + // Domain APIs: a space's dynamic endpoints under a base path of its own. Last, so no OBP route can be + // hidden by a base path. + .orElse(code.api.dynamic.domainapi.Http4sDomainApi.routes.run(req)) .orElse(notFoundCatchAll.run(req)) } } diff --git a/obp-api/src/main/scala/code/api/util/migration/Migration.scala b/obp-api/src/main/scala/code/api/util/migration/Migration.scala index d45dcbbb84..49618ee331 100644 --- a/obp-api/src/main/scala/code/api/util/migration/Migration.scala +++ b/obp-api/src/main/scala/code/api/util/migration/Migration.scala @@ -248,7 +248,13 @@ object Migration extends MdcLoggable { // for NULL silently found nothing once the writer had started using the sentinel. adoptSystemLevelBankIdSentinel("dynamicentity"), dropSupersededIndex("dynamicdata", "dynamicdata_dynamicdataid"), - dropSupersededIndex("dynamicdataaccess", "dynamicdataaccess_dynamicdataid_userid") + dropSupersededIndex("dynamicdataaccess", "dynamicdataaccess_dynamicdataid_userid"), + // Dynamic Resource Docs follow the same space model: a system level doc stores SYS, not NULL, and a + // verb and URL are unique within a space, so the index that made them unique across every space goes + // (Schemifier then creates the per space one on (BankId, RequestUrl, RequestVerb)). The NULLs must + // move before that index exists, or two system docs at one URL would both be NULL and both allowed. + adoptSystemLevelBankIdSentinel("dynamicresourcedoc"), + dropSupersededIndex("dynamicresourcedoc", "dynamicresourcedoc_requesturl_requestverb") ) val endDate = System.currentTimeMillis() val didSomething = outcomes.exists(_.changedSomething) diff --git a/obp-api/src/main/scala/code/api/v7_0_0/Http4s700.scala b/obp-api/src/main/scala/code/api/v7_0_0/Http4s700.scala index 53a552d01c..91f369d7d3 100644 --- a/obp-api/src/main/scala/code/api/v7_0_0/Http4s700.scala +++ b/obp-api/src/main/scala/code/api/v7_0_0/Http4s700.scala @@ -7461,6 +7461,9 @@ object Http4s700 { // Platform Apps: the Consumers this installation runs as part of its own deployment, and the Scopes they need. resourceDocs ++= Http4s700PlatformApps.resourceDocs + // Domain APIs: a space's Dynamic Entities and Dynamic Resource Docs published under a base path of its own. + resourceDocs ++= Http4s700DomainApis.resourceDocs + // Groups: bring the members of a Group in line with its current Roles. resourceDocs ++= Http4s700Groups.resourceDocs diff --git a/obp-api/src/main/scala/code/api/v7_0_0/Http4s700DomainApis.scala b/obp-api/src/main/scala/code/api/v7_0_0/Http4s700DomainApis.scala new file mode 100644 index 0000000000..b99e789085 --- /dev/null +++ b/obp-api/src/main/scala/code/api/v7_0_0/Http4s700DomainApis.scala @@ -0,0 +1,297 @@ +/** +Open Bank Project - API +Copyright (C) 2011-2026, TESOBE GmbH. + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +Email: contact@tesobe.com +TESOBE GmbH. +Osloer Strasse 16/17 +Berlin 13359, Germany + +This product includes software developed at +TESOBE (http://www.tesobe.com/) + + */ +package code.api.v7_0_0 + +import cats.effect.IO +import code.api.Constant.ApiPathZero +import code.api.dynamic.domainapi.{DomainApiPaths, Http4sDomainApi} +import code.api.dynamic.entity.helper.DynamicEntitySpace +import code.api.util.APIUtil.{EmptyBody, ResourceDoc} +import code.api.util.ApiRole._ +import code.api.util.ApiTag._ +import code.api.util.ErrorMessages._ +import code.api.util.http4s.Http4sRequestAttributes.EndpointHelpers +import code.api.util.{CallContext, CustomJsonFormats, Glossary} +import code.domainapi.{DomainApiDbProvider, DomainApiTrait, DomainApis} +import code.util.Helper +import com.github.dwickern.macros.NameOf.nameOf +import com.openbankproject.commons.ExecutionContext.Implicits.global +import com.openbankproject.commons.util.ApiVersion +import net.liftweb.common.Box +import org.http4s._ +import org.http4s.dsl.io._ +import org.json4s.Formats + +import scala.collection.mutable.ArrayBuffer +import scala.concurrent.Future + +/** + * This object holds the v7.0.0 endpoints that manage Domain APIs: a space's Dynamic Entities and Dynamic + * Resource Docs published under a base path of its own (see [[code.domainapi.DomainApis]] and the Glossary + * item "Domain APIs"). The calls under a base path are served by + * [[code.api.dynamic.domainapi.Http4sDomainApi]], not here. + * + * As for the v7.0.0 Dynamic Entity definitions, BANK_ID is a bank's id or SYS for the system space; the + * ResourceDocs declare allowSystemSpace() so the middleware lets SYS through and checks each Role at the + * BANK_ID in the URL. + * + * Declared in its own object to keep Http4s700's initialiser under the JVM's 64KB method limit. + */ +object Http4s700DomainApis { + + implicit val formats: Formats = CustomJsonFormats.formats + + private val implementedInApiVersion = ApiVersion.v7_0_0 + private val prefixPath = Root / ApiPathZero.toString / implementedInApiVersion.toString + + val resourceDocs = ArrayBuffer[ResourceDoc]() + + private val MaxTitleLength = 255 + private val MaxDescriptionLength = 2000 + + private def provider = DomainApis.domainApiProvider.vend + + private def space(bankIdInUrl: String): String = + DynamicEntitySpace.bankIdOrSystem(DynamicEntitySpace.bankIdOrNoneForSystem(bankIdInUrl)) + + private case class Checked(basePath: String, version: String, title: String, description: String) + + /** + * The checks a registration must pass, on create and on update (`domainApiId` is the one being updated, + * which may keep its own base path): the base path's shape, the version, the title, that no other + * Domain API's base path overlaps it, and that no two of the space's endpoints would publish the same + * verb and path. + */ + private def check(spaceId: String, body: PostDomainApiJsonV700, domainApiId: Option[String], cc: CallContext): Future[Checked] = { + val basePath = Option(body.base_path).map(_.trim).getOrElse("") + val version = Option(body.version).map(_.trim).getOrElse("") + val title = Option(body.title).map(_.trim).getOrElse("") + val description = body.description.map(_.trim).getOrElse("") + val basePathProblem = DomainApiPaths.basePathProblem(basePath) + lazy val overlapping = provider.getAllInEverySpace().openOr(Nil) + .filterNot(other => domainApiId.contains(other.domainApiId)) + .filter(other => DomainApiPaths.overlap(other.basePath, basePath)) + .map(other => s"${other.basePath} (bank_id ${other.bankId})") + lazy val clashes = DomainApiPaths.clashes(spaceId, Http4sDomainApi.spaceDocs(spaceId)) + for { + _ <- Helper.booleanToFuture(s"$InvalidDomainApiBasePath${DomainApiPaths.reservedFirstSegments.toList.sorted.mkString(", ")}. Current base_path is $basePath: ${basePathProblem.getOrElse("")}.", 400, Some(cc)) { + basePathProblem.isEmpty + } + _ <- Helper.booleanToFuture(s"$InvalidDomainApiVersion Current version is $version.", 400, Some(cc)) { + DomainApiPaths.versionFits(version, basePath) + } + _ <- Helper.booleanToFuture(InvalidDomainApiTitle, 400, Some(cc)) { + title.nonEmpty && title.length <= MaxTitleLength && description.length <= MaxDescriptionLength + } + _ <- Helper.booleanToFuture(s"$DomainApiBasePathAlreadyExists${overlapping.mkString(", ")}", 409, Some(cc)) { + overlapping.isEmpty + } + _ <- Helper.booleanToFuture(s"$DomainApiPathClash${clashes.mkString("; ")}", 409, Some(cc)) { + clashes.isEmpty + } + } yield Checked(basePath, version, title, description) + } + + private def fullOrFail[T: Manifest](box: Box[T], cc: CallContext): T = + code.api.util.APIUtil.unboxFullOrFail(box, Some(cc), UnknownError, 400) + + private def found(spaceId: String, domainApiId: String, cc: CallContext): Future[DomainApiTrait] = + Future(provider.get(spaceId, domainApiId)).map(box => + code.api.util.APIUtil.unboxFullOrFail(box, Some(cc), s"$DomainApiNotFound Current DOMAIN_API_ID is $domainApiId.", 404)) + + private val domainApisDescription = + s"""A Domain API publishes the Dynamic Entities and Dynamic Resource Docs (Dynamic Queries included) of one + |space under a base path of its own, without OBP's own URL structure in front of them: with the base path + |`carbon-registry/v1` over the system space, `/carbon-registry/v1/activity` answers what + |`/obp/v7.0.0/banks/SYS/dynamic-entities/activity` answers, and `/carbon-registry/v1/openapi.yaml` (or + |`openapi.json`) is its OpenAPI document. It only renames: every call runs the same authentication, Roles + |and access checks as the OBP URL, and a Dynamic Entity record response leaves out `bank_id`. + | + |BANK_ID is the space: a bank's id, or `SYS` for the system space. The Role is checked at that BANK_ID. + | + |`base_path` is two to five segments of lowercase letters, digits, hyphens and dots, ending with the + |major version as `vN`, and must not overlap another Domain API's. `version` is the full semantic + |version, MAJOR.MINOR.PATCH, whose MAJOR is the N of the base path: a compatible change edits `version` + |and leaves every URL alone, and a breaking change gets a new Domain API with a new base path. + | + |A Domain API is refused while two endpoints of its space would answer the same verb and path under it, + |or while a Dynamic Resource Doc's path starts with a segment the Dynamic Entity URLs or the + |documentation use (`my`, `public`, `community`, `openapi.json`, `openapi.yaml`). + | + |On this instance a change is seen at once by the node that made it, and by the others within + |${DomainApiDbProvider.cacheTtlSeconds} seconds. + | + |For more information see ${Glossary.getGlossaryItemLink("Domain APIs")}""".stripMargin + + private val errorsOnWrite = List($BankNotFound, $AuthenticatedUserIsRequired, UserHasMissingRoles, InvalidJsonFormat, + InvalidDomainApiBasePath, InvalidDomainApiVersion, InvalidDomainApiTitle, DomainApiBasePathAlreadyExists, + DomainApiPathClash, UnknownError) + + // Route: POST /obp/v7.0.0/management/banks/BANK_ID/domain-apis (201) + lazy val createDomainApi: HttpRoutes[IO] = HttpRoutes.of[IO] { + case req @ POST -> `prefixPath` / "management" / "banks" / bankIdInUrl / "domain-apis" => + EndpointHelpers.withUserAndBodyCreated[PostDomainApiJsonV700, DomainApiJsonV700](req) { (user, body, cc) => + val spaceId = space(bankIdInUrl) + for { + checked <- check(spaceId, body, None, cc) + created <- Future(provider.create(spaceId, checked.basePath, checked.version, checked.title, checked.description, user.userId)) + .map(fullOrFail(_, cc)) + } yield JSONFactory700DomainApis.createDomainApiJson(created) + } + } + + resourceDocs += ResourceDoc( + implementedInApiVersion, + nameOf(createDomainApi), + "POST", + "/management/banks/BANK_ID/domain-apis", + "Create Domain API", + s"""Publish the Dynamic Entities and Dynamic Resource Docs of a space under a base path of its own. + | + |$domainApisDescription""".stripMargin, + JSONFactory700DomainApis.postDomainApiJsonV700Example, + JSONFactory700DomainApis.domainApiJsonV700Example, + errorsOnWrite, + apiTagDynamic :: apiTagApi :: Nil, + Some(canCreateDomainApi :: Nil), + http4sPartialFunction = Some(createDomainApi) + ).allowSystemSpace() + + // Route: GET /obp/v7.0.0/management/banks/BANK_ID/domain-apis + lazy val getDomainApis: HttpRoutes[IO] = HttpRoutes.of[IO] { + case req @ GET -> `prefixPath` / "management" / "banks" / bankIdInUrl / "domain-apis" => + EndpointHelpers.withUser(req) { (_, cc) => + Future(provider.getAll(space(bankIdInUrl))).map(box => + DomainApisJsonV700(fullOrFail(box, cc).map(JSONFactory700DomainApis.createDomainApiJson))) + } + } + + resourceDocs += ResourceDoc( + implementedInApiVersion, + nameOf(getDomainApis), + "GET", + "/management/banks/BANK_ID/domain-apis", + "Get Domain APIs", + s"""The Domain APIs of a space. + | + |$domainApisDescription""".stripMargin, + EmptyBody, + JSONFactory700DomainApis.domainApisJsonV700Example, + List($BankNotFound, $AuthenticatedUserIsRequired, UserHasMissingRoles, UnknownError), + apiTagDynamic :: apiTagApi :: Nil, + Some(canGetDomainApis :: Nil), + http4sPartialFunction = Some(getDomainApis) + ).allowSystemSpace() + + // Route: GET /obp/v7.0.0/management/banks/BANK_ID/domain-apis/DOMAIN_API_ID + lazy val getDomainApi: HttpRoutes[IO] = HttpRoutes.of[IO] { + case req @ GET -> `prefixPath` / "management" / "banks" / bankIdInUrl / "domain-apis" / domainApiId => + EndpointHelpers.withUser(req) { (_, cc) => + found(space(bankIdInUrl), domainApiId, cc).map(JSONFactory700DomainApis.createDomainApiJson) + } + } + + resourceDocs += ResourceDoc( + implementedInApiVersion, + nameOf(getDomainApi), + "GET", + "/management/banks/BANK_ID/domain-apis/DOMAIN_API_ID", + "Get Domain API", + s"""One Domain API of a space. + | + |$domainApisDescription""".stripMargin, + EmptyBody, + JSONFactory700DomainApis.domainApiJsonV700Example, + List($BankNotFound, $AuthenticatedUserIsRequired, UserHasMissingRoles, DomainApiNotFound, UnknownError), + apiTagDynamic :: apiTagApi :: Nil, + Some(canGetDomainApis :: Nil), + http4sPartialFunction = Some(getDomainApi) + ).allowSystemSpace() + + // Route: PUT /obp/v7.0.0/management/banks/BANK_ID/domain-apis/DOMAIN_API_ID + lazy val updateDomainApi: HttpRoutes[IO] = HttpRoutes.of[IO] { + case req @ PUT -> `prefixPath` / "management" / "banks" / bankIdInUrl / "domain-apis" / domainApiId => + EndpointHelpers.withUserAndBody[PostDomainApiJsonV700, DomainApiJsonV700](req) { (_, body, cc) => + val spaceId = space(bankIdInUrl) + for { + _ <- found(spaceId, domainApiId, cc) + checked <- check(spaceId, body, Some(domainApiId), cc) + updated <- Future(provider.update(spaceId, domainApiId, checked.basePath, checked.version, checked.title, checked.description)) + .map(fullOrFail(_, cc)) + } yield JSONFactory700DomainApis.createDomainApiJson(updated) + } + } + + resourceDocs += ResourceDoc( + implementedInApiVersion, + nameOf(updateDomainApi), + "PUT", + "/management/banks/BANK_ID/domain-apis/DOMAIN_API_ID", + "Update Domain API", + s"""Change a Domain API: its base path, version, title or description. The same checks as on create apply. + |Changing the base path moves every published URL, so it is a breaking change for its clients; a + |compatible change only edits `version`. + | + |$domainApisDescription""".stripMargin, + JSONFactory700DomainApis.postDomainApiJsonV700Example, + JSONFactory700DomainApis.domainApiJsonV700Example, + DomainApiNotFound :: errorsOnWrite, + apiTagDynamic :: apiTagApi :: Nil, + Some(canUpdateDomainApi :: Nil), + http4sPartialFunction = Some(updateDomainApi) + ).allowSystemSpace() + + // Route: DELETE /obp/v7.0.0/management/banks/BANK_ID/domain-apis/DOMAIN_API_ID (204) + lazy val deleteDomainApi: HttpRoutes[IO] = HttpRoutes.of[IO] { + case req @ DELETE -> `prefixPath` / "management" / "banks" / bankIdInUrl / "domain-apis" / domainApiId => + EndpointHelpers.withUserDelete(req) { (_, cc) => + val spaceId = space(bankIdInUrl) + for { + _ <- found(spaceId, domainApiId, cc) + deleted <- Future(provider.delete(spaceId, domainApiId)).map(fullOrFail(_, cc)) + } yield deleted + } + } + + resourceDocs += ResourceDoc( + implementedInApiVersion, + nameOf(deleteDomainApi), + "DELETE", + "/management/banks/BANK_ID/domain-apis/DOMAIN_API_ID", + "Delete Domain API", + s"""Stop publishing a space under a Domain API's base path. The Dynamic Entities and Dynamic Resource Docs + |are not changed and stay available at their OBP URLs. + | + |$domainApisDescription""".stripMargin, + EmptyBody, + EmptyBody, + List($BankNotFound, $AuthenticatedUserIsRequired, UserHasMissingRoles, DomainApiNotFound, UnknownError), + apiTagDynamic :: apiTagApi :: Nil, + Some(canDeleteDomainApi :: Nil), + http4sPartialFunction = Some(deleteDomainApi) + ).allowSystemSpace() +} diff --git a/obp-api/src/main/scala/code/api/v7_0_0/JSONFactory700DomainApis.scala b/obp-api/src/main/scala/code/api/v7_0_0/JSONFactory700DomainApis.scala new file mode 100644 index 0000000000..2bfa6c0c55 --- /dev/null +++ b/obp-api/src/main/scala/code/api/v7_0_0/JSONFactory700DomainApis.scala @@ -0,0 +1,98 @@ +/** +Open Bank Project - API +Copyright (C) 2011-2026, TESOBE GmbH. + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +Email: contact@tesobe.com +TESOBE GmbH. +Osloer Strasse 16/17 +Berlin 13359, Germany + +This product includes software developed at +TESOBE (http://www.tesobe.com/) + + */ +package code.api.v7_0_0 + +import java.util.Date + +import code.api.Constant.HostName +import code.domainapi.DomainApiTrait + +/* + * The JSON of the v7.0.0 Domain API endpoints. Package-level case classes, for the reason given in + * JSONFactory700Operations. + */ + +/** Register or change a Domain API. */ +case class PostDomainApiJsonV700(base_path: String, version: String, title: String, description: Option[String]) + +case class DomainApiJsonV700( + domain_api_id: String, + bank_id: String, + base_path: String, + version: String, + title: String, + description: String, + /** Where the Domain API is published: this instance's host followed by the base path. */ + url: String, + /** Its OpenAPI document. */ + openapi_url: String, + created_by_user_id: String, + created_at: Date, + updated_at: Date +) + +case class DomainApisJsonV700(domain_apis: List[DomainApiJsonV700]) + +object JSONFactory700DomainApis { + + def createDomainApiJson(domainApi: DomainApiTrait): DomainApiJsonV700 = DomainApiJsonV700( + domain_api_id = domainApi.domainApiId, + bank_id = domainApi.bankId, + base_path = domainApi.basePath, + version = domainApi.version, + title = domainApi.title, + description = domainApi.description, + url = s"$HostName/${domainApi.basePath}", + openapi_url = s"$HostName/${domainApi.basePath}/openapi.yaml", + created_by_user_id = domainApi.createdByUserId, + created_at = domainApi.createdAt, + updated_at = domainApi.updatedAt + ) + + val postDomainApiJsonV700Example = PostDomainApiJsonV700( + base_path = "carbon-registry/v1", + version = "1.0.0", + title = "Open Carbon Registry API", + description = Some("Activities and land parcels of the registry.") + ) + + val domainApiJsonV700Example = DomainApiJsonV700( + domain_api_id = "5f4a1c8e-2b7d-4e9a-9c3f-1d2e3f4a5b6c", + bank_id = "SYS", + base_path = "carbon-registry/v1", + version = "1.0.0", + title = "Open Carbon Registry API", + description = "Activities and land parcels of the registry.", + url = "https://api.example.org/carbon-registry/v1", + openapi_url = "https://api.example.org/carbon-registry/v1/openapi.yaml", + created_by_user_id = "9ca9a7e4-6d02-40e3-a129-0b2bf89de9b1", + created_at = new Date(), + updated_at = new Date() + ) + + val domainApisJsonV700Example = DomainApisJsonV700(List(domainApiJsonV700Example)) +} diff --git a/obp-api/src/main/scala/code/domainapi/DomainApi.scala b/obp-api/src/main/scala/code/domainapi/DomainApi.scala new file mode 100644 index 0000000000..c53b1bed2f --- /dev/null +++ b/obp-api/src/main/scala/code/domainapi/DomainApi.scala @@ -0,0 +1,209 @@ +/** +Open Bank Project - API +Copyright (C) 2011-2026, TESOBE GmbH. + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +Email: contact@tesobe.com +TESOBE GmbH. +Osloer Strasse 16/17 +Berlin 13359, Germany + +This product includes software developed at +TESOBE (http://www.tesobe.com/) + + */ + +package code.domainapi + +import java.util.Date +import java.util.concurrent.atomic.AtomicReference + +import code.api.Constant.DYNAMIC_ENTITY_SYSTEM_LEVEL_BANK_ID +import code.api.util.APIUtil +import net.liftweb.common.Box +import net.liftweb.mapper._ +import net.liftweb.util.Helpers.tryo +import net.liftweb.util.SimpleInjector + +/** + * A Domain API publishes the dynamic endpoints of one space (a bank, or SYS for the system space) under a + * base path of its own, such as `carbon-registry/v1`, without OBP's own URL structure in front of them. + * It only renames: a call under the base path is rewritten to the existing URL of the endpoint and runs + * exactly as that call would, with the same authentication, Roles and access checks. + * See the Glossary item "Domain APIs" and ideas/DOMAIN_APIS.md. + */ +object DomainApis extends SimpleInjector { + val domainApiProvider = new Inject(() => buildOne) {} + def buildOne: DomainApiProvider = DomainApiDbProvider +} + +trait DomainApiTrait { + def domainApiId: String + /** A bank id, or SYS for the system space. Never empty. */ + def bankId: String + /** The path the space's endpoints are published under, without a leading or trailing slash. */ + def basePath: String + /** The full semantic version, MAJOR.MINOR.PATCH. */ + def version: String + def title: String + def description: String + def createdByUserId: String + def createdAt: Date + def updatedAt: Date +} + +/** A Domain API as the request path reads it: held in memory, so a request does not query the table. */ +case class DomainApiRoute(domainApiId: String, bankId: String, basePath: String, version: String, + title: String, description: String) { + val basePathSegments: List[String] = basePath.split("/").toList +} + +trait DomainApiProvider { + def create(bankId: String, basePath: String, version: String, title: String, description: String, + createdByUserId: String): Box[DomainApiTrait] + def get(bankId: String, domainApiId: String): Box[DomainApiTrait] + def getAll(bankId: String): Box[List[DomainApiTrait]] + /** Every Domain API of every space, read from the table (not the request path's cached list). */ + def getAllInEverySpace(): Box[List[DomainApiTrait]] + def update(bankId: String, domainApiId: String, basePath: String, version: String, title: String, + description: String): Box[DomainApiTrait] + def delete(bankId: String, domainApiId: String): Box[Boolean] + /** Every Domain API of every space, for the request path; see [[DomainApiDbProvider.routes]]. */ + def routes(): List[DomainApiRoute] +} + +object DomainApiDbProvider extends DomainApiProvider { + + /** + * This is how long a node keeps its in-memory list of Domain APIs before reading the table again. + * + * Every request passes the Domain API front door, so the list cannot be read from the database per + * request. A node forgets its list at once when it writes a Domain API itself; another node of the same + * installation sees the change within this many seconds. + */ + val cacheTtlSeconds: Int = APIUtil.getPropsAsIntValue("domain_api.cache.ttl.seconds", 10) + + private case class Snapshot(routes: List[DomainApiRoute], loadedAt: Long) + private val snapshot = new AtomicReference[Option[Snapshot]](None) + + private def forget(): Unit = snapshot.set(None) + + override def routes(): List[DomainApiRoute] = { + val now = System.currentTimeMillis() + snapshot.get() match { + case Some(s) if now - s.loadedAt < cacheTtlSeconds * 1000L => s.routes + case _ => + val loaded = tryo(DomainApi.findAll()).openOr(Nil).map(d => + DomainApiRoute(d.domainApiId, d.bankId, d.basePath, d.version, d.title, d.description)) + snapshot.set(Some(Snapshot(loaded, now))) + loaded + } + } + + override def create(bankId: String, basePath: String, version: String, title: String, description: String, + createdByUserId: String): Box[DomainApiTrait] = { + val created = tryo { + DomainApi.create + .DomainApiId(APIUtil.generateUUID()) + .BankId(bankId) + .BasePath(basePath) + .Version(version) + .Title(title) + .Description(description) + // A consent user's Domain API belongs to the person the Consent is for (UserReference). + .CreatedByUserId(code.users.Users.users.vend.attributedUserId(createdByUserId, code.users.UserReference.DomainApi_CreatedByUserId).openOr(createdByUserId)) + .saveMe() + } + forget() + created + } + + override def get(bankId: String, domainApiId: String): Box[DomainApiTrait] = + DomainApi.find(By(DomainApi.BankId, bankId), By(DomainApi.DomainApiId, domainApiId)) + + override def getAll(bankId: String): Box[List[DomainApiTrait]] = + tryo(DomainApi.findAll(By(DomainApi.BankId, bankId), OrderBy(DomainApi.BasePath, Ascending))) + + override def getAllInEverySpace(): Box[List[DomainApiTrait]] = + tryo(DomainApi.findAll(OrderBy(DomainApi.BasePath, Ascending))) + + override def update(bankId: String, domainApiId: String, basePath: String, version: String, title: String, + description: String): Box[DomainApiTrait] = { + val updated = DomainApi.find(By(DomainApi.BankId, bankId), By(DomainApi.DomainApiId, domainApiId)).flatMap { row => + tryo { + row.BasePath(basePath).Version(version).Title(title).Description(description).UpdatedAt(new Date()).saveMe() + } + } + forget() + updated + } + + override def delete(bankId: String, domainApiId: String): Box[Boolean] = { + val deleted = DomainApi.find(By(DomainApi.BankId, bankId), By(DomainApi.DomainApiId, domainApiId)).flatMap(row => tryo(row.delete_!)) + forget() + deleted + } +} + +class DomainApi extends DomainApiTrait with LongKeyedMapper[DomainApi] with IdPK { + def getSingleton = DomainApi + + object DomainApiId extends MappedString(this, 36) { + override def dbColumnName = "domain_api_id" + } + // SYS for the system space, never NULL: see DynamicEntitySpace. + object BankId extends MappedString(this, 255) { + override def dbColumnName = "bank_id" + override def defaultValue = DYNAMIC_ENTITY_SYSTEM_LEVEL_BANK_ID + } + object BasePath extends MappedString(this, 255) { + override def dbColumnName = "base_path" + } + object Version extends MappedString(this, 50) { + override def dbColumnName = "version" + } + object Title extends MappedString(this, 255) { + override def dbColumnName = "title" + } + object Description extends MappedString(this, 2000) { + override def dbColumnName = "description" + } + object CreatedByUserId extends MappedString(this, 255) { + override def dbColumnName = "created_by_user_id" + } + object CreatedAt extends MappedDateTime(this) { + override def dbColumnName = "created_at" + override def defaultValue = new Date() + } + object UpdatedAt extends MappedDateTime(this) { + override def dbColumnName = "updated_at" + override def defaultValue = new Date() + } + + override def domainApiId: String = DomainApiId.get + override def bankId: String = BankId.get + override def basePath: String = BasePath.get + override def version: String = Version.get + override def title: String = Title.get + override def description: String = Option(Description.get).getOrElse("") + override def createdByUserId: String = CreatedByUserId.get + override def createdAt: Date = CreatedAt.get + override def updatedAt: Date = UpdatedAt.get +} + +object DomainApi extends DomainApi with LongKeyedMetaMapper[DomainApi] { + override def dbTableName = "domain_api" + override def dbIndexes = UniqueIndex(DomainApiId) :: UniqueIndex(BasePath) :: Index(BankId) :: super.dbIndexes +} diff --git a/obp-api/src/main/scala/code/dynamicResourceDoc/DynamicResourceDoc.scala b/obp-api/src/main/scala/code/dynamicResourceDoc/DynamicResourceDoc.scala index 3264b77e91..1de0e5a669 100644 --- a/obp-api/src/main/scala/code/dynamicResourceDoc/DynamicResourceDoc.scala +++ b/obp-api/src/main/scala/code/dynamicResourceDoc/DynamicResourceDoc.scala @@ -74,9 +74,14 @@ class DynamicResourceDoc extends LongKeyedMapper[DynamicResourceDoc] with IdPK w object DynamicResourceDoc extends DynamicResourceDoc with LongKeyedMetaMapper[DynamicResourceDoc] { - override def dbIndexes: List[BaseIndex[DynamicResourceDoc]] = UniqueIndex(DynamicResourceDocId) :: UniqueIndex(RequestUrl,RequestVerb) :: super.dbIndexes + // A verb and URL are unique within a space (BankId, which is SYS for the system space), not across spaces: + // a doc is served under its space. Databases that predate this had UniqueIndex(RequestUrl, RequestVerb) and + // stored NULL for a system level doc; Migration.database.prepareDynamicEntitySpaceScopedIndexes, which Boot + // runs on every start, drops that index and moves those NULLs to SYS. + override def dbIndexes: List[BaseIndex[DynamicResourceDoc]] = UniqueIndex(DynamicResourceDocId) :: UniqueIndex(BankId, RequestUrl, RequestVerb) :: super.dbIndexes def getJsonDynamicResourceDoc(dynamicResourceDoc: DynamicResourceDoc) = JsonDynamicResourceDoc( - bankId = Some(dynamicResourceDoc.BankId.get), + // The row stores SYS for the system space; in memory the system space is None, as for a Dynamic Entity. + bankId = code.api.dynamic.entity.helper.DynamicEntitySpace.bankIdOrNoneForSystem(dynamicResourceDoc.BankId.get), dynamicResourceDocId = Some(dynamicResourceDoc.DynamicResourceDocId.get), methodBody = dynamicResourceDoc.MethodBody.get, partialFunctionName = dynamicResourceDoc.PartialFunctionName.get, diff --git a/obp-api/src/main/scala/code/dynamicResourceDoc/MappedDynamicResourceDocProvider.scala b/obp-api/src/main/scala/code/dynamicResourceDoc/MappedDynamicResourceDocProvider.scala index ac1160efd5..a2bc52dd3a 100644 --- a/obp-api/src/main/scala/code/dynamicResourceDoc/MappedDynamicResourceDocProvider.scala +++ b/obp-api/src/main/scala/code/dynamicResourceDoc/MappedDynamicResourceDocProvider.scala @@ -42,6 +42,14 @@ import scala.concurrent.duration.DurationInt object MappedDynamicResourceDocProvider extends DynamicResourceDocProvider { + /** + * The bank id a doc's row stores. A system level doc stores the SYS sentinel rather than a SQL NULL, as a + * Dynamic Entity does: a NULL is never equal to another NULL in a unique index on Postgres or H2, so the + * index on (BankId, RequestUrl, RequestVerb) could not keep two system level docs off the same URL. + */ + private def storedBankId(bankId: Option[String]): String = + code.api.dynamic.entity.helper.DynamicEntitySpace.bankIdOrSystem(bankId) + private val getDynamicResourceDocTTL : Int = { if(Props.testMode) 0 //make the scala test work else APIUtil.getPropsValue(s"dynamicResourceDoc.cache.ttl.seconds", "40").toInt @@ -62,19 +70,15 @@ object MappedDynamicResourceDocProvider extends DynamicResourceDocProvider { } } + // A verb and URL are unique within one space, so this looks in the given space only; the system space + // (bankId None) is the rows stored with the SYS bank id. override def getByVerbAndUrl(bankId: Option[String], requestVerb: String, requestUrl: String): Box[JsonDynamicResourceDoc] = - if(bankId.isEmpty){ - DynamicResourceDoc - .find(By(DynamicResourceDoc.RequestVerb, requestVerb), By(DynamicResourceDoc.RequestUrl, requestUrl)) - .map(DynamicResourceDoc.getJsonDynamicResourceDoc) - } else{ - DynamicResourceDoc - .find( - By(DynamicResourceDoc.BankId, bankId.getOrElse("")), - By(DynamicResourceDoc.RequestVerb, requestVerb), - By(DynamicResourceDoc.RequestUrl, requestUrl)) - .map(DynamicResourceDoc.getJsonDynamicResourceDoc) - } + DynamicResourceDoc + .find( + By(DynamicResourceDoc.BankId, storedBankId(bankId)), + By(DynamicResourceDoc.RequestVerb, requestVerb), + By(DynamicResourceDoc.RequestUrl, requestUrl)) + .map(DynamicResourceDoc.getJsonDynamicResourceDoc) override def getAllAndConvert[T: Manifest](bankId: Option[String], transform: JsonDynamicResourceDoc => T): List[T] = { val cacheKey = (bankId.toString+transform.toString()).intern() @@ -96,7 +100,7 @@ object MappedDynamicResourceDocProvider extends DynamicResourceDocProvider { val responseBody = entity.successResponseBody.map(json.compactRender(_)).orNull DynamicResourceDoc.create - .BankId(bankId.getOrElse(null)) + .BankId(storedBankId(bankId)) .DynamicResourceDocId(APIUtil.generateUUID()) .PartialFunctionName(entity.partialFunctionName) .RequestVerb(entity.requestVerb) @@ -124,7 +128,7 @@ object MappedDynamicResourceDocProvider extends DynamicResourceDocProvider { val requestBody = entity.exampleRequestBody.map(json.compactRender(_)).orNull val responseBody = entity.successResponseBody.map(json.compactRender(_)).orNull v.PartialFunctionName(entity.partialFunctionName) - .BankId(bankId.getOrElse(null)) + .BankId(storedBankId(bankId)) .RequestVerb(entity.requestVerb) .RequestUrl(entity.requestUrl) .Summary(entity.summary) diff --git a/obp-api/src/main/scala/code/users/UserReference.scala b/obp-api/src/main/scala/code/users/UserReference.scala index ea8c9140b2..76ad8e9613 100644 --- a/obp-api/src/main/scala/code/users/UserReference.scala +++ b/obp-api/src/main/scala/code/users/UserReference.scala @@ -198,6 +198,7 @@ object UserReference { case object CounterpartyWhereTag_User extends UserReference(UseOnBehalfOfUserId , "code.metadata.counterparties.MappedCounterpartyWhereTag", List("user"), "who tagged the counterparty's location") case object ApiProductSubscription_CreatedByUserId extends UserReference(UseOnBehalfOfUserId , "code.apiproductsubscription.ApiProductSubscription", List("CreatedByUserId"), "a subscription outlives the Consent that took it out") case object DynamicGlossaryItem_CreatedByUserId extends UserReference(UseOnBehalfOfUserId , "code.glossaryitem.DynamicGlossaryItem", List("CreatedByUserId"), "outlives the Consent that created it") + case object DomainApi_CreatedByUserId extends UserReference(UseOnBehalfOfUserId , "code.domainapi.DomainApi", List("CreatedByUserId"), "the Domain API's creator; a published API outlives the Consent that created it") case object Bank_CreatedByUserId extends UserReference(UseOnBehalfOfUserId , "code.model.dataAccess.MappedBank", List("CreatedByUserId"), "creator grant already resolved at the endpoint") case object Organisation_CreatedByUserId extends UserReference(UseOnBehalfOfUserId , "code.organisation.Organisation", List("CreatedByUserId"), "outlives the Consent that created it") case object PayeeLookup_CreatedByUserId extends UserReference(UseOnBehalfOfUserId , "code.payeelookup.PayeeLookup", List("CreatedByUserId"), "outlives the Consent that created it") @@ -287,6 +288,7 @@ object UserReference { CounterpartyWhereTag_User, ApiProductSubscription_CreatedByUserId, DynamicGlossaryItem_CreatedByUserId, + DomainApi_CreatedByUserId, Bank_CreatedByUserId, Organisation_CreatedByUserId, PayeeLookup_CreatedByUserId, diff --git a/obp-api/src/test/scala/code/api/v4_0_0/DynamicQueryTest.scala b/obp-api/src/test/scala/code/api/v4_0_0/DynamicQueryTest.scala index 57fce21c57..b61bf10e30 100644 --- a/obp-api/src/test/scala/code/api/v4_0_0/DynamicQueryTest.scala +++ b/obp-api/src/test/scala/code/api/v4_0_0/DynamicQueryTest.scala @@ -200,9 +200,9 @@ class DynamicQueryTest extends V400ServerSetup { makeGetRequest(call.GET <@ (user1)).code should equal(200) code.metrics.MetricBatchWriter.flush() - Then("both calls have a metric row, with their status, verb and the doc's function name") + Then("both calls have a metric row, with their status, verb and the doc's function name, under the URL that names the doc's space") val metrics = makeGetRequest((baseRequest / "obp" / "v6.0.0" / "management" / "metrics").GET <@ (user1) < s"/obp/dynamic-endpoint/dynamic-resource-doc/dq_audited_$sfx", "limit" -> "10")) + "url" -> s"/obp/dynamic-endpoint/banks/SYS/dynamic-resource-doc/dq_audited_$sfx", "limit" -> "10")) metrics.code should equal(200) val rows = (metrics.body \ "metrics").children rows.map(row => (row \ "status_code").extract[Int]).sorted shouldBe List(200, 403) diff --git a/obp-api/src/test/scala/code/api/v4_0_0/DynamicResourceDocSpaceUrlTest.scala b/obp-api/src/test/scala/code/api/v4_0_0/DynamicResourceDocSpaceUrlTest.scala new file mode 100644 index 0000000000..9fdb69e456 --- /dev/null +++ b/obp-api/src/test/scala/code/api/v4_0_0/DynamicResourceDocSpaceUrlTest.scala @@ -0,0 +1,135 @@ +package code.api.v4_0_0 + +import code.DynamicData.DynamicDataProvider +import code.api.ResourceDocs1_4_0.SwaggerDefinitionsJSON +import code.api.dynamic.endpoint.helper.DynamicEndpoints +import code.api.util.APIUtil.OAuth._ +import code.api.util.ErrorMessages.DynamicResourceDocUrlAmbiguous +import code.dynamicEntity.{DynamicEntityCommons, DynamicEntityProvider} +import code.dynamicResourceDoc.{DynamicResourceDoc, DynamicResourceDocProvider} +import net.liftweb.common.Full +import net.liftweb.mapper.By +import code.entitlement.Entitlement +import com.openbankproject.commons.model.ErrorMessage +import net.liftweb.util.StringHelpers +import org.json4s._ + +import java.net.URLEncoder + +/** + * This suite checks that a Dynamic Resource Doc is served under the space it belongs to: + * /obp/dynamic-endpoint/banks/BANK_ID/dynamic-resource-doc/REQUEST_URL, with SYS for the system space, + * so docs of different spaces at the same URL never answer for one another. The URL without the space + * still works while exactly one doc answers it, and is refused, naming the spaces, when several do. + * + * Each space gets an entity of the same name holding one record that names the space, and a Dynamic + * Query at the same URL reading it, so an answer shows which space's doc ran. + */ +class DynamicResourceDocSpaceUrlTest extends V400ServerSetup { + + private val owner = "space-url-owner" + private val sfx = java.util.UUID.randomUUID().toString.take(8).replace("-", "") + private val Echo = s"Echo$sfx" + private val bankA = testBankId1.value + private val bankB = testBankId2.value + private val requestUrl = s"/space_url_$sfx/names" + + /** Define the entity in a space (None = system), save one record naming the space, and let user1 read it there. */ + private def spaceWithRecord(space: Option[String], name: String): Unit = { + DynamicEntityProvider.connectorMethodProvider.vend.createOrUpdate(DynamicEntityCommons( + Echo, s"""{"$Echo":{"properties":{"${StringHelpers.snakify(Echo)}_id":{"type":"string"},"name":{"type":"string"}}}}""", + None, owner, space, hasPersonalEntity = false)).openOrThrowException("definition") + DynamicDataProvider.connectorMethodProvider.vend.save(space, Echo, + JObject(JField(StringHelpers.snakify(Echo) + "_id", JString(java.util.UUID.randomUUID().toString)), JField("name", JString(name))), + Some(owner), false).openOrThrowException("record") + Entitlement.entitlement.vend.addEntitlement(space.getOrElse("SYS"), resourceUser1.userId, s"CanGetDynamicEntityRecord_$Echo") + } + + /** A Dynamic Query in a space (None = system) at the shared URL, reading that space's entity. */ + private def queryDoc(space: Option[String]): Unit = queryDocAt(space, requestUrl) + + private def queryDocAt(space: Option[String], url: String): Unit = + DynamicResourceDocProvider.provider.vend.create(space, SwaggerDefinitionsJSON.jsonDynamicResourceDoc.copy( + dynamicResourceDocId = None, bankId = space, roles = "", partialFunctionName = s"spaceUrl${space.getOrElse("System").capitalize}", + requestVerb = "GET", requestUrl = url, exampleRequestBody = None, errorResponseBodies = "OBP-50000: Unknown Error.", + methodBody = URLEncoder.encode(s"""{ "from": "$Echo", "select": ["name"], "envelope": { "rows": "names" } }""", "UTF-8"), + programmingLang = "Query"), Some(owner)).openOrThrowException(s"doc in $space") + + private def underSpace(space: String) = dynamicEndpoint_Request / "banks" / space / "dynamic-resource-doc" / s"space_url_$sfx" / "names" + private def withoutSpace = dynamicEndpoint_Request / "dynamic-resource-doc" / s"space_url_$sfx" / "names" + private def names(response: code.setup.APIResponse): List[JValue] = + withClue(s"response: ${response.body}") { + response.code should equal(200) + (response.body \ "names" \ "name") match { case JArray(values) => values; case single => List(single) } + } + + feature("A Dynamic Resource Doc is served under the space it belongs to") { + scenario("system and bank docs at the same URL answer under their own space; the URL without a space works only while it is unambiguous") { + spaceWithRecord(None, "system space") + spaceWithRecord(Some(bankA), "bank A") + spaceWithRecord(Some(bankB), "bank B") + + Given("a system-level doc only") + queryDoc(None) + Then("it answers under /banks/SYS, and under the URL without a space") + names(makeGetRequest(underSpace("SYS") <@ (user1))) shouldBe List(JString("system space")) + names(makeGetRequest(withoutSpace <@ (user1))) shouldBe List(JString("system space")) + And("not under a bank") + makeGetRequest(underSpace(bankA) <@ (user1)).code should equal(404) + + And("the registry shows the URL that names its space") + DynamicEndpoints.dynamicResourceDocs.map(_.requestUrl) should contain(s"/banks/SYS/dynamic-resource-doc$requestUrl") + + When("bank A gets a doc at the same URL") + queryDoc(Some(bankA)) + Then("each answers under its own space, with its own space's data") + names(makeGetRequest(underSpace(bankA) <@ (user1))) shouldBe List(JString("bank A")) + names(makeGetRequest(underSpace("SYS") <@ (user1))) shouldBe List(JString("system space")) + And("the URL without a space no longer says which, so it is refused, naming both spaces") + val ambiguous = makeGetRequest(withoutSpace <@ (user1)) + ambiguous.code should equal(409) + ambiguous.body.extract[ErrorMessage].message shouldBe s"$DynamicResourceDocUrlAmbiguous${List("SYS", bankA).sorted.mkString(", ")}." + + When("bank B gets one too") + queryDoc(Some(bankB)) + names(makeGetRequest(underSpace(bankB) <@ (user1))) shouldBe List(JString("bank B")) + names(makeGetRequest(underSpace(bankA) <@ (user1))) shouldBe List(JString("bank A")) + makeGetRequest(withoutSpace <@ (user1)).body.extract[ErrorMessage].message shouldBe + s"$DynamicResourceDocUrlAmbiguous${List("SYS", bankA, bankB).sorted.mkString(", ")}." + } + + scenario("a URL is unique within a space: the same verb and URL cannot be created twice in one space") { + val url = s"/space_url_unique_$sfx" + def create(space: Option[String]) = DynamicResourceDocProvider.provider.vend.getByVerbAndUrl(space, "GET", url).isEmpty + queryDocAt(None, url) + queryDocAt(Some(bankA), url) + create(None) shouldBe false + create(Some(bankA)) shouldBe false + create(Some(bankB)) shouldBe true + + Then("a system level doc is stored with the SYS bank id, not NULL, and reads back as the system space") + val stored = DynamicResourceDoc.findAll(By(DynamicResourceDoc.RequestUrl, url), By(DynamicResourceDoc.RequestVerb, "GET")) + stored.map(_.BankId.get).sorted shouldBe List(bankA, "SYS").sorted + DynamicResourceDocProvider.provider.vend.getByVerbAndUrl(None, "GET", url).map(_.bankId) shouldBe Full(None) + + And("so the database itself refuses a second system level doc at the same verb and URL") + DynamicResourceDocProvider.provider.vend.create(None, SwaggerDefinitionsJSON.jsonDynamicResourceDoc.copy( + dynamicResourceDocId = None, bankId = None, roles = "", partialFunctionName = "spaceUrlSecondSystem", + requestVerb = "GET", requestUrl = url, exampleRequestBody = None, errorResponseBodies = "OBP-50000: Unknown Error.", + methodBody = URLEncoder.encode(s"""{ "from": "$Echo", "select": ["name"] }""", "UTF-8"), + programmingLang = "Query"), Some(owner)).isDefined shouldBe false + } + + scenario("a bank-level doc alone still answers at the URL without a space, as it did before") { + val onlyBank = s"/space_url_only_bank_$sfx/names" + spaceWithRecord(Some(bankB), "bank B") + DynamicResourceDocProvider.provider.vend.create(Some(bankB), SwaggerDefinitionsJSON.jsonDynamicResourceDoc.copy( + dynamicResourceDocId = None, bankId = Some(bankB), roles = "", partialFunctionName = "spaceUrlOnlyBank", + requestVerb = "GET", requestUrl = onlyBank, exampleRequestBody = None, errorResponseBodies = "OBP-50000: Unknown Error.", + methodBody = URLEncoder.encode(s"""{ "from": "$Echo", "select": ["name"], "envelope": { "rows": "names" } }""", "UTF-8"), + programmingLang = "Query"), Some(owner)).openOrThrowException("doc") + names(makeGetRequest(dynamicEndpoint_Request / "dynamic-resource-doc" / s"space_url_only_bank_$sfx" / "names" <@ (user1))) shouldBe List(JString("bank B")) + names(makeGetRequest(dynamicEndpoint_Request / "banks" / bankB / "dynamic-resource-doc" / s"space_url_only_bank_$sfx" / "names" <@ (user1))) shouldBe List(JString("bank B")) + } + } +} diff --git a/obp-api/src/test/scala/code/api/v7_0_0/DomainApisTest.scala b/obp-api/src/test/scala/code/api/v7_0_0/DomainApisTest.scala new file mode 100644 index 0000000000..e959679be4 --- /dev/null +++ b/obp-api/src/test/scala/code/api/v7_0_0/DomainApisTest.scala @@ -0,0 +1,266 @@ +/** +Open Bank Project - API +Copyright (C) 2011-2026, TESOBE GmbH. + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . + +Email: contact@tesobe.com +TESOBE GmbH. +Osloer Strasse 16/17 +Berlin 13359, Germany + +This product includes software developed at +TESOBE (http://www.tesobe.com/) + + */ +package code.api.v7_0_0 + +import code.DynamicData.DynamicDataProvider +import code.api.Constant.DYNAMIC_ENTITY_SYSTEM_LEVEL_BANK_ID +import code.api.ResourceDocs1_4_0.SwaggerDefinitionsJSON +import code.api.dynamic.domainapi.Http4sDomainApi +import code.api.util.APIUtil.OAuth._ +import code.api.util.ApiRole.{canCreateDomainApi, canDeleteDomainApi, canGetDomainApis, canUpdateDomainApi} +import code.api.util.ErrorMessages._ +import code.api.v6_0_0.V600ServerSetup +import code.domainapi.DomainApis +import code.dynamicEntity.{DynamicEntityCommons, DynamicEntityProvider} +import code.dynamicResourceDoc.DynamicResourceDocProvider +import code.entitlement.Entitlement +import com.openbankproject.commons.model.ErrorMessage +import com.openbankproject.commons.util.ApiVersion +import org.json4s.JsonAST._ +import org.json4s.JsonDSL._ +import org.json4s.native.JsonMethods.{compact, render} +import org.scalatest.Tag + +import java.net.URLEncoder + +/** + * This suite checks Domain APIs: a space's Dynamic Entities and Dynamic Resource Docs published under a + * base path of its own. It covers the v7.0.0 management endpoints (Roles at SYS and at a bank, the checks + * on base path, version and title, overlap and clash refusals), the front door (a Dynamic Entity and a + * Dynamic Query answered under the base path exactly as at their OBP URLs, without bank_id, with the same + * refusal for a caller who may not read), and the OpenAPI document (every documented path is served, and + * the documented example has the fields the real response has). + */ +class DomainApisTest extends V600ServerSetup { + + object VersionOfApi extends Tag(ApiVersion.v7_0_0.toString) + object ApiEndpoint1 extends Tag("createDomainApi") + object ApiEndpoint2 extends Tag("getDomainApis") + object ApiEndpoint3 extends Tag("getDomainApi") + object ApiEndpoint4 extends Tag("updateDomainApi") + object ApiEndpoint5 extends Tag("deleteDomainApi") + + private val SYS = DYNAMIC_ENTITY_SYSTEM_LEVEL_BANK_ID + private val owner = "domain-api-owner" + private val suffix = java.util.UUID.randomUUID().toString.replace("-", "").take(8) + private val entity = s"activity_$suffix" + private val queryPath = s"summary_$suffix" + + private def v7 = baseRequest / "obp" / "v7.0.0" + private def domainApis(bankId: String) = v7 / "management" / "banks" / bankId / "domain-apis" + private def registration(basePath: String, version: String, title: String = "Test Domain API") = + compact(render(("base_path" -> basePath) ~ ("version" -> version) ~ ("title" -> title) ~ ("description" -> "For the test."))) + private def message(response: code.setup.APIResponse) = response.body.extract[ErrorMessage].message + private def grantAt(bankId: String, role: code.api.util.ApiRole) = + Entitlement.entitlement.vend.addEntitlement(bankId, resourceUser1.userId, role.toString) + private def grantAllAt(bankId: String): Unit = + List(canCreateDomainApi, canGetDomainApis, canUpdateDomainApi, canDeleteDomainApi).foreach(grantAt(bankId, _)) + /** Every value of `field` anywhere in `json`, one or many. */ + private def valuesOf(json: JValue, field: String): List[String] = (json \\ field) match { + case JObject(fields) => fields.map(_._2.extract[String]) + case JNothing => Nil + case single => List(single.extract[String]) + } + private def under(basePath: String) = basePath.split("/").foldLeft(baseRequest)(_ / _) + + /** An entity in a space (None = system) holding one record, readable by user1 there. */ + private def entityWithRecord(space: Option[String], entityName: String, name: String): Unit = { + DynamicEntityProvider.connectorMethodProvider.vend.createOrUpdate(DynamicEntityCommons( + entityName, s"""{"$entityName":{"properties":{"${entityName}_id":{"type":"string","example":"1"},"name":{"type":"string","example":"tree planting"}}}}""", + None, owner, space, hasPersonalEntity = false)).openOrThrowException("definition") + DynamicDataProvider.connectorMethodProvider.vend.save(space, entityName, + JObject(JField(s"${entityName}_id", JString(java.util.UUID.randomUUID().toString)), JField("name", JString(name))), + Some(owner), false).openOrThrowException("record") + Entitlement.entitlement.vend.addEntitlement(space.getOrElse(SYS), resourceUser1.userId, s"CanGetDynamicEntityRecord_$entityName") + } + + /** A Dynamic Query in a space (None = system) at `url`, reading `entityName`. */ + private def queryDoc(space: Option[String], url: String, entityName: String, functionName: String): Unit = + DynamicResourceDocProvider.provider.vend.create(space, SwaggerDefinitionsJSON.jsonDynamicResourceDoc.copy( + dynamicResourceDocId = None, bankId = space, roles = "", partialFunctionName = functionName, + requestVerb = "GET", requestUrl = url, exampleRequestBody = None, errorResponseBodies = "OBP-50000: Unknown Error.", + methodBody = URLEncoder.encode(s"""{ "from": "$entityName", "select": ["name"], "envelope": { "rows": "names" } }""", "UTF-8"), + programmingLang = "Query"), Some(owner)).openOrThrowException(s"doc in $space") + + feature("Managing Domain APIs") { + + scenario("create, read, update and delete a Domain API at SYS, with the Roles held at SYS", + ApiEndpoint1, ApiEndpoint2, ApiEndpoint3, ApiEndpoint4, ApiEndpoint5, VersionOfApi) { + val basePath = s"managed-$suffix/v1" + + When("an anonymous caller creates one") + makePostRequest(domainApis(SYS).POST, registration(basePath, "1.0.0")).code should equal(401) + + When("user1 creates one without the Role") + val refused = makePostRequest(domainApis(SYS).POST <@ (user1), registration(basePath, "1.0.0")) + refused.code should equal(403) + message(refused) should include(UserHasMissingRoles) + + grantAllAt(SYS) + + When("user1 creates one with the Role at SYS") + val created = makePostRequest(domainApis(SYS).POST <@ (user1), registration(basePath, "1.0.0")) + withClue(created.body) { created.code should equal(201) } + val domainApiId = (created.body \ "domain_api_id").extract[String] + (created.body \ "bank_id").extract[String] should equal(SYS) + (created.body \ "base_path").extract[String] should equal(basePath) + (created.body \ "url").extract[String] should endWith(s"/$basePath") + + Then("it is listed and can be read") + val listed = makeGetRequest(domainApis(SYS).GET <@ (user1)) + listed.code should equal(200) + valuesOf(listed.body \ "domain_apis", "domain_api_id") should contain(domainApiId) + makeGetRequest((domainApis(SYS) / domainApiId).GET <@ (user1)).code should equal(200) + + And("a base path that overlaps it is refused, in any space") + grantAllAt(testBankId1.value) + val overlapping = makePostRequest(domainApis(testBankId1.value).POST <@ (user1), registration(basePath, "1.0.0")) + overlapping.code should equal(409) + message(overlapping) should include(DomainApiBasePathAlreadyExists) + + When("its version moves to 1.1.0") + val updated = makePutRequest((domainApis(SYS) / domainApiId).PUT <@ (user1), registration(basePath, "1.1.0")) + withClue(updated.body) { updated.code should equal(200) } + (updated.body \ "version").extract[String] should equal("1.1.0") + + When("it is deleted") + makeDeleteRequest((domainApis(SYS) / domainApiId).DELETE <@ (user1)).code should equal(204) + Then("it is gone") + val gone = makeGetRequest((domainApis(SYS) / domainApiId).GET <@ (user1)) + gone.code should equal(404) + message(gone) should include(DomainApiNotFound) + } + + scenario("base path, version and title are checked", ApiEndpoint1, VersionOfApi) { + grantAllAt(SYS) + def attempt(basePath: String, version: String, title: String = "Test Domain API") = + makePostRequest(domainApis(SYS).POST <@ (user1), registration(basePath, version, title)) + + Then("a base path without a version, in capitals, starting with obp, or too long is refused") + List(s"noversion-$suffix", s"Upper-$suffix/v1", "obp/v1", s"a-$suffix/b/c/d/e/v1", s"/lead-$suffix/v1").foreach { basePath => + val response = attempt(basePath, "1.0.0") + withClue(basePath) { response.code should equal(400) } + message(response) should include(InvalidDomainApiBasePath) + } + And("a version that is not MAJOR.MINOR.PATCH, or whose MAJOR is not the base path's, is refused") + List("1.0", "v1.0.0", "2.0.0").foreach { version => + val response = attempt(s"versioned-$suffix/v1", version) + withClue(version) { response.code should equal(400) } + message(response) should include(InvalidDomainApiVersion) + } + And("an empty title is refused") + message(attempt(s"titled-$suffix/v1", "1.0.0", "")) should include(InvalidDomainApiTitle) + } + } + + feature("The Domain API front door") { + + scenario("a Dynamic Entity and a Dynamic Query answer under the base path as at their OBP URLs, without bank_id") { + entityWithRecord(None, entity, "tree planting") + queryDoc(None, s"/$queryPath/names", entity, s"domainApiSummary$suffix") + grantAllAt(SYS) + val basePath = s"registry-$suffix/v1" + val created = makePostRequest(domainApis(SYS).POST <@ (user1), registration(basePath, "1.0.0")) + withClue(created.body) { created.code should equal(201) } + + When("the entity's records are read under the base path and at the OBP URL") + val published = makeGetRequest((under(basePath) / entity).GET <@ (user1)) + val obp = makeGetRequest((v7 / "banks" / SYS / "dynamic-entities" / entity).GET <@ (user1)) + Then("both answer the same records, and only the OBP URL names the space") + withClue(published.body) { published.code should equal(200) } + obp.code should equal(200) + (published.body \ "bank_id") should equal(JNothing) + (obp.body \ "bank_id").extract[String] should equal(SYS) + valuesOf(published.body, "name") should contain("tree planting") + (published.body \ s"${entity}_list") should equal(obp.body \ s"${entity}_list") + + And("a caller who may not read the entity is refused as at the OBP URL") + val anonymous = makeGetRequest((under(basePath) / entity).GET) + anonymous.code should equal(makeGetRequest((v7 / "banks" / SYS / "dynamic-entities" / entity).GET).code) + + When("the Dynamic Query is called under the base path") + val query = makeGetRequest((under(basePath) / queryPath / "names").GET <@ (user1)) + Then("it answers") + withClue(query.body) { query.code should equal(200) } + valuesOf(query.body \ "names", "name") should contain("tree planting") + + And("a path the space does not serve is a 404") + makeGetRequest((under(basePath) / s"nothing_$suffix").GET <@ (user1)).code should equal(404) + + When("the OpenAPI document is read") + val document = makeGetRequest((under(basePath) / "openapi.json").GET) + withClue(document.body) { document.code should equal(200) } + Then("it carries the Domain API's title, version and server, and its paths") + (document.body \ "info" \ "title").extract[String] should equal("Test Domain API") + (document.body \ "info" \ "version").extract[String] should equal("1.0.0") + (document.body \ "servers")(0) \ "url" match { + case JString(url) => url should endWith(s"/$basePath") + case other => fail(s"no server url: $other") + } + val paths = (document.body \ "paths") match { case JObject(fields) => fields.map(_._1); case _ => Nil } + paths should contain(s"/$entity") + paths should contain(s"/$queryPath/names") + paths.exists(_.contains("dynamic-entities")) shouldBe false + paths.exists(_.contains("banks")) shouldBe false + makeGetRequest((under(basePath) / "openapi.yaml").GET).code should equal(200) + + Then("the documentation is true to the endpoints: each documented path without a placeholder is served, and the documented example has the response's fields") + val route = DomainApis.domainApiProvider.vend.routes().find(_.basePath == basePath) + .getOrElse(fail("the previous scenario registers this Domain API")) + val docs = Http4sDomainApi.publishedDocs(route).filter(_.requestVerb.equalsIgnoreCase("GET")) + withClue(Http4sDomainApi.spaceDocs(route.bankId).map(d => s"${d.requestVerb} ${d.requestUrl} ${d.createdByBankId}").mkString("\n")) { + docs should not be empty + } + docs.filterNot(_.requestUrl.split("/").exists(_.matches("[A-Z][A-Z0-9_]*"))).foreach { doc => + val response = makeGetRequest(doc.requestUrl.split("/").filter(_.nonEmpty).foldLeft(under(basePath))(_ / _).GET <@ (user1)) + withClue(s"GET ${doc.requestUrl} (${doc.partialFunctionName}): ${response.body}") { + response.code should not equal (404) + if (response.code == 200 && doc.partialFunctionName.toLowerCase.contains("dynamicentity")) { + val documented = doc.successResponseBody match { case JObject(fields) => fields.map(_._1).toSet; case _ => Set.empty[String] } + val answered = response.body match { case JObject(fields) => fields.map(_._1).toSet; case _ => Set.empty[String] } + answered should equal(documented) + } + } + } + } + + scenario("a Domain API is refused while two endpoints of its space would answer the same path") { + val bankId = testBankId2.value + val clashing = s"clash_$suffix" + entityWithRecord(Some(bankId), clashing, "clash") + Given("a Dynamic Query at the path the entity's list is published at") + queryDoc(Some(bankId), s"/$clashing", clashing, s"domainApiClash$suffix") + grantAllAt(bankId) + When("a Domain API is registered over that bank") + val refused = makePostRequest(domainApis(bankId).POST <@ (user1), registration(s"clash-$suffix/v1", "1.0.0")) + Then("it is refused, naming the clash") + refused.code should equal(409) + message(refused) should include(DomainApiPathClash) + message(refused) should include(s"/$clashing") + } + } +} diff --git a/release_notes.md b/release_notes.md index 2cfaf2fa5e..c1a28aedef 100644 --- a/release_notes.md +++ b/release_notes.md @@ -3,6 +3,23 @@ ### Most recent changes at top of file ``` Date Commit Action +03/10/2026 TBD CHANGED: a Dynamic Resource Doc is served under its space, at + /obp/dynamic-endpoint/banks/BANK_ID/dynamic-resource-doc/REQUEST_URL, with + SYS for the system space. A verb and URL are unique within a space instead of + across every space, so the system space and several banks may each have a doc + at the same URL. The URL without a space + (/obp/dynamic-endpoint/dynamic-resource-doc/REQUEST_URL) still works while + only one space has a doc there; when more than one does it returns 409 with + the new error OBP-40067, naming the spaces to call it under. API Metrics + record the URL with its space, also for a call made without it. + CHANGED: a system level Dynamic Resource Doc stores SYS as its bank id + instead of NULL, as a Dynamic Entity does. v4.0.0 responses for a system + level doc no longer include "bank_id": null. + CHANGED index on DynamicResourceDoc: the unique index on + (RequestUrl, RequestVerb) is replaced by one on + (BankId, RequestUrl, RequestVerb). On every start, before the tables are + checked, existing NULL bank ids are set to SYS and the old index is dropped; + this does not depend on the migration_scripts props. 30/09/2026 TBD CHANGED in v7.0.0: a Dynamic Entity record at /obp/v7.0.0/banks/BANK_ID/dynamic-entities/... is followed by a metadata object: created and updated, each with at (UTC), user_id (the User who made