Merging in Hongwei's PR #2905 dynamic resource doc java support +

OPEN_CORRIDOR_INTERFACE_C_PUBLISH_PLAN.md

@@ -356,10 +356,11 @@ parts:
 > (`OBP-OUTGOING-SETTLEMENT-ACCOUNT` → `OBP-INCOMING-SETTLEMENT-ACCOUNT`),
 > writes `settled_by_transaction_ids` AND `settled_by_transaction_request_id`
 > (TR B's id — kept even at net zero) attributes on each covered promise,
-> flips them COMPLETED, and enqueues the messages into the `OpenCorridorOutbox`
-> Mapper table in the same request DB transaction. `OpenCorridorOutboxRelay`
-> (Boot-started when `open_corridor_enabled`, `open_corridor.outbox_relay_interval`
-> default 10s) publishes with exponential backoff and records each §4.2 reply:
+> flips them COMPLETED, and enqueues the messages into the `MessageOutbox`
+> Mapper table in the same request DB transaction. `MessageOutboxRelay`
+> (Boot-started on every instance, relaying Open Corridor rows only when
+> `open_corridor_enabled`; interval `message_outbox.relay_interval_seconds`,
+> default 10s, formerly `open_corridor.outbox_relay_interval`) publishes with exponential backoff and records each §4.2 reply:
 > settlement rows stay PENDING through SUBMITTED/SETTLING (redelivery-as-polling,
 > §4.4) until FINAL; refutable business errors (COMMITMENT-MISMATCH etc.) go
 > STICKY for operator reconciliation, never swallowed. Fails fast pre-mutation

docs/MAKER_CHECKER_DYNAMIC_CODE_DESIGN.md

@@ -125,8 +125,11 @@ request. Rows are never deleted; the table is the audit log.
 - `ApprovedHash` on DynamicResourceDoc, DynamicMessageDoc, ConnectorMethod, AbacRule.
 - `IsActive` (default `true`) on every runtime-loaded dynamic model that lacks it.
 
-The runtime loads a row only if `IsActive` is true and, for code, only if `MethodBodyHash ==
-ApprovedHash`. When maker/checker is disabled for a target type the hash check is skipped.
+The runtime loads a row only if `IsActive` is true and, for code, only if the row's code hash
+equals `ApprovedHash`. The code hash is recomputed from the row (its programming language and its
+decoded method body, `APIUtil.dynamicCodeHash`) rather than read from the stored `MethodBodyHash`, so
+an edit made directly in the database, to the body or to the language, withdraws the approval. When
+maker/checker is disabled for a target type the hash check is skipped.
 
 ## 4. Endpoints (v7.0.0)
 
@@ -193,8 +196,14 @@ fail loudly. On approval the server, in order:
 Rejection and withdrawal take `{ "comment": "…" }`. Withdrawal is by the requestor only.
 *(impl)* Hashes are bare SHA-256 hex, matching the existing `MethodBodyHash` column; a `sha256:`
 prefix is accepted on approval. `payload_hash` is over the canonical JSON of the request body;
-`current_payload_hash` and `ApprovedHash` are the target's decoded method body hash (for ABAC
-rules, the hash of `rule_code`).
+`current_payload_hash` and `ApprovedHash` are the target's code hash: SHA-256 of its programming
+language (trimmed, lower case, blank meaning `scala`), a newline, and its decoded method body (for
+ABAC rules, the hash of `rule_code`). The language is included because the same text can be stored
+as Scala, Java or Javascript and is compiled by the language's compiler, so approving a body as one
+language must not approve it as another. The language version is not included: it belongs to the
+deployed runtime, and including it would withdraw every approval at each compiler upgrade. Until
+2026-10 the hash covered the body only; `MakerChecker.rehashDynamicCodeWithLanguage` moves existing
+rows once at boot, carrying over only the approvals still valid for the row's current body.
 
 ### Reading
 

obp-api/src/main/resources/props/sample.props.template

@@ -1857,7 +1857,6 @@ dynamic_code_allowed_obp_methods=[\
       ErrorMessages.getClass.getTypeName -> "*",\
       ExecutionContext.Implicits.getClass.getTypeName -> "global",\
       JSONFactory400.getClass.getTypeName -> "createBanksJson",\
-      classOf[Sandbox].getTypeName -> "runInSandbox",\
       classOf[CallContext].getTypeName -> "*",\
       classOf[ResourceDoc].getTypeName -> "getPathParams",\
       "scala.reflect.runtime.package\$" -> "universe",\

obp-api/src/main/scala/bootstrap/liftweb/Boot.scala

@@ -315,6 +315,10 @@ class Boot extends MdcLoggable {
     // Please note that migration scripts are executed after Lift Mapper Schemifier
     Migration.database.executeScripts(startedBeforeSchemifier = false)
 
+    // Code hashes cover the programming language as well as the body; move existing rows (and the
+    // approvals still valid for them) to that form once. Must run before the seed below.
+    code.dynamicchangerequest.MakerChecker.rehashDynamicCodeWithLanguage()
+
     // Maker/checker for dynamic code: when first enabled, pre-existing code rows get their current
     // body hash recorded as approved so enabling the feature does not silently disable them.
     code.dynamicchangerequest.MakerChecker.seedApprovedHashesIfEnabled()

obp-api/src/main/scala/code/api/constant/constant.scala

@@ -408,7 +408,12 @@
   final val CREATE_LOCALISED_RESOURCE_DOC_JSON_TTL: Int = APIUtil.getPropsValue(s"createLocalisedResourceDocJson.cache.ttl.seconds", "3600").toInt
   final val GET_DYNAMIC_RESOURCE_DOCS_TTL: Int = APIUtil.getPropsValue(s"dynamicResourceDocsObp.cache.ttl.seconds", "3600").toInt
   final val GET_STATIC_RESOURCE_DOCS_TTL: Int = APIUtil.getPropsValue(s"staticResourceDocsObp.cache.ttl.seconds", "3600").toInt
-  final val SHOW_USED_CONNECTOR_METHODS: Boolean = APIUtil.getPropsAsBoolValue(s"show_used_connector_methods", false)
+  // def, not final val: DynamicUtil.Validation.validateDependency (dynamic-code dependency
+  // checking) needs this to react to a props change without a restart -- e.g. test-time
+  // setPropsValues overrides. A final val here would freeze at whatever value was true the
+  // moment this object was first touched (typically during server boot, well before any test
+  // scenario runs), and no later prop override could ever reach it.
+  def SHOW_USED_CONNECTOR_METHODS: Boolean = APIUtil.getPropsAsBoolValue(s"show_used_connector_methods", false)
 
   // Rate Limiting Cache Prefixes (with global namespace and versioning)
   // Both call_counter and rl_active are versioned for consistent cache invalidation

obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicEndpoints.scala

@@ -33,7 +33,7 @@ import code.api.dynamic.endpoint.helper.practise.{DynamicEndpointCodeGenerator,
 import code.api.dynamic.endpoint.helper.practise.PractiseEndpointGroup
 import code.api.util.DynamicUtil.{DynamicCodeBody, Validation}
 import code.api.util.APIUtil.{BooleanBody, DoubleBody, EmptyBody, LongBody, Http4sEndpointIO, PrimaryDataBody, ResourceDoc, StringBody, getDisabledEndpointOperationIds}
-import code.api.util.{CallContext, DynamicUtil}
+import code.api.util.{APIUtil, CallContext, DynamicUtil}
 import net.liftweb.common.{Box, Failure, Full}
 import org.json4s.{JNothing, JValue}
 import org.json4s.JsonAST.{JBool, JDouble, JInt, JString}
@@ -105,6 +105,17 @@ trait EndpointGroup {
  * @param methodBody it is url-encoded string for the api level code.
  */
 object CompiledObjects {
+  /**
+   * This is the set of `programming_lang` values a Dynamic Resource Doc may have, compared after
+   * APIUtil.normaliseDynamicCodeLanguage (trimmed, lower case, blank meaning Scala). Create, update,
+   * validate and the dry-run compile all check against it, and CompiledObjects chooses its compiler
+   * from the same normalised value, so nothing that passes the check can reach the wrong compiler.
+   */
+  val supportedLanguages: List[String] = List("scala", "java")
+  def isSupportedLanguage(programmingLang: String): Boolean =
+    supportedLanguages.contains(APIUtil.normaliseDynamicCodeLanguage(programmingLang))
+  val supportedLanguagesText: String = "Scala, Java"
+
   /**
    * The native http4s template a method body is inlined into. Returns the full source and the
    * 1-based line on which the method body starts, so compiler positions can be mapped back to it.
@@ -153,10 +164,18 @@ object CompiledObjects {
 
   /**
    * Dry run without constructing a CompiledObjects (whose constructor compiles for real): compiler
-   * diagnostics with line numbers relative to the method body the author wrote. Empty = compiles.
+   * diagnostics with line numbers relative to the method body the author wrote, in the body's
+   * programming language (Scala or Java). Empty = compiles.
    */
-  def compileProblems(exampleRequestBody: Option[JValue], successResponseBody: Option[JValue], methodBody: String): List[DynamicUtil.CompileProblem] = {
+  def compileProblems(exampleRequestBody: Option[JValue], successResponseBody: Option[JValue], methodBody: String, programmingLang: String = "Scala"): List[DynamicUtil.CompileProblem] = {
     val decodedMethodBody = URLDecoder.decode(methodBody, "UTF-8")
+    // Java bodies are compiled as written (no template, no generated case classes), so the example
+    // bodies play no part, as they play none in CompiledObjects' own Java branch.
+    if (APIUtil.normaliseDynamicCodeLanguage(programmingLang) == "java") DynamicUtil.checkJavaCode(decodedMethodBody)
+    else scalaCompileProblems(exampleRequestBody, successResponseBody, decodedMethodBody)
+  }
+
+  private def scalaCompileProblems(exampleRequestBody: Option[JValue], successResponseBody: Option[JValue], decodedMethodBody: String): List[DynamicUtil.CompileProblem] = {
     val requestBody: Product = exampleRequestBody match {
       case Some(JString(s)) if StringUtils.isBlank(s) => toCaseObject(None)
       case _ => toCaseObject(exampleRequestBody)
@@ -186,7 +205,7 @@ object CompiledObjects {
   }
 }
 
-case class CompiledObjects(exampleRequestBody: Option[JValue], successResponseBody: Option[JValue], methodBody: String) {
+case class CompiledObjects(exampleRequestBody: Option[JValue], successResponseBody: Option[JValue], methodBody: String, programmingLang: String = "Scala") {
   val decodedMethodBody = URLDecoder.decode(methodBody, "UTF-8")
   val requestBody: Product = exampleRequestBody match {
       //this case means, we accept the empty string "" from json post body, we need to map it to None.
@@ -196,7 +215,24 @@ case class CompiledObjects(exampleRequestBody: Option[JValue], successResponseBo
   }
   val successResponse: Product = toCaseObject(successResponseBody)
 
-  private val partialFunction: Http4sEndpointIO = {
+  private val partialFunction: Http4sEndpointIO = APIUtil.normaliseDynamicCodeLanguage(programmingLang) match {
+    case "java" =>
+      DynamicUtil.createJavaHttp4sEndpoint(decodedMethodBody) match {
+        case Full(func) => func
+        case Failure(msg: String, exception: Box[Throwable], _) =>
+          throw exception.getOrElse(new RuntimeException(msg))
+        case _ => throw new RuntimeException("compiled code return nothing")
+      }
+    case _ /* "scala", the default; create and update reject anything else (isSupportedLanguage) */ =>
+      scalaPartialFunction
+  }
+
+  // Unchanged Scala-template compile path, factored out so the `partialFunction` match above stays
+  // readable. Only evaluated for Scala-language docs (the default) — Java-language docs never
+  // touch this, so example/response-body JValues that don't fit the Scala case-class generator
+  // (irrelevant for Java, since it doesn't use RequestRootJsonClass/ResponseRootJsonClass) are a
+  // non-issue there.
+  private def scalaPartialFunction: Http4sEndpointIO = {
 
     //If the requestBody is PrimaryDataBody, return None. otherwise, return the exampleRequestBody:Option[JValue]
     // In side OBP resourceDoc, requestBody and successResponse must be Product type，
@@ -237,15 +273,28 @@ case class CompiledObjects(exampleRequestBody: Option[JValue], successResponseBo
    * this will check all the dynamic scala code dependencies at compile time.
    *
    *Search for the usage, you can see how to use it in OBP code.
+   *
+   * Scala-only: for the Scala language, `this.partialFunction` IS the compiled user code, so
+   * validating its bytecode directly is correct. For Java, `this.partialFunction` is instead
+   * OBP's own Http4sEndpointIO wrapper (built by DynamicUtil.createJavaHttp4sEndpoint) around the
+   * real compiled Java class -- its bytecode legitimately calls internal OBP helpers
+   * (DynamicUtil.javaValueToJValue/logger, CustomJsonFormats.formats, JsonAliases.compactRender)
+   * that were never meant to be dependency-whitelisted, since they are framework glue, not
+   * user-supplied code. createJavaHttp4sEndpoint already validates the real compiled Java class
+   * internally (see its own doc comment) before ever returning that wrapper, so re-validating the
+   * wrapper here is both redundant and wrong -- it would reject every Java doc unconditionally.
    */
-  def validateDependency() = Validation.validateDependency(this.partialFunction)
+  def validateDependency() = APIUtil.normaliseDynamicCodeLanguage(programmingLang) match {
+    case "java" => ()
+    case _ => Validation.validateDependency(this.partialFunction)
+  }
 
   /**
    * Dry run: compiler diagnostics for this body, with line numbers relative to the method body the
    * author wrote (the wrapper's own lines are subtracted). Empty = compiles. Nothing is evaluated or cached.
    */
   def compileProblems(): List[DynamicUtil.CompileProblem] =
-    CompiledObjects.compileProblems(exampleRequestBody, successResponseBody, methodBody)
+    CompiledObjects.compileProblems(exampleRequestBody, successResponseBody, methodBody, programmingLang)
 
   /**
    * Wraps the compiled partial function as an endpoint. This used to bind a per-bank

obp-api/src/main/scala/code/api/dynamic/endpoint/helper/DynamicResourceDocsEndpointGroup.scala

@@ -52,6 +52,22 @@ object DynamicResourceDocsEndpointGroup extends EndpointGroup with code.util.Hel
       try {
         Some(toResourceDoc(dynamicDoc))
       } catch {
+        // Validation.validateDependency / createJavaHttp4sEndpoint's own rejection path both throw
+        // this specifically for a dependency-whitelist miss -- distinct from a genuine compile
+        // failure, and reachable here (not just at create/update time) because CompiledObjects'
+        // validation runs fresh on every construction and dynamic_code_allowed_obp_methods
+        // can be tightened after a doc was already registered. Logging it as a "deprecated Lift
+        // contract" problem sends whoever reads this log to re-author a body that is not the
+        // problem, instead of at the whitelist they (or someone else) just edited.
+        case e: code.api.JsonResponseException =>
+          val reason = e.jsonResponse match {
+            case APIUtil.JsonResponseExtractor(msg, _) => msg
+            case _ => Option(e.getMessage).getOrElse("")
+          }
+          logger.error(s"[DynamicResourceDocsEndpointGroup] skipping dynamic resource doc '${dynamicDoc.requestVerb} ${dynamicDoc.requestUrl}' " +
+            s"(id=${dynamicDoc.dynamicResourceDocId.getOrElse("")}, programming_lang=${dynamicDoc.programmingLang}): rejected by dependency " +
+            s"validation (dynamic_code_allowed_obp_methods). $reason")
+          None
         case e: Throwable =>
           logger.error(s"[DynamicResourceDocsEndpointGroup] skipping dynamic resource doc '${dynamicDoc.requestVerb} ${dynamicDoc.requestUrl}' " +
             s"(id=${dynamicDoc.dynamicResourceDocId.getOrElse("")}): its methodBody could not be compiled under the native http4s contract. " +
@@ -80,7 +96,7 @@ object DynamicResourceDocsEndpointGroup extends EndpointGroup with code.util.Hel
    * 
    */
   private val toResourceDoc: JsonDynamicResourceDoc => ResourceDoc = { dynamicDoc =>
-    val compiledObjects = CompiledObjects(dynamicDoc.exampleRequestBody, dynamicDoc.successResponseBody, dynamicDoc.methodBody)
+    val compiledObjects = CompiledObjects(dynamicDoc.exampleRequestBody, dynamicDoc.successResponseBody, dynamicDoc.methodBody, dynamicDoc.programmingLang)
     ResourceDoc(
       // partialFunction is a no-op stub — the runtime dispatch uses the native handler in
       // dynamicHttp4sFunction (the compiled artifact is OBPEndpointIO, not the Lift OBPEndpoint).

obp-api/src/main/scala/code/api/util/APIUtil.scala

@@ -3410,6 +3410,29 @@ object APIUtil extends MdcLoggable with CustomJsonFormats{
     digest.map(b => f"$b%02x").mkString
   }
 
+  /**
+   * This is the hash stored as `MethodBodyHash` on runtime-compiled code (Dynamic Resource Docs,
+   * Dynamic Message Docs, Connector Methods), and therefore the value a maker/checker approval binds
+   * to (`ApprovedHash`).
+   *
+   * It covers the programming language as well as the body. The same text can be stored as Scala,
+   * Java or Javascript, and the runtime picks a compiler from the language, so an approval of a body
+   * as one language must not carry over when only the language is changed. The language is
+   * normalised (trimmed, lower case, blank meaning `scala`, the default every one of these types
+   * applies), because the runtime treats `Java` and `java` as the same language.
+   *
+   * The language version is deliberately not included: it is a property of the deployed runtime, not
+   * of the row, and including it would withdraw every approval at each compiler upgrade.
+   *
+   * Until 2026-10 the hash covered the body only; `MakerChecker.rehashDynamicCodeWithLanguage` moves
+   * existing rows to this form once.
+   */
+  def dynamicCodeHash(programmingLang: String, decodedMethodBody: String): String =
+    sha256Hex(normaliseDynamicCodeLanguage(programmingLang) + "\n" + Option(decodedMethodBody).getOrElse(""))
+
+  def normaliseDynamicCodeLanguage(programmingLang: String): String =
+    Option(programmingLang).map(_.trim.toLowerCase).filter(_.nonEmpty).getOrElse("scala")
+
   /**
    *  Create the explicit CounterpartyId, (Used in `Create counterparty for an account` endpoint ).
    *  This is just a UUID, use both in Counterparty.counterpartyId and CounterpartyMetadata.counterpartyId
@@ -4526,8 +4549,10 @@ object APIUtil extends MdcLoggable with CustomJsonFormats{
    * 
    * than the return value may be (getUserAndSessionContextFuture, ***,***),(map,***,***), (getOrElse,***,***) ......
    */ 
-  def getDependentMethods(className: String, methodName:String, signature: String): List[(String, String, String)] = {
-    if (SHOW_USED_CONNECTOR_METHODS) {
+  // force bypasses the SHOW_USED_CONNECTOR_METHODS gate below -- see
+  // DynamicUtil.getDynamicCodeDependentMethods' doc comment for why security validation needs this.
+  def getDependentMethods(className: String, methodName:String, signature: String, force: Boolean = false): List[(String, String, String)] = {
+    if (SHOW_USED_CONNECTOR_METHODS || force) {
       val methods = ListBuffer[(String, String, String)]()
       //NOTE: MEMORY_USER this ctClass will be cached in ClassPool, it may load too many classes into heap. 
       //eg:  className == code.api.UKOpenBanking.v3_1_0.APIMethods_AccountAccessApi$$anonfun$createAccountAccessConsents$lzycompute$1