Skip to content

feat(logging)!: Use StructuredLogHandler for Cloud Run Jobs by default - #18707

Open
fajpunk wants to merge 1 commit into
googleapis:mainfrom
fajpunk:fix-cloudrun-job-default-logging
Open

fajpunk wants to merge 1 commit into
googleapis:mainfrom
fajpunk:fix-cloudrun-job-default-logging

Conversation

@fajpunk

@fajpunk fajpunk commented Oct 15, 2026 •

Copy link
Copy Markdown

BREAKING CHANGE: Use StructuredLogHandler instead of CloudLoggingHandler by default in Cloud Run Jobs. There's no reason that I can think that we can't just log to stdout in Cloud Run Jobs by default like we do in Cloud Run Services and Cloud Run Functions. Starting a background thread for log shipping adds complexity and can cause errors, especiallly when interactiing with other libs like aiohttp, leading to errors like this:

  File "/usr/local/lib/python3.13/site-packages/aiohttp/connector.py", line 379, in __del__
    self._loop.call_exception_handler(context)
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 1925, in call_exception_handler
    logger.error('Exception in default exception handler',
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1554, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1670, in _log
    self.handle(record)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1686, in handle
    self.callHandlers(record)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1746, in callHandlers
    hdlr.handle(record)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1027, in handle
    self.emit(record)
  File "/usr/local/lib/python3.13/site-packages/google/cloud/logging_v2/handlers/handlers.py", line 218, in emit
    self.transport = self._transport_cls(
  File "/usr/local/lib/python3.13/site-packages/google/cloud/logging_v2/handlers/transports/background_thread.py", line 324, in __init__
    self.worker.start()
  File "/usr/local/lib/python3.13/site-packages/google/cloud/logging_v2/handlers/transports/background_thread.py", line 173, in start
    self._thread.start()
  File "/usr/local/lib/python3.13/threading.py", line 976, in start
    _start_joinable_thread(self._bootstrap, handle=self._handle,
PythonFinalizationError: can't create new thread at interpreter shutdown

Of course just because I can't think of a reason why CloudLoggingHandler should be the default doesn't mean there isn't one, and this is a breaking change that could cause problems for folks depending on the existing behavior.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #15407 🦕

@fajpunk
fajpunk requested a review from a team as a code owner October 15, 2026 21:11

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for Cloud Run jobs (cloud_run_job) to the default logging handler, returning a StructuredLogHandler when detected, and includes a corresponding unit test. The feedback suggests combining the duplicate elif blocks for Cloud Run revisions and jobs into a single membership check, and updating the mocked environment variables in the unit test to use string values instead of integers to accurately mimic os.environ.

Comment on lines 391 to +394
elif monitored_resource.type == _RUN_RESOURCE_TYPE:
return StructuredLogHandler(**kw, project_id=self.project)
elif monitored_resource.type == _JOB_RESOURCE_TYPE:
return StructuredLogHandler(**kw, project_id=self.project)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Instead of having separate elif blocks that return the exact same StructuredLogHandler instance, we can combine them into a single elif block using membership testing (in). This reduces duplication and improves maintainability.

Suggested change
elif monitored_resource.type == _RUN_RESOURCE_TYPE:
return StructuredLogHandler(**kw, project_id=self.project)
elif monitored_resource.type == _JOB_RESOURCE_TYPE:
return StructuredLogHandler(**kw, project_id=self.project)
elif monitored_resource.type in (_RUN_RESOURCE_TYPE, _JOB_RESOURCE_TYPE):
return StructuredLogHandler(**kw, project_id=self.project)

Comment thread packages/google-cloud-logging/tests/unit/test_client.py Outdated
@fajpunk
fajpunk force-pushed the fix-cloudrun-job-default-logging branch 2 times, most recently from 118a445 to e2001f1 Compare October 15, 2026 21:20
fajpunk added a commit to lsst/dax_ppdb that referenced this pull request Oct 15, 2026
By default, when we ask Google Cloud to configure logging for us in
a Cloud Run Job, it gives us a `CloudLoggingHandler`. This starts
a background thread and ships log messages to google via an API.
A `StructuredLogHandler` just prints log messges to stdout. This is all
we really need. The added complexity of managing another thread is
making it harder to debug some issues with asyncio resource management
in `cloud-sql-python-connector`.

Some might argue that this is a bug in the Google Cloud Python SDK, as
they say in their docs that `StructuredLogHandler` is the default
handler for Cloud Run. "Cloud Run" is like 1,000 different things though
so who's to say what that really means, and if it includes Cloud Run
Jobs. In case it does, Here's
a [PR](googleapis/google-cloud-python#18707) to
change the Google Cloud Python SDK behavior.
@fajpunk fajpunk changed the title feat!(logging): Use StructuredLogHandler for Cloud Run Jobs by default feat(logging)!: Use StructuredLogHandler for Cloud Run Jobs by default Oct 15, 2026
@parthea parthea self-assigned this Oct 16, 2026
@parthea parthea added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Oct 16, 2026
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Oct 16, 2026
Use StructuredLogHandler instead of CloudLoggingHandler by default in
Cloud Run Jobs. There's no reason that I can think that we can't just
log to stdout in Cloud Run Jobs by default like we do in Cloud Run
Services and Cloud Run Functions. Starting a background thread for log
shipping adds complexity and can cause errors, especiallly when
interactiing with other libs like `aiohttp`, leading to errors like
this:

```
  File "/usr/local/lib/python3.13/site-packages/aiohttp/connector.py", line 379, in __del__
    self._loop.call_exception_handler(context)
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 1925, in call_exception_handler
    logger.error('Exception in default exception handler',
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1554, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1670, in _log
    self.handle(record)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1686, in handle
    self.callHandlers(record)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1746, in callHandlers
    hdlr.handle(record)
  File "/usr/local/lib/python3.13/logging/__init__.py", line 1027, in handle
    self.emit(record)
  File "/usr/local/lib/python3.13/site-packages/google/cloud/logging_v2/handlers/handlers.py", line 218, in emit
    self.transport = self._transport_cls(
  File "/usr/local/lib/python3.13/site-packages/google/cloud/logging_v2/handlers/transports/background_thread.py", line 324, in __init__
    self.worker.start()
  File "/usr/local/lib/python3.13/site-packages/google/cloud/logging_v2/handlers/transports/background_thread.py", line 173, in start
    self._thread.start()
  File "/usr/local/lib/python3.13/threading.py", line 976, in start
    _start_joinable_thread(self._bootstrap, handle=self._handle,
PythonFinalizationError: can't create new thread at interpreter shutdown
```

Of course just because *I* can't think of a reason why
`CloudLoggingHandler` should be the default doesn't mean there isn't
one, and this is a breaking change that could cause problems for folks
depending on the existing behavior.

Fixes googleapis#15407

BREAKING CHANGE: This changes the default log handler for all Cloud Run
Jobs
@fajpunk
fajpunk force-pushed the fix-cloudrun-job-default-logging branch from e2001f1 to 16dbc1d Compare October 16, 2026 19:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Let Cloud Run Job use StructuredLogHandler instead of ClougLoggingHandler

3 participants