Repository navigation
base64.b85encode uses significant amount of RAM #101178
Description
Bug report
On the same string:
- b64encode: RAM: 604MB, CPU: 79%, Time taken: 1.18sec and gave a result
- b85encode: RAM: 6.9GB, CPU: 64%, Time taken: 45sec and crashed due to insufficient RAM (most likely)
b85encode takes up my entire RAM and crashes
On IPython:
In [1]: import base64
In [2]: contents = b'a' * 250 * 1024 * 1024
In [3]: %time len(base64.b64encode(contents).decode("ascii")) / 1024 / 1024
CPU times: user 489 ms, sys: 414 ms, total: 904 ms
Wall time: 974 ms
Out[3]: 333.33333587646484
In [4]: %time len(base64.b85encode(contents).decode("ascii")) / 1024 / 1024
Killed
Here is the GNU time stats:
$ /usr/bin/time /opt/python3.9/bin/python -c "import base64; print(len(base64.b64encode(b'a' * 250 * 1024 * 1024)) / 1024 / 1024)"
333.33333587646484
0.55user 0.39system 0:01.18elapsed 79%CPU (0avgtext+0avgdata 604008maxresident)k
0inputs+0outputs (0major+151155minor)pagefaults 0swaps
$ /usr/bin/time /opt/python3.9/bin/python -c "import base64; print(len(base64.b85encode(b'a' * 250 * 1024 * 1024)) / 1024 / 1024)"
26.08user 3.39system 0:45.86elapsed 64%CPU (0avgtext+0avgdata 6966080maxresident)k
512inputs+0outputs (2major+1756895minor)pagefaults 0swaps
I have gotten same results in Python 3.6, 3.7, 3.8, 3.9
Your environment
- CPython versions tested on: Python 3.9 installed with miniconda
- Operating system and architecture: CentOS 7, AWS t3.large machine
Activity
eendebakpt commentedon Apr 24, 2025
Thanks for the patience! I pinged the cpython triage team to have a look.
Merge branch 'main' into pythongh-101178-rework-base85
Merge branch 'main' into pythongh-101178-rework-base85
Merge branch 'main' into pythongh-101178-rework-base85
Merge branch 'main' into pythongh-101178-rework-base85
Merge branch 'main' into pythongh-101178-rework-base85
gh-101178: Add Ascii85, Base85, and Z85 support to binascii (GH-102753)
pythongh-101178: Fix possible integer overflow in Ascii85 encoder wit…
pythongh-101178: Add Ascii85, Base85, and Z85 support to binascii (py…
gh-101178: Fix possible integer overflow in Ascii85 encoder with wrap…
pythongh-101178: Fix possible integer overflow in Ascii85 encoder wit…
gh-101178: Fix UB in `binascii.b2a_ascii85` (#148166)
pythongh-101178: Add Ascii85, Base85, and Z85 support to binascii (py…
pythongh-101178: Fix possible integer overflow in Ascii85 encoder wit…
pythongh-101178: Fix UB in `binascii.b2a_ascii85` (python#148166)