Downloading artifacts as files has a memory leak

[alahiff]

In download_content in simvue/api/objects/artifact/base.py is this code:

        get_json_from_response(
            response=_response,
            allow_parse_failure=True,
            expected_status=[http.HTTPStatus.OK],
            scenario=f"Retrieval of file for {self.label()} '{self._identifier}'",
        )

When downloading all the output files associated with a run, the above code causes the Python API to use a huge amount of memory. For reasons I don't understand not all runs are affected, but here is one example which is:

from simvue import Client
client = Client()
client.get_artifacts_as_files('iZgnHUrNkXXJi6pcFVZDQp', category='output')

When the above code involving get_json_from_response is commented out everything is fine. I assume this function is reading the full output into memory (which shouldn't be done when downloading artifacts), but weirdly this memory remains used until the script exits (or in my case the VM dies).

Note that the above shared run only has 181 MB of output files, but use of the above function causes multiple GB of memory to be used.

[alahiff]

In get_json_from_response is response.json(). This puts the entire response in memory, so defeats the purpose of the streaming later on.