502 Bad Gateway Error When Using Whisper.php in Laravel with Herd #7

Activity

1. CodeWithKyrian commented on Dec 17, 2024

   

   CodeWithKyrian

   on Dec 17, 2024

   Owner

   More actions

   From the logs, we can confirm that the model loaded successfully, so we can rule that out as the cause of the issue.

   My best guess is that the problem lies with the readAudio function, and the application is running out of memory. Let me explain:

   • The readAudio function samples the audio data at 16kHz and, in this initial version of the package, stores the sampled data in a PHP array.
   • For longer audio files, there are far more samples to hold in memory. Because PHP imposes a default memory limit (typically 128MB), processing large audio files can easily exceed this limit, leading to errors.

   To check if this is the issue, set the memory_limit to unlimited in your script: ini_set('memory_limit', '-1'); then run your script again and see if it works.

   Once confirmed, you can adjust the memory_limit to a reasonable value that works for your use case, for example: ini_set('memory_limit', '1G');

   In the future, I plan to store the sampled data in a C array or tensor instead of a PHP array to reduce the memory usage and improve performance.

   Let me know if this solves the issue or if you need further assistance.
2. mrfelipemartins commented on Dec 17, 2024

   

   mrfelipemartins

   on Dec 17, 2024

   Author

   More actions

   I already tried increasing the PHP memory_limit to -1 as suggested, but unfortunately, it made no difference. The audio file I’m testing is very small (approximately 5 seconds), so I am not sure memory exhaustion is the cause here.

   Additionally, the request fails very quickly, within 300ms, which seems too short for a memory-related issue to occur. This makes me suspect that the process might be exiting prematurely, possibly due to an unexpected crash or another error that isn’t being logged.
3. CodeWithKyrian commented on Dec 17, 2024

   

   CodeWithKyrian

   on Dec 17, 2024

   Owner

   More actions

   Oh. I see. So does the readAudio function complete successfully? Maybe dump out the count? I'm trying to figure out if the issue is from it, or perhaps from the transcribe method itself (though it should've shown in the logs if so)
4. mrfelipemartins commented on Dec 17, 2024

   

   mrfelipemartins

   on Dec 17, 2024

   Author

   More actions

   I can confirm the issue lies within the transcribe method. I dumped the output of the readAudio function, it completes successfully and I can see the generated array without any problems.
5. CodeWithKyrian commented on Dec 17, 2024

   

   CodeWithKyrian

   on Dec 17, 2024

   Owner

   More actions

   Okay. Just took a second look at the logs and realized that the transcribe method did run, just up to a point before it failed. Hold on
6. mrfelipemartins commented on Dec 17, 2024

   

   mrfelipemartins

   on Dec 17, 2024

   Author

   More actions

   Alright, I’ll dig into this further, but I suspect the issue is related to Laravel Herd. I tested the same endpoint using php artisan serve, and it worked perfectly as expected. This suggests that something in Herd’s configuration might be prematurely terminating the process or aborting the request.

   I’ll continue debugging and investigate Herd’s behavior in more detail. I’ll report back with any findings or results.
7. CodeWithKyrian commented on Dec 17, 2024

   

   CodeWithKyrian

   on Dec 17, 2024

   Owner

   More actions

   I have a very similar setup—Apple Silicon M1, PHP 8.3 (via Herd), and Laravel 11—and I just tested it using artisan serve, and it worked perfectly. Thanks for mentioning the Herd stuff, cos I was already wondering what else. I'll continue testing on my end as well
8. CodeWithKyrian commented on Dec 22, 2024

   

   CodeWithKyrian

   on Dec 22, 2024

   Owner

   More actions

   Hey. I wanted to drop an update for you. So I’ve tracked down the issue, though I’m still working on a proper solution. It turns out the problem is related to Metal on Apple Silicon Macs. Even though Metal isn’t actively used unless you pass the useGpu option when creating a context, its libraries are still loaded.

   When running through php artisan serve, the dependencies for Metal load correctly. However, when using Herd, something in the nginx and php-fpm configurations prevents certain dependencies from loading properly, causing the issue.

   One workaround that I’ve tested is recompiling whisper.cpp and disabling Metal, which does solve the problem. But I’d prefer to dig deeper and fully resolve this without sacrificing GPU support on Mac.

   I’ll keep you posted as I figure out more!
9. mrfelipemartins commented on Dec 22, 2024

   

   mrfelipemartins

   on Dec 22, 2024

   Author

   More actions

   Thanks for the update! That explanation makes a lot of sense, and I really appreciate you looking into this.

   As a temporary workaround, I’ve been using the following snippet to get things working in Herd with Laravel Concurrency. While I know it’s not the ideal solution, it does the job for now. I opted to use concurrency with a single callback instead of defer, as I need to wait for the result:

   [$result] = Concurrency::run([
       function() use ($responseFormat, $file) {
           Whisper::setLogger(Log::getLogger());
   
           $whisper = Whisper::fromPretrained('large-v3', baseDir: storage_path('app/models'));
   
           $audio = readAudio(storage_path('app/private/' . $file));
   
           $segmentsJson = [];
   
           return $whisper->transcribe($audio, 4);
       }
   ]);

   This approach is far from perfect, but it’s working for me at the moment.

   Thanks again for your efforts!
10. SanderMuller commented on May 1, 2025

    

    SanderMuller

    on May 1, 2025

    More actions

    It works for me on Laravel Herd (Apple Silicon M3) without the Concurrency workaround. So potentially this issue has been solved on the side of Herd