fix(page): let on(:request) handlers abort requests behind an authenticated proxy

CHANGELOG.md

@@ -3,6 +3,9 @@
 ### Added
 
 ### Changed
+- Behind a proxy with `:user` and `:password`, Ferrum stops continuing intercepted requests once you subscribe to
+  `on(:request)`, like it already does without a proxy: your callbacks have to continue or abort every request, or it
+  stays paused until the page times out [#332]
 - Browser details (user agent, versions) come from `Browser.getVersion` over CDP for every browser, local or remote.
   `/json/version` is only asked for the websocket url when `url:` is given, so `ws_url:` works against services
   without an HTTP endpoint and a local launch makes no HTTP call
@@ -15,6 +18,8 @@
 - A `Ferrum::Node` found before a navigation or reload kept its cached CDP node id, which Chrome reuses for the new
   document, so `focus`, `click`, `scroll_into_view` and friends silently hit another element. Node ids are now dropped
   on `DOM.documentUpdated` and such nodes raise `Ferrum::NodeNotFoundError` [#86]
+- Behind a proxy with credentials, `request.abort` in `on(:request)` and `network.blacklist`/`whitelist` were ignored
+  because Ferrum's own proxy handler continued every request first. It now only continues requests nobody else handles [#332]
 
 ### Removed
 - `Browser::Process#default_user_agent`, `#browser_version`, `#protocol_version`, `#v8_version` and `#webkit_version`,

docs/7-network.md

@@ -133,6 +133,9 @@ Set request interception for given options. This method is only sets request
 interception, you should use `on` callback to catch requests and abort or
 continue them.
 
+Every paused request has to be continued, aborted or responded to by one of your `on(:request)` callbacks, including
+ones that only log or inspect it. A request nobody handles stays paused, and the page waits for it until it times out.
+
 * options `Hash`
     * :pattern `String` \* by default
     * :resource_type `Symbol` one of the [resource types](https://chromedevtools.github.io/devtools-protocol/tot/Network#type-ResourceType)

docs/9-proxy.md

@@ -34,6 +34,25 @@ browser.create_page(proxy: { host: "y.y.y.y", port: 31337, user: "user", passwor
 end
 ```
 
+## Intercepting requests behind a proxy
+
+A proxy with `:user` and `:password` is authorized through request interception, so Ferrum continues every request
+for you as long as nothing else subscribes to `on(:request)`. Once you subscribe, or set `network.blacklist` or
+`network.whitelist`, your code takes over and has to continue or abort every request:
+
+```ruby
+browser = Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", user: "user", password: "pa$$" })
+browser.network.intercept
+browser.on(:request) do |request|
+  if request.resource_type == "Image"
+    request.abort
+  else
+    request.continue
+  end
+end
+browser.go_to("https://github.com/")
+```
+
 ## Built-in proxy server
 
 Ferrum also ships a small forwarding proxy server, `Ferrum::Proxy`, which is handy

lib/ferrum/page.rb

@@ -569,8 +569,8 @@ def prepare_page
       if use_authorized_proxy?
         network.authorize(user: @proxy_user,
                           password: @proxy_password,
-                          type: :proxy) do |request, _index, _total|
-          request.continue
+                          type: :proxy) do |request, _index, total|
+          request.continue if total == 1
         end
       end
 

lib/ferrum/proxy.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require "logger"
 require "tempfile"
 
 begin

spec/browser_spec.rb

@@ -342,6 +342,35 @@ def proxy_tunnel(socket, request, browser_url)
         ensure
           browser&.quit
         end
+
+        it "lets requests be aborted", local_browser: true do
+          browser = Ferrum::Browser.new(
+            proxy: { host: proxy.host, port: proxy.port, **options }
+          )
+          browser.network.intercept
+          browser.on(:request) { |request| request.match?(/example\.com/) ? request.abort : request.continue }
+
+          expect { browser.go_to("https://example.com") }.to raise_error(
+            Ferrum::StatusError,
+            "Request to https://example.com failed (net::ERR_BLOCKED_BY_CLIENT)"
+          )
+        ensure
+          browser&.quit
+        end
+
+        it "supports blacklist", local_browser: true do
+          browser = Ferrum::Browser.new(
+            proxy: { host: proxy.host, port: proxy.port, **options }
+          )
+          browser.network.blacklist = [/example\.com/]
+
+          expect { browser.go_to("https://example.com") }.to raise_error(
+            Ferrum::StatusError,
+            "Request to https://example.com failed (net::ERR_BLOCKED_BY_CLIENT)"
+          )
+        ensure
+          browser&.quit
+        end
       end
 
       context "with rotation", skip: "Think how to make it working on CI" do