diff --git a/CHANGELOG.md b/CHANGELOG.md index 2bc64b01..8e94696c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,12 @@ ## [Unreleased](https://github.com/rubycdp/ferrum/compare/v0.19.1...main) ## ### Added +- `keyboard.hold(:shift) { }` holds modifiers for every keyboard and mouse event in the block, e.g. a shift+drag, and + releases them afterwards. `mouse.move` accepts `modifiers:` like `down` and `up` [#581] ### Changed +- `keyboard.down` and `keyboard.up` raise `ArgumentError` for a modifier such as `:shift` and point to `hold`. `down` + used to crash with `NoMethodError`, `up` sent an empty keyup event [#581] - Behind a proxy with `:user` and `:password`, Ferrum stops continuing intercepted requests once you subscribe to `on(:request)`, like it already does without a proxy: your callbacks have to continue or abort every request, or it stays paused until the page times out [#332] @@ -20,6 +24,7 @@ on `DOM.documentUpdated` and such nodes raise `Ferrum::NodeNotFoundError` [#86] - Behind a proxy with credentials, `request.abort` in `on(:request)` and `network.blacklist`/`whitelist` were ignored because Ferrum's own proxy handler continued every request first. It now only continues requests nobody else handles [#332] +- `mouse.click(modifiers:)` sent its `mousemove` without the modifiers [#581] ### Removed - `Browser::Process#default_user_agent`, `#browser_version`, `#protocol_version`, `#v8_version` and `#webkit_version`, diff --git a/docs/10-mouse-keyboard.md b/docs/10-mouse-keyboard.md index bf450f53..4ed7b390 100644 --- a/docs/10-mouse-keyboard.md +++ b/docs/10-mouse-keyboard.md @@ -32,7 +32,8 @@ Click given coordinates, fires mouse move, down and up events. * :delay `Float` defaults to 0. Delay between mouse down and mouse up events * :button `Symbol` :left | :right, defaults to :left * :count `Integer` defaults to 1 - * :modifiers `Integer` bitfield for key modifiers. See`keyboard.modifiers` + * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. + If omitted, uses the keys held by `keyboard.hold`, or none outside of it #### down(\*\*options) : `Mouse` @@ -41,7 +42,8 @@ Mouse down for given coordinates. * options `Hash` * :button `Symbol` :left | :right, defaults to :left * :count `Integer` defaults to 1 - * :modifiers `Integer` bitfield for key modifiers. See`keyboard.modifiers` + * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. + If omitted, uses the keys held by `keyboard.hold`, or none outside of it #### up(\*\*options) : `Mouse` @@ -50,9 +52,10 @@ Mouse up for given coordinates. * options `Hash` * :button `Symbol` :left | :right, defaults to :left * :count `Integer` defaults to 1 - * :modifiers `Integer` bitfield for key modifiers. See`keyboard.modifiers` + * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. + If omitted, uses the keys held by `keyboard.hold`, or none outside of it -#### move(x:, y:, steps: 1) : `Mouse` +#### move(x:, y:, steps: 1, modifiers: nil) : `Mouse` Mouse move to given x and y. @@ -60,6 +63,8 @@ Mouse move to given x and y. * :x `Integer` * :y `Integer` * :steps `Integer` defaults to 1. Sends intermediate mousemove events. + * :modifiers `Integer` bitfield for key modifiers. See `keyboard.modifiers`. + If omitted, uses the keys held by `keyboard.hold`, or none outside of it ## Keyboard @@ -69,13 +74,15 @@ Mouse move to given x and y. Dispatches a keydown event. -* key `String` | `Symbol` Name of key such as "a", :enter, :backspace +* key `String` | `Symbol` Name of key such as "a", :enter, :backspace. Raises `ArgumentError` for a modifier, + use `hold` instead #### up(key) : `Keyboard` Dispatches a keyup event. -* key `String` | `Symbol` Name of key such as "b", :enter, :backspace +* key `String` | `Symbol` Name of key such as "b", :enter, :backspace. Raises `ArgumentError` for a modifier, + use `hold` instead #### type(\*keys) : `Keyboard` @@ -84,6 +91,27 @@ Sends a keydown, keypress/input, and keyup event for each character in the text. * text `String` | `Array | Array` A text to type into a focused element, `[:Shift, "s"], "tring"` +#### hold(\*keys) { } : `Object` + +Holds modifier keys down while the block runs. Every keyboard and mouse event inside the block carries them, and they +are released when the block returns or raises. Returns the block's value. + +* keys `Symbol` :alt | :ctrl | :control | :meta | :command | :shift + +```ruby +page.keyboard.hold(:shift) do + page.mouse.move(x: 10, y: 10).down.move(x: 200, y: 200, steps: 10).up +end +``` + +#### held_modifiers : `Integer` + +Modifier keys currently pressed by `hold`, in the same format as `modifiers` returns. It's 0 outside a `hold` block. + +```ruby +page.keyboard.hold(:shift) { page.keyboard.held_modifiers } # => 8 +``` + #### modifiers(keys) : `Integer` Returns bitfield for a given keys @@ -103,7 +131,5 @@ page.keyboard.type("Hello World") page.keyboard.type(:Enter) # Use keyboard shortcuts -page.keyboard.down(:Shift) -page.keyboard.type("h", "e", "l", "l", "o") -page.keyboard.up(:Shift) +page.keyboard.hold(:shift) { page.keyboard.type("hello") } ``` diff --git a/lib/ferrum/keyboard.rb b/lib/ferrum/keyboard.rb index 6370e404..63d386c0 100644 --- a/lib/ferrum/keyboard.rb +++ b/lib/ferrum/keyboard.rb @@ -32,8 +32,15 @@ class Keyboard f10: "F10", f11: "F11", f12: "F12", meta: "Meta", command: "Meta" }.freeze + # Modifier keys currently pressed by {#hold}, in the same format as + # {#modifiers} returns. It's 0 outside a {#hold} block. + # + # @return [Integer] + attr_reader :held_modifiers + def initialize(page) @page = page + @held_modifiers = 0 end # @@ -44,7 +51,10 @@ def initialize(page) # # @return [self] # + # @raise [ArgumentError] If the key is a modifier, use {#hold} instead. + # def down(key) + reject_modifier(key) key = normalize_keys(Array(key)).first type = key[:text] ? "keyDown" : "rawKeyDown" @page.command("Input.dispatchKeyEvent", slowmoable: true, type: type, **key) @@ -59,7 +69,10 @@ def down(key) # # @return [self] # + # @raise [ArgumentError] If the key is a modifier, use {#hold} instead. + # def up(key) + reject_modifier(key) key = normalize_keys(Array(key)).first @page.command("Input.dispatchKeyEvent", slowmoable: true, type: "keyUp", **key) self @@ -86,6 +99,44 @@ def type(*keys) self end + # + # Holds modifier keys down while the block runs: every keyboard and mouse + # event dispatched inside it carries them, and they are released when the + # block returns or raises. + # + # @param [Array<:alt, :ctrl, :control, :meta, :command, :shift>] keys + # + # @yield The input to dispatch with the modifiers held. + # + # @return [Object] The block's value. + # + # @raise [ArgumentError] If no block is given or a key isn't a modifier. + # + # @example + # page.keyboard.hold(:shift) do + # page.mouse.move(x: 10, y: 10).down.move(x: 200, y: 200, steps: 10).up + # end + # + def hold(*keys) + raise ArgumentError, "block is required" unless block_given? + + keys = keys.map { |key| modifier_name(key) || raise(ArgumentError, "not a modifier key: #{key.inspect}") } + previous = @held_modifiers + pressed = [] + + begin + keys.each { |key| press_modifier(key, pressed) } + result = yield + release_modifiers(pressed) + result + rescue StandardError + release_modifiers(pressed, quietly: true) + raise + ensure + @held_modifiers = previous + end + end + # # Returns bitfield for a given keys. # @@ -99,6 +150,42 @@ def modifiers(keys) private + def modifier_name(key) + name = key.to_s.downcase + name if MODIFIERS.key?(name) + end + + def reject_modifier(key) + return unless key.is_a?(Symbol) && modifier_name(key) + + raise ArgumentError, "#{key.inspect} is a modifier, use keyboard.hold(#{key.inspect}) { ... } instead" + end + + def press_modifier(name, pressed) + bit = MODIFIERS[name] + return if @held_modifiers.anybits?(bit) + + @held_modifiers |= bit + dispatch_modifier("rawKeyDown", name) + pressed.push(name) + end + + def release_modifiers(pressed, quietly: false) + while (name = pressed.pop) + @held_modifiers &= ~MODIFIERS[name] + begin + dispatch_modifier("keyUp", name) + rescue Error + raise unless quietly + end + end + end + + def dispatch_modifier(type, name) + key = to_options(KEYS.fetch(KEYS_MAPPING[name.to_sym])) + @page.command("Input.dispatchKeyEvent", slowmoable: true, type: type, modifiers: @held_modifiers, **key) + end + # rubocop:disable Metrics/PerceivedComplexity # rubocop:disable Metrics/CyclomaticComplexity def normalize_keys(keys, pressed_keys = [], memo = []) @@ -120,33 +207,34 @@ def normalize_keys(keys, pressed_keys = [], memo = []) nil else key = KEYS.fetch(KEYS_MAPPING[key.to_sym] || key.to_sym) - key[:modifiers] = pressed_keys.flatten.map { |k| MODIFIERS[k] }.reduce(0, :|) + key[:modifiers] = pressed_keys.flatten.map { |k| MODIFIERS[k] }.reduce(@held_modifiers, :|) to_options(key) end when String raise ArgumentError, "empty keys passed" if keys.empty? pressed = pressed_keys.flatten + mask = pressed.map { |k| MODIFIERS[k] }.reduce(@held_modifiers, :|) keys.each_char.map do |char| key = KEYS[char] || {} - if pressed.empty? + if mask.zero? key = key.merge(text: char, unmodifiedText: char) [to_options(key)] else - text = pressed == ["shift"] ? char.upcase : char + text = mask == MODIFIERS["shift"] ? char.upcase : char key = key.merge( text: text, unmodifiedText: text, isKeypad: key["location"] == 3, - modifiers: pressed.map { |k| MODIFIERS[k] }.reduce(0, :|) + modifiers: mask ) # Synthetic key events alone do not trigger the browser's editing # shortcuts (e.g. select-all). When the chord matches a platform # accelerator, name the editing command explicitly so Chrome # executes it via the CDP `commands` field. - command = editing_command(pressed, char) + command = editing_command(mask, char) key = key.merge(commands: [command]) if command modifiers = pressed.map { |k| to_options(KEYS.fetch(KEYS_MAPPING[k.to_sym])) } @@ -175,12 +263,11 @@ def to_options(hash) # is not an editing accelerator. The accelerator modifier is Cmd on macOS # and Ctrl elsewhere, matching Chrome's own key bindings. Currently only # select-all is mapped; extend the guard for further commands as needed. - def editing_command(pressed, char) + def editing_command(mask, char) return unless char.casecmp?("a") - return if pressed.include?("shift") accelerator = Utils::Platform.mac? ? MODIFIERS["meta"] : MODIFIERS["ctrl"] - "selectAll" if pressed.map { |k| MODIFIERS[k] }.compact.uniq == [accelerator] + "selectAll" if mask == accelerator end end end diff --git a/lib/ferrum/mouse.rb b/lib/ferrum/mouse.rb index f34985e5..355f2536 100644 --- a/lib/ferrum/mouse.rb +++ b/lib/ferrum/mouse.rb @@ -81,12 +81,13 @@ def scroll_to(top, left) # @option options [Integer] :count (1) # # @option options [Integer] :modifiers - # Bitfield for key modifiers. See`keyboard.modifiers`. + # Bitfield for key modifiers. See `keyboard.modifiers`. + # If omitted, uses the keys held by `keyboard.hold`, or none outside of it. # # @return [self] # def click(x:, y:, delay: 0, wait: CLICK_WAIT, **options) - move(x: x, y: y) + move(x: x, y: y, modifiers: options[:modifiers]) down(**options) sleep(delay) # Potential wait because if some network event is triggered then we have @@ -107,7 +108,8 @@ def click(x:, y:, delay: 0, wait: CLICK_WAIT, **options) # @option options [Integer] :count (1) # # @option options [Integer] :modifiers - # Bitfield for key modifiers. See`keyboard.modifiers`. + # Bitfield for key modifiers. See `keyboard.modifiers`. + # If omitted, uses the keys held by `keyboard.hold`, or none outside of it. # # @return [self] # @@ -127,7 +129,8 @@ def down(**options) # @option options [Integer] :count (1) # # @option options [Integer] :modifiers - # Bitfield for key modifiers. See`keyboard.modifiers`. + # Bitfield for key modifiers. See `keyboard.modifiers`. + # If omitted, uses the keys held by `keyboard.hold`, or none outside of it. # # @return [self] # @@ -145,9 +148,13 @@ def up(**options) # @param [Integer] steps # Sends intermediate mousemove events. # + # @param [Integer, nil] modifiers + # Bitfield for key modifiers. See `keyboard.modifiers`. + # If omitted, uses the keys held by `keyboard.hold`, or none outside of it. + # # @return [self] # - def move(x:, y:, steps: 1) + def move(x:, y:, steps: 1, modifiers: nil) from_x = @x from_y = @y @x = x @@ -164,7 +171,8 @@ def move(x:, y:, steps: 1) x: new_x, y: new_y, button: button, - buttons: @buttons) + buttons: @buttons, + modifiers: modifiers || @page.keyboard.held_modifiers) end self @@ -175,8 +183,8 @@ def move(x:, y:, steps: 1) def mouse_event(type:, button: :left, count: 1, modifiers: nil, wait: 0) button = validate_button(button) register_event_button(type, button) - options = { x: @x, y: @y, type: type, button: button, buttons: @buttons, clickCount: count } - options.merge!(modifiers: modifiers) if modifiers + options = { x: @x, y: @y, type: type, button: button, buttons: @buttons, clickCount: count, + modifiers: modifiers || @page.keyboard.held_modifiers } @page.command("Input.dispatchMouseEvent", wait: wait, slowmoable: true, **options) end diff --git a/sig/ferrum/keyboard.rbs b/sig/ferrum/keyboard.rbs index 2ce38756..57f1a4a6 100644 --- a/sig/ferrum/keyboard.rbs +++ b/sig/ferrum/keyboard.rbs @@ -5,6 +5,9 @@ module Ferrum KEYS_MAPPING: Hash[Symbol, String] @page: Page + @held_modifiers: ::Integer + + attr_reader held_modifiers: ::Integer def initialize: (Page page) -> void @@ -14,14 +17,28 @@ module Ferrum def type: (*(String | Symbol | ::Array[String | Symbol])) -> self + def hold: [T] (*(String | Symbol) keys) { () -> T } -> T + def modifiers: (::Array[Symbol]) -> ::Integer private + def modifier_name: (String | Symbol key) -> String? + + def reject_modifier: (String | Symbol | ::Array[String | Symbol] key) -> void + + def press_modifier: (String name, ::Array[String] pressed) -> void + + def release_modifiers: (::Array[String] pressed, ?quietly: bool) -> void + + def dispatch_modifier: (String type, String name) -> void + def normalize_keys: ((::Array[untyped] | Symbol | String) keys, ?::Array[::Array[String]] pressed_keys, ?::Array[Hash[Symbol, untyped]] memo) -> ::Array[Hash[Symbol, untyped]] def combine_strings: (::Array[untyped] keys) -> ::Array[untyped] def to_options: (Hash[String, untyped] hash) -> Hash[Symbol, untyped] + + def editing_command: (::Integer mask, String char) -> String? end end diff --git a/sig/ferrum/mouse.rbs b/sig/ferrum/mouse.rbs index 7d87eeeb..5c9050e3 100644 --- a/sig/ferrum/mouse.rbs +++ b/sig/ferrum/mouse.rbs @@ -21,7 +21,7 @@ module Ferrum def up: (?button: Symbol, ?count: ::Integer, ?modifiers: ::Integer, ?wait: ::Numeric) -> self - def move: (x: ::Numeric, y: ::Numeric, ?steps: ::Integer) -> self + def move: (x: ::Numeric, y: ::Numeric, ?steps: ::Integer, ?modifiers: ::Integer?) -> self private diff --git a/spec/keyboard_spec.rb b/spec/keyboard_spec.rb index 17ef08a0..0b24b0bb 100644 --- a/spec/keyboard_spec.rb +++ b/spec/keyboard_spec.rb @@ -11,6 +11,10 @@ expect { browser.keyboard.down(:page_down) }.not_to raise_error expect { browser.keyboard.down(1) }.to raise_error(ArgumentError, "unexpected argument") end + + it "rejects a modifier key" do + expect { browser.keyboard.down(:shift) }.to raise_error(ArgumentError, /use keyboard.hold\(:shift\)/) + end end describe "#up" do @@ -23,5 +27,84 @@ expect { browser.keyboard.up(:page_down) }.not_to raise_error expect { browser.keyboard.up(1) }.to raise_error(ArgumentError, "unexpected argument") end + + it "rejects a modifier key" do + expect { browser.keyboard.up(:Control) }.to raise_error(ArgumentError, /use keyboard.hold\(:Control\)/) + end + end + + describe "#hold" do + before do + browser.go_to("/type") + browser.execute(<<~JS) + window.events = []; + ["keydown", "keyup", "mousemove", "mousedown", "mouseup"].forEach(type => + document.addEventListener(type, e => events.push(`${type}:${e.shiftKey}:${e.ctrlKey}`))); + JS + end + + it "holds a modifier for mouse events and releases it after the block" do + browser.keyboard.hold(:shift) do + browser.mouse.move(x: 10, y: 10).down.move(x: 50, y: 50, steps: 2).up + end + browser.mouse.move(x: 60, y: 60) + + expect(browser.evaluate("events")).to eq( + %w[keydown:true:false mousemove:true:false mousedown:true:false mousemove:true:false + mousemove:true:false mouseup:true:false keyup:false:false mousemove:false:false] + ) + expect(browser.keyboard.held_modifiers).to eq(0) + end + + it "presses nested modifiers once" do + browser.keyboard.hold(:shift) do + browser.keyboard.hold(:shift, :ctrl) { browser.mouse.move(x: 10, y: 10) } + browser.mouse.move(x: 20, y: 20) + end + + expect(browser.evaluate("events")).to eq( + %w[keydown:true:false keydown:true:true mousemove:true:true keyup:true:false + mousemove:true:false keyup:false:false] + ) + end + + it "releases modifiers when the block raises" do + expect do + browser.keyboard.hold(:shift) { raise "boom" } + end.to raise_error(RuntimeError, "boom") + browser.mouse.move(x: 10, y: 10) + + expect(browser.evaluate("events")).to eq(%w[keydown:true:false keyup:false:false mousemove:false:false]) + expect(browser.keyboard.held_modifiers).to eq(0) + end + + it "returns the block value" do + expect(browser.keyboard.hold(:shift) { 42 }).to eq(42) + end + + it "types with the held modifier" do + input = browser.at_css("#empty_input") + + input.focus + browser.keyboard.hold(:shift) { browser.keyboard.type("abc") } + + expect(input.value).to eq("ABC") + end + + it "selects all with the held command modifier" do + input = browser.at_css("#filled_input") + modifier = Ferrum::Utils::Platform.mac? ? :meta : :ctrl + + input.focus + browser.keyboard.hold(modifier) { browser.keyboard.type("a") } + browser.keyboard.type(:backspace) + + expect(input.value).to eq("") + end + + it "rejects keys that are not modifiers" do + expect { browser.keyboard.hold(:enter) { nil } }.to raise_error(ArgumentError, "not a modifier key: :enter") + expect { browser.keyboard.hold(:shift) }.to raise_error(ArgumentError, "block is required") + end end end diff --git a/spec/mouse_spec.rb b/spec/mouse_spec.rb index df683516..3b292ff8 100644 --- a/spec/mouse_spec.rb +++ b/spec/mouse_spec.rb @@ -57,6 +57,23 @@ end describe "#move" do + it "sends modifiers" do + browser.go_to("/simple") + browser.execute(<<~JS) + window.events = []; + ["mousemove", "mousedown", "mouseup"].forEach(type => + document.addEventListener(type, e => events.push(`${type}:${e.shiftKey}:${e.altKey}`))); + JS + + browser.mouse.move(x: 10, y: 10, modifiers: 8) + browser.keyboard.hold(:shift) { browser.mouse.move(x: 20, y: 20, modifiers: 1) } + browser.mouse.click(x: 30, y: 30, modifiers: 8) + + expect(browser.evaluate("events")).to eq( + %w[mousemove:true:false mousemove:false:true mousemove:true:false mousedown:true:false mouseup:true:false] + ) + end + it "drags and drops with a button held" do browser.go_to("/html5_drag")