Add SSLSocket server handshake aliases

[samuel-williams-shopify]

TL;DR

Add OpenSSL::SSL::SSLSocket#start and #start_nonblock as aliases for #accept and #accept_nonblock.

Context

When OpenSSL::SSL::SSLServer#start_immediately is disabled, SSLServer#accept returns the accepted transport before performing the TLS handshake. The caller then invokes SSLSocket#accept on that already-accepted socket to perform the server-side handshake.

io-endpoint currently defines these aliases itself so accepted connections can be dispatched to a fiber or thread and the TLS handshake can be initiated with #start, without treating handshake initiation as another connection acceptance operation in generic server code.

Related to #760.

Changes

• Alias SSLSocket#start to SSLSocket#accept.
• Alias SSLSocket#start_nonblock to SSLSocket#accept_nonblock.
• Test that both aliases resolve to the existing methods.

The existing methods and behavior are unchanged.

Tophatting

• bundle exec rake compile
• bundle exec ruby -Itest -Ilib test/openssl/test_ssl.rb --name=test_server_handshake_aliases
• bundle exec rake test (628 tests, 4,606 assertions, 0 failures, 0 errors, 2 expected FIPS omissions)

[rhenium]

TLS is a client-initiated protocol, so if we had SSLSocket#start I'd expect it to mean #connect rather than #accept.

As I've written in #760, I don't really see problems with the current names #accept and #connect. What problem does adding aliases solve?

SSLSocket#{accept,connect} were taken from the corresponding OpenSSL C API SSL_accept() and SSL_connect(), so they are consistent in this regard at least.

[samuel-williams-shopify]

Thanks. The concrete problem is that accept cannot represent a post-accept negotiation capability in generic socket-handling code: every socket responds to accept, including sockets representing connections that have already been accepted.

io-endpoint accepts a connection, dispatches it to a fiber or thread, and then performs any required application-level negotiation:

if socket.respond_to?(:start)
  socket.start
end

The actual implementation is here: https://github.com/socketry/io-endpoint/blob/5e7c6aff6ba7b52c1a30f6d3a6e5a5aa86b41bf0/lib/io/endpoint/wrapper.rb#L217-L226

For an SSLSocket returned with SSLServer#start_immediately = false, that operation is the server-side TLS handshake. A plain socket requires no such step.

Therefore, io-endpoint currently adds SSLSocket#start itself as an alias for accept. The distinct name allows generic code to discover and invoke post-accept negotiation without checking for SSLSocket specifically or accidentally calling accept on an ordinary connected socket.

This PR does not replace or change accept and connect; it standardizes that distinct capability name for the server-side handshake.

[rhenium]

Shouldn't the TLS handshake generally handled outside generic socket handling code, since it is specific to TLS?

I don't think #start is currently an established convention for IO/Socket-like objects, and for SSLSocket in particular, it seems misleading because the server / #accept is not the party that initiates the handshake.

So, for now, I'm inclined to leave #accept as-is.

---

For an SSLSocket returned with SSLServer#start_immediately = false, that operation is the server-side TLS handshake. A plain socket requires no such step.

It's a good point that SSLServer does use start in this attribute, but I'd consider it a mistake rather than a good precedent. More broadly, SSLServer seems very incomplete. It's unusable when start_immediately is set to true, and it becomes a leaky abstraction when it's set to false.

Perhaps we should deprecate this class and recommend using TCPServer and SSLSocket directly, which I suspect most users are already doing.

[samuel-williams-shopify]

Shouldn't the TLS handshake generally be handled outside generic socket handling code, since it is specific to TLS?

The generic accept loop does not implement TLS. It handles the lifecycle transition from “transport connection accepted” to “socket ready for the application,” while the socket itself performs any protocol-specific negotiation:

if socket.respond_to?(:start)
  socket.start
end

This is the actual io-endpoint implementation.

That transition needs to occur after dispatching the connection to its fiber or thread, so negotiation for one connection does not serialize the accept loop. The generic code should not need to know whether that negotiation is TLS or inspect the concrete socket class.

I don't think #start is currently an established convention for IO/Socket-like objects.

I agree that it is not currently established; this PR proposes it as that convention. accept cannot serve this purpose because every socket responds to it, and checking explicitly for SSLSocket would make the generic accept loop protocol-specific.

If start is not the right name, what interface would you recommend for this generic post-connection negotiation hook? For example, should it be called handshake, negotiate, or something else? Alternatively, is there a different way you would structure this without requiring generic server code to identify each protocol-specific socket type?

Perhaps we should deprecate this class and recommend using TCPServer and SSLSocket directly.

Using those classes directly does not remove this lifecycle transition. After accepting the TCP connection and constructing the SSLSocket, it still needs to be dispatched before performing the server-side handshake.

I would like to find an upstream interface for that operation rather than requiring libraries to define their own convention or add type-specific branches.

[rhenium]

This is the actual io-endpoint implementation.

IMO SSLServer#start_immediately = false that makes SSLServer#accept return an SSLSocket object that hasn't finished the handshake was a design mistake. We should at least discourage future code from using it: #1096

I see no good reason for setting up the instance (SSLSocket.new) and performing the handshake (either #accept or #connect) to be handled by different layers of code. If they are handled together, the current names SSLSocket#{accept,connect} seem fine, since it's clear they do not refer to the syscalls accept(2) or connect(2).

I think #accept could potentially have a different name, such as #start_server, #do_handshake_as_server, or #ssl_accept. Another option would be to decompose it into two steps, #set_accept_state and #do_handshake, to mirror the OpenSSL API SSL_set_accept_state() and SSL_do_handshake().

But none of these names seem clearly enough better than #accept to justify renaming it now.