Keep access tokens and raw PII out of the Messenger transport - #49
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #49 +/- ##
=========================================
Coverage 97.95% 97.95%
- Complexity 151 156 +5
=========================================
Files 32 32
Lines 488 489 +1
=========================================
+ Hits 478 479 +1
Misses 10 10
🚀 New features to boost your workflow:
|
7107a61 to
b06fbd4
Compare
9a99ee8 to
dcbe1cb
Compare
b06fbd4 to
db88227
Compare
dcbe1cb to
af21153
Compare
db88227 to
bb2ba25
Compare
af21153 to
bac5d1e
Compare
bb2ba25 to
865684c
Compare
bac5d1e to
c9f1d9d
Compare
77d1a3e to
d13d703
Compare
b740807 to
e87c757
Compare
d13d703 to
6857dd6
Compare
e9fee59 to
861c973
Compare
|
The two latent |
861c973 to
e3e0dce
Compare
|
|
||
| public function __construct(PreparedEvent $preparedEvent) | ||
| { | ||
| // Stripping the tokens here rather than trusting the caller to have called withoutAccessTokens() means there |
There was a problem hiding this comment.
Why don't you call the intended method on PreparedEvent to remove the access tokens?
There was a problem hiding this comment.
No good reason. I had folded stripping and pixel deduplication into one loop, and paid for it by not using the method that exists for exactly this. It is now just:
$this->preparedEvent = $preparedEvent->withoutAccessTokens();The deduplication is gone with it. It guarded against a provider returning the same pixel twice, which is the provider's bug, and the configuration already rejects a pixel id listed twice.
| // A message serialised by 0.1.x carries an Event object instead of a prepared event. Messenger's PhpSerializer | ||
| // decodes it without complaint because the class still exists, but the property of the current shape is left | ||
| // uninitialised. Retrying cannot help, and the old body still holds the access token and the raw personal | ||
| // data this shape was introduced to keep out of the transport, so fail fast: the message goes to the failure | ||
| // transport with a reason instead of through three retries first. See UPGRADE.md | ||
| if (!(new \ReflectionProperty($message, 'preparedEvent'))->isInitialized($message)) { | ||
| throw new UnrecoverableMessageHandlingException('This SendEvent was serialised by a previous release of the bundle and cannot be handled. Drain the transport on that release before deploying, see UPGRADE.md'); |
There was a problem hiding this comment.
I don't care about this. Maybe put it in a upgrade document
There was a problem hiding this comment.
Removed, along with its test. UPGRADE.md already had a "Deploying" paragraph; I reworded it for the behaviour without the guard: an old-shape message fails, is retried and ends up in the failure transport, so drain the transport on the old release before deploying and clear the failure transport afterwards with messenger:failed:remove.
| if (null !== $pixel->accessToken) { | ||
| return true; | ||
| foreach ($preparedEvent->pixels as $pixel) { | ||
| $accessToken = $this->accessTokenResolver->resolve($pixel->id); |
There was a problem hiding this comment.
The PreparedEvent has the withAccessTokens method, so maybe we should just give it all our access tokens and remove the access token resolver?
There was a problem hiding this comment.
Done. AccessTokenResolverInterface and ConfigurationBasedAccessTokenResolver are deleted, and the handler is now:
$preparedEvent = $message->preparedEvent->withAccessTokens($this->accessTokens());One choice to flag: "all our access tokens" are taken from PixelProviderInterface, not read from the pixels configuration directly. For the default setup that is the configuration, so nothing changes. But an application that aliased the provider to its own service would otherwise lose server-side tracking entirely, since its tokens are not in the config. The cost is that a custom provider is now also called in the worker, so it has to work without a request; the interface docblock, the README and UPGRADE.md say so, and those replace the three "alias the resolver as well" caveats, which are gone.
If you would rather have it config-only, it is a two-line change.
What I kept: dropping pixels that still have no token after withAccessTokens(). That fixed a confirmed bug (#21, merged in #41: an unset META_ACCESS_TOKEN gave a token-less pixel, Meta answered 400, and Messenger retried into the failure transport), and the SDK client still posts to every pixel it is handed. It would arguably sit better in Client::sendPreparedEvent() itself; if the SDK ever skips token-less pixels, this loop can go too.
538493f to
176cc89
Compare
SendEvent carried the Event object, so routing it to a transport wrote the access token and every raw email, phone number and name into that transport's storage, and into the failure transport on failure. Hashing only happened later, inside Client::sendEvent(). Require setono/meta-conversions-api-php-sdk ^2.0.0-alpha.3, which has a first-class answer to this: Event::prepare() returns a PreparedEvent whose payload is already normalized and hashed, and ClientInterface::sendPreparedEvent() sends it. SendEvent now carries that PreparedEvent, stripped with withoutAccessTokens() in its constructor so no caller can put a token on the transport. The handler adds the tokens back with withAccessTokens(), taking them from the PixelProviderInterface. The handler also maps the SDK's exception model onto Messenger: invalid input and a rejection by Meta are unrecoverable, while a transport failure, a 5xx response and an error Meta flags as transient are left for Messenger to retry. Skipping a pixel without an access token is left to the SDK client, which does that as of alpha.3, so the handler no longer filters pixels or logs. An integration test pins that behaviour to the real client. Fixes #17
176cc89 to
6c3f795
Compare
Fixes #17
Problem
SendEventcarried the wholeEventobject. At dispatch time that object holds the pixels including their access tokens, and the raw email addresses, phone numbers, names and dates of birth the application attached, because normalisation and hashing only happened inside the SDK client, in the handler.Routing the command to a transport, which is the recommended setup, therefore wrote all of that into the transport's storage, into the failure transport when it failed, and into anything that dumps messages such as
messenger:failed:show. Failure transports are often kept indefinitely, so it was a retention problem as well as a secrets problem.Change
Requires
setono/meta-conversions-api-php-sdk^2.0.0-alpha.3, which has a first-class answer to this (Setono/meta-conversions-api-php-sdk#15):Event::prepare()returns aPreparedEventwhose payload is already normalised and hashed, andClientInterface::sendPreparedEvent()sends it.SendEventcarries the SDK'sPreparedEvent, and its constructor callswithoutAccessTokens().prepare()keeps the tokens on the pixels, so doing it in the constructor, rather than trusting each caller, means there is no way to put a token on the transport.withAccessTokens(), taking them fromPixelProviderInterface. For the default setup that is thepixelsconfiguration. A provider of your own keeps working server side, but is now also called in the worker, so it has to work without a request; the interface docblock, README andUPGRADE.mdsay so.InvalidArgumentException, and aResponseExceptionthat is neither a 5xx nor flaggedtransientby Meta, becomeUnrecoverableMessageHandlingException, keeping the original asprevious: an invalid access token is rejected again however many times Messenger retries. ATransportException, a 5xx and a transient error are left alone, so Messenger retries them. Previously everything was retried three times regardless.InvalidArgumentExceptionbecomes unrecoverable, so the message goes to the failure transport and can be retried once the token is configured, instead of being dropped with a log line as before.There is no bundle-specific abstraction in this: no resolver interface, no
PreparedEventsubclass, no pixel filtering, no runtime guard for old messages.UPGRADE.mdcovers draining the transport before deploying instead.Two things to know before merging
composer require setono/meta-conversions-api-php-sdk:^2.0@alpha.UPGRADE.mdsays so. A stable 1.0 of this bundle should wait for a stable 2.0 of the SDK, at which point the constraint becomes^2.0and that note goes.ClientInterfacegained a method, and the bundle now calls it. Anyone who decorates or replaces the SDK client must implementsendPreparedEvent(). A decorator that only wrapssendEvent()is no longer on the path the bundle's events take.ClientExceptionis gone too; both are inUPGRADE.md.Tests
SendEventTest: serialising a message built from an event with an access token, an email address, a phone number and a first name contains none of those four strings, while the payload holds the SHA-256 of the email; the constructor strips tokens from a prepared event that still has them; the message round-trips throughserialize().SendEventHandlerTest: the payload reachessendPreparedEvent()untouched with the token back andsendEvent()is never called; pixels without a token, one the provider does not know and one it has no token for, reach the client as they are. Five failures that must not be retried (invalid input, no pixel with a token, an invalid token, an explicitly non-transient error, a 403 from a proxy) and three that must (a transport failure, a 503, a transient rate limit), each asserting the original exception survives.PipelineTest, end to end through a booted kernel, asserts the token round trip: stripped before dispatch, back on the pixel the client receives.Verified on PHP 8.1 and 8.4, with
--prefer-lowestand highest, plus PHPStan, ECS, Rector, the dependency analyser,composer validateandcomposer normalize.