fix(core): preserve type through offloading + reject removed messageTypeField#429
fix(core): preserve type through offloading + reject removed messageTypeField#429
Conversation
…time `messageTypeField` was removed in core 25.x in favor of `messageTypeResolver`. Until now the option was simply absent from `CommonQueueOptions`, but TypeScript's excess-property checks are weakened across the long `Omit`/`&`/generic chains used by publisher options (`SnsPublisherManager.newPublisherOptions` etc.), so callers who hadn't migrated were silently passing a no-op option. The runtime impact is severe and silent: with payload offloading enabled, the offload step in `AbstractQueueService` only preserves the message `type` field on the offloaded body when `messageTypeResolver` is configured. Callers still on `messageTypeField` end up publishing SNS messages without `type` in the body, and any subscription with `FilterPolicyScope: 'MessageBody'` filtering on `type` silently drops the message before SQS delivery. Production-confirmed against file-storage-service after its 24.x → 25.x bump. Type the field as `?: never` with deprecation guidance so it now fails compilation with a clear error pointing at the migration. Add type-level guards in core and sns test suites that lock the contract in place. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughAdds a deprecated removal-marker Changes
Sequence Diagram(s)sequenceDiagram
participant Publisher as Publisher
participant Queue as AbstractQueueService
participant Store as PayloadStore
participant SNS as SNS (topic)
Publisher->>Queue: publish(message, options)
Queue->>Queue: determine messageTypePath (resolver or fallback 'type')
alt message exceeds threshold -> offload
Queue->>Store: store(payload) -> returns payloadRef
Store-->>Queue: payloadRef
Queue->>SNS: send({ payloadRef, type: message[typePath]? })
else small message -> inline
Queue->>SNS: send(message)
end
SNS-->>Publisher: ack
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/core/lib/types/queueOptionsTypes.ts`:
- Around line 117-126: The field messageTypeField?: never does not fully prevent
callers from passing messageTypeField: undefined unless TypeScript's
exactOptionalPropertyTypes is enabled; either enable
"exactOptionalPropertyTypes": true in the packages/core/tsconfig.json to make
optional properties typed as exact and enforce the never ban, or replace the
declaration with a stricter pattern (e.g., a branded/deprecated type or separate
deprecated interface) so passing undefined is a compile-time error—update the
definition of messageTypeField in types/queueOptionsTypes.ts (and update any
related interfaces) to use the chosen strict approach.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 496c146c-d9b6-4c07-8a99-6577601ad056
📒 Files selected for processing (3)
packages/core/lib/types/queueOptionsTypes.tspackages/core/test/types/queueOptionsTypes.types.spec.tspackages/sns/test/SnsPublisherManager.types.spec.ts
…ssage Per PR review: `messageTypeField?: never` produces a generic "Type 'string' is not assignable to type 'never'" diagnostic. Switch to a removal-marker string literal so the diagnostic surfaces the migration hint right at the offending call site, e.g. "Type 'type' is not assignable to type '__REMOVED_USE_messageTypeResolver_INSTEAD__ | undefined'". Note on the `messageTypeField: undefined` loophole: closing it would require enabling `exactOptionalPropertyTypes` package-wide, which produces ~13 unrelated type errors in core (HandlerContainer, MessageSchemaContainer, AbstractQueueService, startupResourcePolling) and is a separate refactor. Passing `undefined` is operationally identical to omitting the field, so this guard catches the realistic regression — `messageTypeField: '<name>'`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mode
`offloadMessagePayloadIfNeeded` previously only re-attached the message
`type` field on the offloaded payload when `messageTypeResolver` was a
`messageTypePath` config. With no resolver, or with `literal` /
`resolver` modes, the field was silently stripped.
This was inconsistent with the rest of the offloader, which copies
identity fields (`messageIdField`, `messageTimestampField`,
`messageDeduplicationIdField`, `messageDeduplicationOptionsField`)
unconditionally based on their defaulted field names ('id',
'timestamp', ...), and with `MessageSchemaContainer.resolveSchema`,
which falls back to a single default schema when no resolver is
configured.
Symptoms: large messages get offloaded to S3 and the SNS publish
succeeds, but any subscription with `FilterPolicyScope: 'MessageBody'`
filtering on `type` silently drops them — the body the filter sees
no longer contains a `type` field. Production-confirmed against
file-storage-service after its 24.x → 25.x bump
(lokalise/file-storage-service#1027).
Default the type-preservation path to the conventional top-level `type`
when the resolver doesn't provide a body path. The literal/resolver
modes still drive routing/dispatch as before; this is purely about
keeping the field available for downstream filters.
Add direct unit tests for the offload behavior across all four resolver
configurations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
| it('rejects the removed `messageTypeField` option', () => { | ||
| const _bad: NewPublisherOptions = { | ||
| // @ts-expect-error - `messageTypeField` was removed in core 25.x; use `messageTypeResolver` instead | ||
| messageTypeField: 'type', | ||
| messageIdField: 'id', | ||
| } |
There was a problem hiding this comment.
Could we use the expect type of option from vitest?
Summary
Two related fixes for the same regression class — silent drop of the message `type` field through payload offloading.
1. Type-level rejection of the removed
messageTypeFieldoptionmessageTypeFieldwas removed in core 25.x in favor ofmessageTypeResolver. Until now the option was simply absent fromCommonQueueOptions, but TypeScript's excess-property checks are weakened across the longOmit/&/generic chains used by publisher options (e.g.SnsPublisherManager.newPublisherOptions), so callers who hadn't migrated kept silently passing a no-op option.Fix: type `messageTypeField` as a removal-marker string-literal (`'REMOVED_USE_messageTypeResolver_INSTEAD'`). Passing it now fails compilation with a diagnostic that surfaces the migration hint at the call site.
2. Consistent type-field preservation through offloading
AbstractQueueService.offloadMessagePayloadIfNeededpreserved thetypefield only whenmessageTypeResolverwas themessageTypePathflavor. With no resolver, or withliteral/resolvermodes, the field was silently stripped. This was inconsistent with:messageIdField/messageTimestampField/messageDeduplicationIdField/messageDeduplicationOptionsFieldunconditionally based on their defaulted field names (`'id'`, `'timestamp'`, ...);MessageSchemaContainer.resolveSchema, which falls back to a single default schema when no resolver is configured.Fix: default the type-preservation path to the conventional top-level `type` when the resolver doesn't provide a body path. The literal/resolver modes still drive routing/dispatch as before; this only keeps the field present for downstream consumers (notably SNS subscriptions whose FilterPolicy filters on the body `type`).
Production-confirmed regression in file-storage-service: after the 24.x → 25.x bump, large `statistic.generated` messages were getting offloaded successfully and SNS-published successfully, but never reached SQS because the SQS subscription's `FilterPolicyScope: 'MessageBody'` filter on `type` no longer matched. See lokalise/file-storage-service#1027.
Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit
Documentation
Bug Fixes
Tests