feat(rest/auth): SigV4 authentication for AWS Glue Iceberg REST#4
Merged
mmaslankaprv merged 11 commits intomainfrom Apr 22, 2026
Merged
feat(rest/auth): SigV4 authentication for AWS Glue Iceberg REST#4mmaslankaprv merged 11 commits intomainfrom
mmaslankaprv merged 11 commits intomainfrom
Conversation
Needed for AWS SigV4 request signing in the REST catalog client (AWS Glue's Iceberg REST endpoint rejects OAuth bearer tokens and requires SigV4). The CMake resolver is QUIET so the dep stays optional: when found, ICEBERG_REST_HAVE_SIGV4 is exported for downstream targets to gate the SigV4 code path; when absent, the REST client still builds.
Adds a SignableRequest struct (method, URL, query params, body) and a virtual Authenticate(SignableRequest, headers) overload that by default forwards to the existing headers-only method. Needed for SigV4, which must hash the body and include the method / URL / query in its canonical request. Existing None/Basic/OAuth2 sessions ignore the extra context and need no changes.
Every HttpClient method (Get/Post/PostForm/Head/Delete) now forwards method, URL, query params, and body to BuildHeaders so the AuthSession's richer Authenticate overload can see what's being sent. PostForm pre-encodes the form body once via UrlEncoder so the bytes that get signed match what libcurl puts on the wire. No-op for existing auth schemes (they don't override the new overload); unlocks SigV4 signing in a follow-up.
Declares property keys for SigV4 static credentials (access-key-id, secret-access-key, session-token) and a credentials-provider selector that toggles between static keys and the aws-crt-cpp default credential chain. Also adds the execute-api signing-service default that matches the Java Iceberg client. Consumed by the SigV4 manager in a follow-up; no behaviour change on its own.
Wraps aws-crt-cpp's Sigv4HttpRequestSigner + AwsSigningConfig to sign outgoing REST requests for AWS Glue's Iceberg REST endpoint. - SigV4Signer holds a shared ICredentialsProvider built from config: either static keys (rest.auth.sigv4.access-key-id/...) or the aws-crt-cpp default chain (Environment -> Profile -> STS Web Identity -> IMDS/ECS), with auto-detection when the provider type is unset. - Signing options mirror the SigV4 spec for non-S3 services: double URI encoding, path normalization, explicit sha256 payload hash via x-amz-content-sha256 (worked around the Java Iceberg empty-body SDK bug). - SigV4AuthSession runs an optional delegate session first so its Authorization header is covered by the signature; the delegate's Authorization is renamed to X-Iceberg-Access-Delegation before signing to avoid aws-c-auth's reserved-header rejection. - MakeSigV4Manager is registered in the auth-manager registry when ICEBERG_REST_WITH_SIGV4 is defined; otherwise the registry keeps returning NotImplemented. - CMakeLists gates the sigv4_signer.cc source + AWS::aws-crt-cpp link on the cache variable set by the toolchain resolver.
- auth_manager_test.cc: 12 SigV4 manager cases covering case-insensitive auth-type, required-field validation, header emission, session token pass-through, OAuth2-delegate wrapping, recursion rejection, and the default / static / unknown credentials-provider paths. - sigv4_signer_test.cc (new): canonical SigV4 vectors with the signing time pinned via SigV4Signer::MakeForTests. Expected Authorization headers were generated with Python botocore (SigV4Auth with the same parameters as our signer) and are asserted bytewise so any regression in our aws-crt-cpp wrapping is caught. - CMakeLists gates sigv4_signer_test.cc on ICEBERG_REST_HAVE_SIGV4 and defines ICEBERG_REST_WITH_SIGV4 for the whole rest_catalog_test target so conditional tests compile.
Whitespace-only reformats across the SigV4 additions; no behaviour change. Tests still pass.
- Replace the C-style ``unsigned char digest[]`` scratch buffer with ``std::array`` and feed ``digest.data()`` to ``SHA256`` (modernize- avoid-c-arrays). - Swap the hex lookup table to a ``constexpr std::string_view`` instead of a C-string array (modernize-avoid-c-arrays). - Use braced-init return in ``ExtractPath`` (modernize-return-braced- init-list). No behaviour change; the canonical-vector tests still produce the same Authorization headers.
Without aws-crt-cpp on the lint runner, clang-tidy reports sigv4_signer.cc's ``#include <aws/crt/Api.h>`` as file-not-found and fails the job even though the rest of the file is conceptually clean. Switch the lint job to resolve dependencies via Conan (``conan install . -s build_type=Release --build=missing``) and configure with the resulting ``conan-release`` preset. aws-crt-cpp comes in as a Conan package and the generated compile_commands.json now carries the right include paths for the SigV4 source. Updated the cpp-linter ``database`` and ``extra-args`` inputs to point at the new ``build/Release`` layout.
- sigv4_signer.cc: ByteCursorToStdString now uses braced-init return to satisfy modernize-return-braced-init-list (missed in the previous sweep). - cpp-linter.yml: override ICEBERG_BUILD_TESTS=ON in the lint build so the gtest/gmock test targets land in compile_commands.json. Without this, clang-tidy can't resolve <gmock/gmock.h> in auth_manager_test or sigv4_signer_test because the conanfile disables tests for the package build by default.
Enabling ICEBERG_BUILD_TESTS with the default bundle ON was pulling in arrow-backed test targets (avro/arrow/parquet/scan), which Conan had to compile from source for gcc-14 — pushing the lint job past 30 minutes. Since the lint only needs compile_commands.json entries for our sigv4_signer / rest auth / rest test files, configure with BUNDLE=OFF and build only the ``rest_catalog_test`` target. cpp-linter still gets full coverage of every file touched in this PR.
bharathv
approved these changes
Apr 21, 2026
Comment on lines
+591
to
+593
| set(ICEBERG_REST_HAVE_SIGV4 | ||
| FALSE | ||
| PARENT_SCOPE) |
There was a problem hiding this comment.
can we just fail? This seems unlikely?
Member
Author
There was a problem hiding this comment.
this is for completeness as Meson build doesn't require the AWS CRT
Comment on lines
+54
to
+70
| struct SigV4Config { | ||
| /// AWS region (e.g., "us-east-1"). Required. | ||
| std::string region; | ||
| /// AWS signing service name. "glue" for AWS Glue's Iceberg REST endpoint; | ||
| /// "execute-api" for API Gateway; "s3tables" for S3 Tables. Required. | ||
| std::string service; | ||
| /// Which credential source the signer should consult. | ||
| SigV4CredentialsProvider provider = SigV4CredentialsProvider::kStatic; | ||
| /// Static AWS access key ID. Required when ``provider == kStatic``. | ||
| std::string access_key_id; | ||
| /// Static AWS secret access key. Required when ``provider == kStatic``. | ||
| std::string secret_access_key; | ||
| /// Optional STS session token. When present, X-Amz-Security-Token is | ||
| /// added to the request and included in the signed header set. | ||
| /// Only consulted when ``provider == kStatic``. | ||
| std::string session_token; | ||
| }; |
Member
Author
There was a problem hiding this comment.
hmm, i do not see any place in which we conditionally check the config type
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds AWS Signature V4 (SigV4) authentication to the REST catalog client so it can talk to AWS Glue's Iceberg REST endpoint (
https://glue.<region>.amazonaws.com/iceberg). Glue rejects OAuth bearer tokens and requires SigV4 on every request.Uses
aws-crt-cpp(AWS's idiomatic C++ wrapper overaws-c-auth) so we don't hand-roll crypto. The dep is optional — whenaws-crt-cppisn't found via Conan/vcpkg the REST client still builds and thesigv4auth type returnsNotImplemented.Commit series (single-concern)
build(conan)— addsaws-crt-cppand a QUIET CMake resolver that exportsICEBERG_REST_HAVE_SIGV4when the dep is available.feat(rest/auth)— extendsAuthSessionwith aSignableRequestoverload (method/URL/query/body). Default forwards to the existing headers-only method, so None/Basic/OAuth2 sessions need no changes.feat(rest/http)— plumbs request context throughHttpClient::BuildHeadersand all five call sites;PostFormpre-encodes the body so signing sees the exact bytes libcurl sends.feat(rest/auth)— adds SigV4 property keys (static access-key-id / secret-access-key / session-token, a credentials-provider selector, and theexecute-apisigning-service default that matches the Java Iceberg client).feat(rest/auth)— implementsSigV4Signer+SigV4AuthSession+SigV4Manageron top ofaws-crt-cpp, registers the factory whenICEBERG_REST_WITH_SIGV4is defined, wires the Conan target intoiceberg_rest.test(rest/auth)— 12 manager tests + 4 canonical-vector signer tests (signing timestamp pinned, expectedAuthorizationheaders cross-checked with Pythonbotocore).Configuration
rest.auth.typesigv4.rest.auth.sigv4.regionus-east-1.rest.auth.sigv4.serviceexecute-apigluefor AWS Glue.rest.auth.sigv4.credentials-providerstaticordefault. Auto-detects:staticwhenaccess-key-idis set, elsedefault(env → profile → STS Web Identity → IMDS/ECS).rest.auth.sigv4.access-key-idrest.auth.sigv4.secret-access-keyrest.auth.sigv4.session-tokenrest.auth.sigv4.delegate-auth-typeoauth2) so its headers are covered by the signature.For Glue on EKS (IRSA):
rest.auth.type=sigv4,rest.auth.sigv4.service=glue,rest.auth.sigv4.region=<region>— no keys needed, the pod's service-account token resolves via the default chain.Design notes
should_normalize_uri_path=true.AwsSigningConfig::SetSignedBodyValue, avoiding aws-c-auth's empty-body handling bug that bit the Java Iceberg impl (apache/iceberg#6951).Authorization: Bearerheader would collide with theAuthorizationheader SigV4 emits and would also trigger aws-c-auth'sAWS_AUTH_SIGNING_ILLEGAL_REQUEST_HEADER. We rename it toX-Iceberg-Access-Delegationbefore signing, matching the JavaRESTSigV4AuthSession.aws-c-*is initialized once per process.AwsSigningConfig::SetCredentialsProvider, so caching and refresh are handled insideaws-c-auth— static keys are frozen for the life of the signer; the default chain refreshes.Test plan
conan install . -s build_type=Release --build=missingcmake --preset conan-release && cmake --build build/Release --target rest_catalog_testctest --test-dir build/Release -R rest_catalog --output-on-failure→ 199/199 pass locally, including 12 new manager cases + 4 canonical-vector signer cases.