Skip to content

Resolve last release-related Network Isolation issues#46456

Open
scbedd wants to merge 4 commits intomainfrom
network-isolated-verify-tags
Open

Resolve last release-related Network Isolation issues#46456
scbedd wants to merge 4 commits intomainfrom
network-isolated-verify-tags

Conversation

@scbedd
Copy link
Copy Markdown
Member

@scbedd scbedd commented Apr 21, 2026

  • WAS going to pypi to check of package is released, now attempt to pip download so that overriden PIP_INDEX_URL will be checked instead.
  • Still facing failure The issue was that we were exiting with the last $LASTEXITCODE. This was causing failure because the success case is not being able to resolve the dep. Instead we were treating that as the error case accidentally.

Proven successful

@scbedd scbedd marked this pull request as ready for review April 21, 2026 23:26
Copilot AI review requested due to automatic review settings April 21, 2026 23:26
@scbedd scbedd requested review from benbp and weshaggard as code owners April 21, 2026 23:26
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Python release publish-check logic in the engineering PowerShell scripts to work in network-isolated environments by avoiding direct calls to pypi.org and preventing $LASTEXITCODE from incorrectly causing task failures.

Changes:

  • Replaces PyPI REST probing with a pip download-based existence check that respects configured package index settings.
  • Adds temp download directory handling and cleanup.
  • Captures and resets $LASTEXITCODE to prevent “successful not-found” cases from propagating as pipeline failures.
Show a summary per file
File Description
eng/scripts/Language-Settings.ps1 Switches publish-status detection to pip download against configured indexes and avoids leaking non-zero $LASTEXITCODE to the ADO task.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 3

Comment on lines +233 to +236
Write-Host "Checking whether $pkgId==$pkgVersion is already published (using pip download)"

$pipArgs = @("download", "--no-deps", "--no-cache-dir", "--dest", $tmpDir, "$pkgId==$pkgVersion")

Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check uses pip download with --no-cache-dir, which guarantees the artifact is fetched each time even though you only need to know whether the version exists. If this runs across multiple packages, it can add noticeable time/bandwidth to release validation; consider a metadata-only existence check (e.g., querying the simple index) or an approach that avoids downloading the full distribution when possible.

See below for a potential fix:

# Uses the configured package index (PIP_INDEX_URL) rather than calling pypi.org
# directly, so this works in network-isolated (CFS) environments. The Azure
# Artifacts feed has upstream to PyPI, so packages on PyPI are still found.
# Query the simple index instead of downloading an artifact because this check
# only needs to know whether the version exists.
function IsPythonPackageVersionPublished($pkgId, $pkgVersion)
{
  try
  {
    Write-Host "Checking whether $pkgId==$pkgVersion is already published (using package index metadata)"

    $indexBaseUrl = if ($env:PIP_INDEX_URL) { $env:PIP_INDEX_URL } else { "https://pypi.org/simple" }
    if ($env:PIP_INDEX_URL) {
      Write-Host "Using index from PIP_INDEX_URL"
    }
    else {
      Write-Host "PIP_INDEX_URL is not set; checking public PyPI."
    }

    $indexBaseUrl = $indexBaseUrl.TrimEnd("/")
    if (-not $indexBaseUrl.EndsWith("/simple")) {
      $indexBaseUrl = "$indexBaseUrl/simple"
    }

    $normalizedPkgId = ([System.Text.RegularExpressions.Regex]::Replace($pkgId.ToLowerInvariant(), "[-_.]+", "-"))
    $packageIndexUrl = "$indexBaseUrl/$normalizedPkgId/"

    $escapedPkgId = [System.Text.RegularExpressions.Regex]::Escape($pkgId)
    $escapedPkgId = [System.Text.RegularExpressions.Regex]::Replace($escapedPkgId, "[-_.]+", "[-_.]+")
    $escapedPkgVersion = [System.Text.RegularExpressions.Regex]::Escape($pkgVersion)
    $artifactPattern = "(?i)(?:^|/)$escapedPkgId-$escapedPkgVersion(?:[-_.].*?)?\.(?:tar\.gz|zip|whl)(?:#|$)"

    $response = Invoke-WebRequest -Uri $packageIndexUrl -UseBasicParsing

    foreach ($link in $response.Links) {
      $href = [string]$link.href
      if ($href -match $artifactPattern) {
        Write-Host "Package $pkgId==$pkgVersion was found on the package index."
        return $True
      }
    }

    Write-Host "Package $pkgId==$pkgVersion was not found on the package index (not yet published)."
    return $False
  }
  catch
  {
    $statusCode = $null
    if ($_.Exception.Response -and $_.Exception.Response.StatusCode) {
      $statusCode = [int]$_.Exception.Response.StatusCode
    }

    if ($statusCode -eq 404) {
      Write-Host "Package $pkgId==$pkgVersion was not found on the package index (package not present on simple index)."
      return $False
    }

    # Any other failure is unexpected — fail hard to avoid accidentally re-publishing.
    Write-Host "Package version check failed unexpectedly:"
    Write-Host $_
    exit(1)
  }

Copilot uses AI. Check for mistakes.
Write-Host "PIP_INDEX_URL is not set; pip will fall back to public PyPI."
}

$pipOutput = pip $pipArgs 2>&1
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pip is invoked directly here, but elsewhere in these eng scripts the pattern is python -m pip ... (and sometimes uv pip ...). Calling pip can fail if the pip shim isn’t on PATH or points to a different Python environment than python. Consider using python -m pip download ... (or uv pip download when uv is available) for consistency and to ensure the correct interpreter/index configuration is used.

Suggested change
$pipOutput = pip $pipArgs 2>&1
$pipOutput = & python -m pip @pipArgs 2>&1

Copilot uses AI. Check for mistakes.
# Any other failure is unexpected — fail hard to avoid accidentally re-publishing.
Write-Host "Package version check failed unexpectedly:"
Write-Host $outputStr
exit(1)
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using exit(1) inside this helper function terminates the entire script immediately, which makes the function harder to reuse/test and bypasses any upstream error handling patterns. Prefer throwing a terminating error (or returning an error value and letting the caller decide) so failures can be handled consistently by the surrounding script logic.

Suggested change
exit(1)
throw "Package version check failed unexpectedly for ${pkgId}==${pkgVersion}:`n$outputStr"

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants