Resolve last release-related Network Isolation issues#46456
Resolve last release-related Network Isolation issues#46456
release-related Network Isolation issues#46456Conversation
…stream) enabled instead of going directly to pypi
There was a problem hiding this comment.
Pull request overview
This PR updates the Python release publish-check logic in the engineering PowerShell scripts to work in network-isolated environments by avoiding direct calls to pypi.org and preventing $LASTEXITCODE from incorrectly causing task failures.
Changes:
- Replaces PyPI REST probing with a
pip download-based existence check that respects configured package index settings. - Adds temp download directory handling and cleanup.
- Captures and resets
$LASTEXITCODEto prevent “successful not-found” cases from propagating as pipeline failures.
Show a summary per file
| File | Description |
|---|---|
eng/scripts/Language-Settings.ps1 |
Switches publish-status detection to pip download against configured indexes and avoids leaking non-zero $LASTEXITCODE to the ADO task. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 3
| Write-Host "Checking whether $pkgId==$pkgVersion is already published (using pip download)" | ||
|
|
||
| $pipArgs = @("download", "--no-deps", "--no-cache-dir", "--dest", $tmpDir, "$pkgId==$pkgVersion") | ||
|
|
There was a problem hiding this comment.
This check uses pip download with --no-cache-dir, which guarantees the artifact is fetched each time even though you only need to know whether the version exists. If this runs across multiple packages, it can add noticeable time/bandwidth to release validation; consider a metadata-only existence check (e.g., querying the simple index) or an approach that avoids downloading the full distribution when possible.
See below for a potential fix:
# Uses the configured package index (PIP_INDEX_URL) rather than calling pypi.org
# directly, so this works in network-isolated (CFS) environments. The Azure
# Artifacts feed has upstream to PyPI, so packages on PyPI are still found.
# Query the simple index instead of downloading an artifact because this check
# only needs to know whether the version exists.
function IsPythonPackageVersionPublished($pkgId, $pkgVersion)
{
try
{
Write-Host "Checking whether $pkgId==$pkgVersion is already published (using package index metadata)"
$indexBaseUrl = if ($env:PIP_INDEX_URL) { $env:PIP_INDEX_URL } else { "https://pypi.org/simple" }
if ($env:PIP_INDEX_URL) {
Write-Host "Using index from PIP_INDEX_URL"
}
else {
Write-Host "PIP_INDEX_URL is not set; checking public PyPI."
}
$indexBaseUrl = $indexBaseUrl.TrimEnd("/")
if (-not $indexBaseUrl.EndsWith("/simple")) {
$indexBaseUrl = "$indexBaseUrl/simple"
}
$normalizedPkgId = ([System.Text.RegularExpressions.Regex]::Replace($pkgId.ToLowerInvariant(), "[-_.]+", "-"))
$packageIndexUrl = "$indexBaseUrl/$normalizedPkgId/"
$escapedPkgId = [System.Text.RegularExpressions.Regex]::Escape($pkgId)
$escapedPkgId = [System.Text.RegularExpressions.Regex]::Replace($escapedPkgId, "[-_.]+", "[-_.]+")
$escapedPkgVersion = [System.Text.RegularExpressions.Regex]::Escape($pkgVersion)
$artifactPattern = "(?i)(?:^|/)$escapedPkgId-$escapedPkgVersion(?:[-_.].*?)?\.(?:tar\.gz|zip|whl)(?:#|$)"
$response = Invoke-WebRequest -Uri $packageIndexUrl -UseBasicParsing
foreach ($link in $response.Links) {
$href = [string]$link.href
if ($href -match $artifactPattern) {
Write-Host "Package $pkgId==$pkgVersion was found on the package index."
return $True
}
}
Write-Host "Package $pkgId==$pkgVersion was not found on the package index (not yet published)."
return $False
}
catch
{
$statusCode = $null
if ($_.Exception.Response -and $_.Exception.Response.StatusCode) {
$statusCode = [int]$_.Exception.Response.StatusCode
}
if ($statusCode -eq 404) {
Write-Host "Package $pkgId==$pkgVersion was not found on the package index (package not present on simple index)."
return $False
}
# Any other failure is unexpected — fail hard to avoid accidentally re-publishing.
Write-Host "Package version check failed unexpectedly:"
Write-Host $_
exit(1)
}
| Write-Host "PIP_INDEX_URL is not set; pip will fall back to public PyPI." | ||
| } | ||
|
|
||
| $pipOutput = pip $pipArgs 2>&1 |
There was a problem hiding this comment.
pip is invoked directly here, but elsewhere in these eng scripts the pattern is python -m pip ... (and sometimes uv pip ...). Calling pip can fail if the pip shim isn’t on PATH or points to a different Python environment than python. Consider using python -m pip download ... (or uv pip download when uv is available) for consistency and to ensure the correct interpreter/index configuration is used.
| $pipOutput = pip $pipArgs 2>&1 | |
| $pipOutput = & python -m pip @pipArgs 2>&1 |
| # Any other failure is unexpected — fail hard to avoid accidentally re-publishing. | ||
| Write-Host "Package version check failed unexpectedly:" | ||
| Write-Host $outputStr | ||
| exit(1) |
There was a problem hiding this comment.
Using exit(1) inside this helper function terminates the entire script immediately, which makes the function harder to reuse/test and bypasses any upstream error handling patterns. Prefer throwing a terminating error (or returning an error value and letting the caller decide) so failures can be handled consistently by the surrounding script logic.
| exit(1) | |
| throw "Package version check failed unexpectedly for ${pkgId}==${pkgVersion}:`n$outputStr" |
Still facing failureThe issue was that we were exiting with the last $LASTEXITCODE. This was causing failure because the success case is not being able to resolve the dep. Instead we were treating that as the error case accidentally.Proven successful