Draft
Conversation
- Complete type annotations for profiling.tracing and profiling.sampling - Replace all Incomplete usages with proper types throughout - Add return types, parameter types, and attribute types to all stubs - Only exception: unwinder: Incomplete in sample.pyi (_remote_debugging C ext) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
Member
Author
|
Going through a monkey patched version of Details❯ uv run dead --files 'typeshed/stdlib/profiling|cpython/Lib/test/test_profiling' --tests test_profiling | rg never
dump_stats is never read, defined in typeshed/stdlib/profiling/tracing/__init__.pyi:21
snapshot_stats is never read, defined in typeshed/stdlib/profiling/tracing/__init__.pyi:23
runcall is never read, defined in typeshed/stdlib/profiling/tracing/__init__.pyi:26
_Utils is never read, defined in typeshed/stdlib/profiling/tracing/_utils.pyi:4
MIN_SAMPLES_FOR_TUI is never read, defined in typeshed/stdlib/profiling/sampling/sample.pyi:20
MACOS_PERMISSION_ERROR is never read, defined in typeshed/stdlib/profiling/sampling/__main__.pyi:8
LINUX_PERMISSION_ERROR is never read, defined in typeshed/stdlib/profiling/sampling/__main__.pyi:9
WINDOWS_PERMISSION_ERROR is never read, defined in typeshed/stdlib/profiling/sampling/__main__.pyi:10
GENERIC_PERMISSION_ERROR is never read, defined in typeshed/stdlib/profiling/sampling/__main__.pyi:11
handle_permission_error is never read, defined in typeshed/stdlib/profiling/sampling/__main__.pyi:13
base_opcode is never read, defined in typeshed/stdlib/profiling/sampling/opcode_utils.pyi:1
MILLISECONDS_PER_SECOND is never read, defined in typeshed/stdlib/profiling/sampling/constants.pyi:2
_INTERNAL_FRAME_SUFFIXES is never read, defined in typeshed/stdlib/profiling/sampling/constants.pyi:15
_TemplateLoader is never read, defined in typeshed/stdlib/profiling/sampling/heatmap_collector.pyi:34
_TreeBuilder is never read, defined in typeshed/stdlib/profiling/sampling/heatmap_collector.pyi:44
build_file_tree is never read, defined in typeshed/stdlib/profiling/sampling/heatmap_collector.pyi:46
_HtmlRenderer is never read, defined in typeshed/stdlib/profiling/sampling/heatmap_collector.pyi:48
render_hierarchical_html is never read, defined in typeshed/stdlib/profiling/sampling/heatmap_collector.pyi:52
GECKO_CATEGORIES is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:6
CATEGORY_OTHER is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:7
CATEGORY_PYTHON is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:8
CATEGORY_NATIVE is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:9
CATEGORY_GC is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:10
CATEGORY_GIL is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:11
CATEGORY_CPU is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:12
CATEGORY_CODE_TYPE is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:13
CATEGORY_OPCODES is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:14
CATEGORY_EXCEPTION is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:15
DEFAULT_SUBCATEGORY is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:16
GECKO_FORMAT_VERSION is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:17
GECKO_PREPROCESSED_VERSION is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:18
RESOURCE_TYPE_LIBRARY is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:19
FRAME_ADDRESS_NONE is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:20
FRAME_INLINE_DEPTH_ROOT is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:21
PROCESS_TYPE_MAIN is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:22
STACKWALK_DISABLED is never read, defined in typeshed/stdlib/profiling/sampling/gecko_collector.pyi:23
get_stats is never read, defined in typeshed/stdlib/profiling/sampling/binary_collector.pyi:23, typeshed/stdlib/profiling/sampling/binary_reader.pyi:23
convert_binary_to_format is never read, defined in typeshed/stdlib/profiling/sampling/binary_reader.pyi:25
ArgumentError is never read, defined in typeshed/stdlib/profiling/sampling/_sync_coordinator.pyi:4
SyncError is never read, defined in typeshed/stdlib/profiling/sampling/_sync_coordinator.pyi:5
TargetError is never read, defined in typeshed/stdlib/profiling/sampling/_sync_coordinator.pyi:6
COMPRESSION_NONE is never read, defined in typeshed/stdlib/profiling/sampling/binary_collector.pyi:8
COMPRESSION_ZSTD is never read, defined in typeshed/stdlib/profiling/sampling/binary_collector.pyi:9
collect_failed_sample is never read, defined in typeshed/stdlib/profiling/sampling/binary_collector.pyi:19, typeshed/stdlib/profiling/sampling/collector.pyi:22, typeshed/stdlib/profiling/sampling/live_collector/collector.pyi:126
intern is never read, defined in typeshed/stdlib/profiling/sampling/string_table.pyi:3
get_string is never read, defined in typeshed/stdlib/profiling/sampling/string_table.pyi:4
get_strings is never read, defined in typeshed/stdlib/profiling/sampling/string_table.pyi:5
CustomFormatter is never read, defined in typeshed/stdlib/profiling/sampling/cli.pyi:38
DiffFlamegraphAction is never read, defined in typeshed/stdlib/profiling/sampling/cli.pyi:40
FORMAT_EXTENSIONS is never read, defined in typeshed/stdlib/profiling/sampling/cli.pyi:49
COLLECTOR_MAP is never read, defined in typeshed/stdlib/profiling/sampling/cli.pyi:50
get_dimensions is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:32, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:56, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:7
refresh is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:11, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:34, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:58
redraw is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:13, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:35, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:59
get_input is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:17, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:37, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:61
set_nodelay is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:19, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:38, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:62
has_colors is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:21, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:39, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:63
init_color_pair is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:23, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:40, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:64
get_color_pair is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:25, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:41, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:65
get_attr is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:27, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:42, typeshed/stdlib/profiling/sampling/live_collector/display.pyi:66
find_text is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/display.pyi:70
set_enabled is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/trend_tracker.pyi:12
get_trend is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/trend_tracker.pyi:14
format_rate_with_units is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/widgets.pyi:57
draw_thread_status is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/widgets.pyi:60
draw_finished_banner is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/widgets.pyi:63
draw_stats_rows is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/widgets.pyi:70
render_filter_input_prompt is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/widgets.pyi:76
increment_status_flag is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/collector.pyi:62
as_status_dict is never read, defined in typeshed/stdlib/profiling/sampling/live_collector/collector.pyi:63Idea is: if it's not referenced in tests, it's a signal that you probably won't need this (there is some false negatives so I'll do this by hand). |
Removed: - tracing/__main__.pyi: deleted — __main__ stubs are executable entry points, not importable API surface - tracing/_utils.pyi: deleted — _Utils is a private helper used only inside the tracing module itself - sampling/__main__.pyi: deleted — PERMISSION_ERROR constants and handle_permission_error are CLI implementation details, not importable API - sampling/binary_collector.pyi: removed COMPRESSION_NONE, COMPRESSION_ZSTD — internal constants not part of user-facing API - sampling/cli.pyi: removed CustomFormatter, DiffFlamegraphAction, FORMAT_EXTENSIONS, COLLECTOR_MAP, and associated imports — internal CLI scaffolding and dispatch dicts used only inside cli.py; also removed re-exports of errors/collectors/sample that duplicate their own modules - sampling/constants.pyi: removed MICROSECONDS_PER_SECOND (re-exported only internally), MILLISECONDS_PER_SECOND and _INTERNAL_FRAME_SUFFIXES (private implementation details) - sampling/gecko_collector.pyi: removed GECKO_CATEGORIES, CATEGORY_*, DEFAULT_SUBCATEGORY, GECKO_FORMAT_VERSION, GECKO_PREPROCESSED_VERSION, RESOURCE_TYPE_LIBRARY, FRAME_ADDRESS_NONE, FRAME_INLINE_DEPTH_ROOT, PROCESS_TYPE_MAIN, STACKWALK_DISABLED — integer constants that encode the internal Gecko profile format; not part of user-facing API - sampling/heatmap_collector.pyi: removed _TemplateLoader, _TreeBuilder, _HtmlRenderer — private classes (underscore-prefixed) used only internally for HTML rendering - sampling/opcode_utils.pyi: removed base_opcode, variant_opcode — these are local variables inside get_opcode_info, not module-level attributes; the stubs were incorrect - sampling/sample.pyi: removed MIN_SAMPLES_FOR_TUI — internal threshold constant for TUI display logic - sampling/_sync_coordinator.pyi: removed CoordinatorError, ArgumentError, SyncError, TargetError — all exceptions live in a private module (_sync_coordinator) and are only raised and caught internally; never exposed through any public module Kept despite `dead` flagging: - tracing/__init__.pyi: dump_stats, snapshot_stats, runcall — public methods on the exported Profile class; standard profiling API - sampling/binary_collector.pyi: get_stats, collect_failed_sample — public methods; collect_failed_sample is part of the Collector interface called in sample.py - sampling/binary_reader.pyi: get_stats, convert_binary_to_format — public method and public utility function - sampling/string_table.pyi: intern, get_string, get_strings — core API of StringTable - sampling/live_collector/display.pyi: get_dimensions, refresh, redraw, get_input, set_nodelay, has_colors, init_color_pair, get_color_pair, get_attr, find_text — abstract methods on DisplayInterface define the ABC contract; concrete implementations on CursesDisplay and MockDisplay must also be stubbed - sampling/live_collector/trend_tracker.pyi: set_enabled, get_trend — public methods on public class - sampling/live_collector/widgets.pyi: format_rate_with_units, draw_thread_status, draw_finished_banner, draw_stats_rows, render_filter_input_prompt — public methods on public widget classes - sampling/live_collector/collector.pyi: increment_status_flag, as_status_dict, collect_failed_sample — public methods on ThreadData dataclass and LiveStatsCollector Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Judgement calls — removed: - sampling/_css_utils.pyi: deleted — private module (_css_utils) used only internally by stack_collector and heatmap_collector; get_combined_css is not part of any public API - sampling/_format_utils.pyi: deleted — private module (_format_utils) used only internally; fmt is a simple formatting helper not intended for external use - sampling/_sync_coordinator.pyi: deleted — private module only containing main() -> Never, which is a subprocess entry point never called externally - sampling/_child_monitor.pyi: removed get_child_pids and is_python_process — private utility functions used only inside _child_monitor itself; ChildProcessMonitor kept because it is re-exported from cli.pyi - sampling/heatmap_collector.pyi: removed re-exports of get_combined_css and fmt — these are private implementation imports, not part of HeatmapCollector's public API - sampling/stack_collector.pyi: removed re-export of get_combined_css — same reason as above - sampling/sample.pyi: removed re-export of fmt — private formatting helper leaked into a public module's stub Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
live_collector/__init__.pyi re-exported ~30 TUI layout/color constants (MICROSECONDS_PER_SECOND, MIN_TERMINAL_WIDTH, WIDTH_THRESHOLD_*, COLOR_PAIR_*, COL_WIDTH_*, etc.) in __all__. Nothing outside the live_collector package ever imports them — only LiveStatsCollector and MockDisplay are actually used by callers in the source and tests. Judgement calls — removed: - live_collector/__init__.pyi: stripped all constant imports and __all__ entries down to the 10 public classes (LiveStatsCollector, DisplayInterface, CursesDisplay, MockDisplay, and the Widget subclasses) — the constants are internal TUI dimensions and curses color pair IDs, not user-facing API - live_collector/constants.pyi: removed DISPLAY_UPDATE_HZ, COL_WIDTH_NSAMPLES, COL_SPACING, COL_WIDTH_SAMPLE_PCT, COL_WIDTH_TIME, MIN_AVAILABLE_SPACE — these became unreferenced once the __init__ re-exports were dropped; they are not imported by any other stub in the package Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
Guiding principle: stubs should only contain what users of the profiling API would actually import or tab-complete. The live_collector subpackage is primarily a TUI display component; its internal widget/display architecture is not part of any user-facing API. Judgement calls — removed: - live_collector/widgets.pyi: deleted — Widget, ProgressBarWidget, HeaderWidget, TableWidget, FooterWidget, HelpWidget, OpcodePanel are internal TUI rendering components; nobody writing code against the profiling API would import them - live_collector/trend_tracker.pyi: deleted — TrendTracker is an internal widget state helper used only inside the TUI rendering loop - live_collector/constants.pyi: deleted — TUI layout dimensions and curses color pair IDs (COLOR_PAIR_*, WIDTH_THRESHOLD_*, MIN_TERMINAL_*, etc.) have no meaning outside the live_collector implementation - live_collector/display.pyi: removed CursesDisplay (internal curses wrapper) and MockDisplay (test helper only referenced in tests, not production API); kept DisplayInterface because it is the type of the display= parameter on LiveStatsCollector.__init__ - live_collector/collector.pyi: removed all constant imports (COLOR_PAIR_*, MICROSECONDS_PER_SECOND, MIN_TERMINAL_*, etc.) that were leaked implementation imports not used in any type annotations; removed widget imports (HeaderWidget, TableWidget, etc.) and TrendTracker; removed ThreadData dataclass (internal per-sample accumulator); stripped LiveStatsCollector of TUI-state attributes (stdscr, selected_row, scroll_offset, paused, show_help, filter_*, view_mode, current_thread_index, per_thread_data, *_widget, display_update_interval_sec, thread_status_counts, gc_frame_samples, opcode_stats) and internal methods (simplify_path, process_frames, init_curses, cleanup_curses) - live_collector/__init__.pyi: stripped __all__ and re-exports to just LiveStatsCollector and DisplayInterface Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
It was removed in the first cleanup commit as "re-exported only internally", but pstats_collector.pyi legitimately imports it from sampling.constants — where it is actually defined in the source. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Judgement calls, per the principle: will people want to import these symbols in their code that uses the profiling API? profiling.tracing ----------------- - Remove Profile.snapshot_stats(): not in __all__, not in docs. It is an internal implementation step called by create_stats(); users call create_stats() directly. - Remove module-level main(): not in __all__, not in docs. It is the CLI entry point wired up by the (already-deleted) __main__.py stub. - Keep dump_stats, runcall, create_stats, print_stats: all explicitly documented in the profiling.tracing module reference. - Keep private _T, _P, _Label aliases: needed internally by the stub to express the runcall() overload and stats dict type. profiling.sampling — collector submodule stubs ----------------------------------------------- - Remove normalize_location, extract_lineno, filter_internal_frames from collector.pyi: implementation helpers used inside collector subclasses; not in sampling.__all__, not documented. Users subclass Collector with collect() and export() only. - Remove DEFAULT_LOCATION and THREAD_STATUS_* re-exports from collector.pyi: leaked from constants.py; import from profiling.sampling.constants if needed. - Remove get_opcode_mapping from opcode_utils.pyi: internal mapping used to build opcode-name look-up tables; not documented, never imported by user code. - Remove re-exports of filter_internal_frames, format_opcode, get_opcode_info from gecko_collector.pyi: imported for internal use, not part of GeckoCollector's public contract. - Strip GeckoCollector internal tracking attributes (has_gil_start, no_gil_start, on_cpu_start, etc.): 20+ state dicts tracking GIL/CPU marker transitions; purely implementation detail. Users call __init__ → collect → export. - Remove FileStats, TreeNode, get_python_path_info, extract_module_name from heatmap_collector.pyi: internal data structures and path helpers for the HTML generation pipeline; not in sampling.__all__, not documented. - Strip HeatmapCollector internal state counters (line_samples, file_samples, call_graph, etc.): implementation detail of the line-level sample accumulation. - Remove re-exports of extract_lineno, normalize_location, format_opcode, get_opcode_info from heatmap_collector.pyi: same pattern — imported for internal use only. - Remove re-exports of extract_lineno, get_opcode_mapping, StringTable from stack_collector.pyi. - Strip CollapsedStackCollector.stack_counter and FlamegraphCollector internal state (thread_status_counts, per_thread_stats, etc.): sample accumulation internals. - Strip DiffFlamegraphCollector.baseline_binary_path attribute: constructor parameter, not a user-facing attribute. - Remove extract_lineno, MICROSECONDS_PER_SECOND, PROFILING_MODE_CPU re-exports from pstats_collector.pyi. - Strip PstatsCollector internal state (result, stats, callers, skip_idle attributes): implementation detail of sample aggregation. - Remove GeckoCollector, PstatsCollector, CollapsedStackCollector, FlamegraphCollector re-exports from binary_reader.pyi: imported internally by binary_reader.py, not part of BinaryReader's API. profiling.sampling — sample and cli modules ------------------------------------------- - Strip sample.pyi of all re-exports (BinaryCollector, Collector, all five PROFILING_MODE_* constants, GeckoCollector, HeatmapCollector, LiveStatsCollector, PstatsCollector, CollapsedStackCollector, FlamegraphCollector): sample.py imports these for internal use; they are not part of sample's public interface and are already accessible from their own modules. - Remove unwinder: Incomplete from SampleProfiler: internal C extension handle, not user-accessible. - Strip cli.pyi to main() only: cli.py is the CLI implementation; all the symbols it re-exported (ChildProcessMonitor, BinaryCollector, BinaryReader, 6 SORT_MODE constants) belong to their own modules. - Delete _child_monitor.pyi: private module (_-prefixed); previously kept because cli.pyi re-exported ChildProcessMonitor, but now that cli.pyi is stripped there is no public re-export, and users have no reason to import from profiling.sampling._child_monitor directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
1 similar comment
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: mypy (https://github.com/python/mypy)
.../projects/mypy/mypy/__init__.py: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Beta freeze is about to come, so here's a work-in-progress on
profiling.From here, I'd like to remove all private interfaces that aren't meant for public consumption and thus don't need to be in the typeshed.