feat: add builder execution path with AppendChild step#7677
feat: add builder execution path with AppendChild step#7677joseph-isaacs wants to merge 5 commits intodevelopfrom
Conversation
Merging this PR will degrade performance by 24.82%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | chunked_opt_bool_into_canonical[(10, 1000)] |
1 ms | 1.3 ms | -24.82% |
| ❌ | Simulation | chunked_opt_bool_into_canonical[(100, 100)] |
220 µs | 256.2 µs | -14.14% |
| ❌ | Simulation | chunked_varbinview_opt_into_canonical[(10, 1000)] |
2.5 ms | 2.8 ms | -12.21% |
| ❌ | Simulation | chunked_varbinview_into_canonical[(10, 1000)] |
1.7 ms | 2 ms | -15.53% |
| ⚡ | Simulation | alp_rd_decompress_f64 |
2.4 ms | 1.1 ms | ×2.2 |
| ⚡ | Simulation | decompress[alp_for_bp_f64] |
2.9 ms | 1.8 ms | +58.54% |
| ⚡ | Simulation | patched_take_10k_first_chunk_only |
301.9 µs | 271.4 µs | +11.24% |
| ⚡ | Simulation | patched_take_10k_adversarial |
258.4 µs | 228.1 µs | +13.29% |
| ⚡ | Simulation | take_10k_first_chunk_only |
270.4 µs | 225.4 µs | +19.97% |
| ⚡ | Simulation | take_10k_dispersed |
284.4 µs | 239.3 µs | +18.85% |
| ⚡ | Simulation | patched_take_10k_dispersed |
315.5 µs | 284.9 µs | +10.73% |
| ⚡ | Simulation | decompress_rd[f32, (10000, 0.0)] |
166.2 µs | 85.4 µs | +94.59% |
| ⚡ | Simulation | decompress_rd[f64, (10000, 0.0)] |
258.2 µs | 121.9 µs | ×2.1 |
| ⚡ | Simulation | decompress_rd[f64, (10000, 0.1)] |
258.7 µs | 121.8 µs | ×2.1 |
| ⚡ | Simulation | decompress_rd[f64, (100000, 0.0)] |
2,294.7 µs | 978.6 µs | ×2.3 |
| ⚡ | Simulation | decompress_rd[f32, (100000, 0.0)] |
1,287 µs | 495.5 µs | ×2.6 |
| ⚡ | Simulation | decompress_rd[f64, (100000, 0.01)] |
2.3 ms | 1 ms | ×2.3 |
| ⚡ | Simulation | decompress_rd[f64, (10000, 0.01)] |
258.6 µs | 121.7 µs | ×2.1 |
| ⚡ | Simulation | decompress_rd[f32, (10000, 0.1)] |
165.9 µs | 81.7 µs | ×2 |
| ⚡ | Simulation | decompress_rd[f64, (100000, 0.1)] |
2.3 ms | 1 ms | ×2.3 |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Comparing ji/append-to-builder-iter (178f4a2) with develop (b13ba9c)
Footnotes
-
33 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
ddfb8fb to
2bdb707
Compare
2bdb707 to
b70fa7e
Compare
Benchmarks: CompressionVortex (geomean): 0.999x ➖ unknown / unknown (0.999x ➖, 6↑ 4↓)
|
Benchmarks: Random AccessVortex (geomean): 0.968x ➖ unknown / unknown (0.986x ➖, 3↑ 0↓)
|
Benchmarks: TPC-H SF=10 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.985x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.984x ➖, 0↑ 0↓)
datafusion / parquet (0.995x ➖, 0↑ 0↓)
datafusion / arrow (0.972x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed (0.987x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.995x ➖, 0↑ 0↓)
duckdb / parquet (0.991x ➖, 0↑ 0↓)
duckdb / duckdb (0.992x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.055x ➖, 0↑ 2↓)
datafusion / vortex-compact (0.942x ➖, 2↑ 0↓)
datafusion / parquet (0.955x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed (1.005x ➖, 0↑ 1↓)
duckdb / vortex-compact (0.966x ➖, 1↑ 1↓)
duckdb / parquet (0.963x ➖, 1↑ 0↓)
Full attributed analysis
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) duckdb / vortex-file-compressed (0.994x ➖, 0↑ 1↓)
duckdb / vortex-compact (0.993x ➖, 0↑ 0↓)
duckdb / parquet (0.997x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.946x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.943x ➖, 1↑ 0↓)
datafusion / parquet (0.959x ➖, 1↑ 0↓)
datafusion / arrow (0.938x ➖, 3↑ 0↓)
duckdb / vortex-file-compressed (0.950x ➖, 3↑ 0↓)
duckdb / vortex-compact (0.942x ➖, 2↑ 0↓)
duckdb / parquet (0.972x ➖, 1↑ 0↓)
duckdb / duckdb (0.928x ➖, 5↑ 0↓)
Full attributed analysis
|
Benchmarks: PolarSignals ProfilingVortex (geomean): 1.095x ➖ datafusion / vortex-file-compressed (1.095x ➖, 0↑ 4↓)
|
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.124x ❌, 0↑ 66↓)
datafusion / vortex-compact (1.100x ➖, 0↑ 49↓)
datafusion / parquet (1.115x ❌, 0↑ 53↓)
duckdb / vortex-file-compressed (1.096x ➖, 0↑ 46↓)
duckdb / vortex-compact (1.075x ➖, 0↑ 26↓)
duckdb / parquet (1.074x ➖, 0↑ 25↓)
duckdb / duckdb (1.102x ❌, 0↑ 40↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.862x ➖, 3↑ 1↓)
datafusion / vortex-compact (0.896x ➖, 2↑ 0↓)
datafusion / parquet (1.117x ➖, 0↑ 5↓)
duckdb / vortex-file-compressed (0.944x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.972x ➖, 0↑ 0↓)
duckdb / parquet (0.938x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.940x ➖, 1↑ 0↓)
datafusion / vortex-compact (0.975x ➖, 0↑ 0↓)
datafusion / parquet (0.977x ➖, 1↑ 2↓)
duckdb / vortex-file-compressed (1.041x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.053x ➖, 0↑ 0↓)
duckdb / parquet (1.042x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb S3Verdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.960x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.953x ➖, 0↑ 0↓)
datafusion / parquet (1.019x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.897x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.898x ➖, 1↑ 0↓)
duckdb / parquet (0.999x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.994x ➖, 1↑ 0↓)
datafusion / parquet (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.962x ➖, 8↑ 0↓)
duckdb / parquet (0.998x ➖, 1↑ 0↓)
duckdb / duckdb (0.972x ➖, 5↑ 0↓)
Full attributed analysis
|
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
File Sizes: PolarSignals ProfilingNo file size changes detected. |
File Sizes: FineWeb NVMeNo file size changes detected. |
File Sizes: TPC-H SF=1 on NVMENo file size changes detected. |
File Sizes: TPC-DS SF=1 on NVMENo file size changes detected. |
File Sizes: TPC-H SF=10 on NVMENo file size changes detected. |
File Sizes: Statistical and Population GeneticsNo file size changes detected. |
File Sizes: Clickbench on NVMEFile Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Adds
ExecutionStep::AppendChildandexecute_into_builderto supportiterative execution that appends child arrays directly into builders,
avoiding intermediate materialization for chunked arrays.