Conversation
Signed-off-by: Mikhail Kot <to@myrrc.dev>
There was a problem hiding this comment.
The rationale from prev discussions was that our perf should not rely on LTO. We can't make assumption about other crates that import Vortex whether LTO is enabled. Story would be diff if Vortex would be the executable top-level binary. Should we leave perf on the table due to missing cross crate opt, we should scan our APIs for missing inline annotations which also benefit clients that don't use LTO.
Merging this PR will degrade performance by 15.94%
Performance Changes
Comparing Footnotes
|
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals ProfilingVortex (geomean): 0.996x ➖ datafusion / vortex-file-compressed (0.996x ➖, 0↑ 0↓)
|
File Sizes: PolarSignals ProfilingNo file size changes detected. |
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.927x ➖, 4↑ 1↓)
datafusion / vortex-compact (0.904x ➖, 2↑ 0↓)
datafusion / parquet (0.927x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed (0.984x ➖, 0↑ 1↓)
duckdb / vortex-compact (0.949x ➖, 1↑ 0↓)
duckdb / parquet (0.933x ➖, 2↑ 0↓)
Full attributed analysis
|
File Sizes: FineWeb NVMeNo file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.959x ➖, 5↑ 1↓)
datafusion / vortex-compact (0.942x ➖, 12↑ 0↓)
datafusion / parquet (0.967x ➖, 3↑ 0↓)
duckdb / vortex-file-compressed (0.956x ➖, 11↑ 1↓)
duckdb / vortex-compact (0.966x ➖, 6↑ 1↓)
duckdb / parquet (0.992x ➖, 1↑ 2↓)
duckdb / duckdb (0.981x ➖, 6↑ 5↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.992x ➖, 0↑ 1↓)
datafusion / vortex-compact (0.964x ➖, 0↑ 0↓)
datafusion / parquet (1.031x ➖, 0↑ 2↓)
datafusion / arrow (1.052x ➖, 0↑ 6↓)
duckdb / vortex-file-compressed (0.984x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.985x ➖, 0↑ 0↓)
duckdb / parquet (1.014x ➖, 1↑ 2↓)
duckdb / duckdb (0.989x ➖, 0↑ 0↓)
Full attributed analysis
|
File Sizes: TPC-DS SF=1 on NVMENo file size changes detected. |
File Sizes: TPC-H SF=1 on NVMENo file size changes detected. |
Benchmarks: FineWeb S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.031x ➖, 0↑ 0↓)
datafusion / vortex-compact (1.045x ➖, 0↑ 0↓)
datafusion / parquet (0.960x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.067x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.075x ➖, 0↑ 0↓)
duckdb / parquet (1.038x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) duckdb / vortex-file-compressed (1.013x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.021x ➖, 0↑ 0↓)
duckdb / parquet (1.011x ➖, 0↑ 0↓)
Full attributed analysis
|
File Sizes: Statistical and Population GeneticsNo file size changes detected. |
Benchmarks: TPC-H SF=10 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.014x ➖, 0↑ 1↓)
datafusion / vortex-compact (1.016x ➖, 0↑ 1↓)
datafusion / parquet (1.006x ➖, 0↑ 0↓)
datafusion / arrow (1.090x ➖, 0↑ 8↓)
duckdb / vortex-file-compressed (0.994x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.116x ❌, 0↑ 16↓)
duckdb / parquet (1.002x ➖, 1↑ 0↓)
duckdb / duckdb (1.101x ❌, 0↑ 10↓)
Full attributed analysis
|
File Sizes: TPC-H SF=10 on NVMENo file size changes detected. |
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.986x ➖, 0↑ 2↓)
datafusion / vortex-compact (0.886x ➖, 2↑ 0↓)
datafusion / parquet (0.956x ➖, 2↑ 1↓)
duckdb / vortex-file-compressed (1.032x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.021x ➖, 0↑ 0↓)
duckdb / parquet (1.003x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.041x ➖, 0↑ 7↓)
datafusion / parquet (1.039x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (0.976x ➖, 7↑ 0↓)
duckdb / parquet (1.019x ➖, 0↑ 2↓)
duckdb / duckdb (1.006x ➖, 0↑ 1↓)
Full attributed analysis
|
File Sizes: Clickbench on NVMEFile Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.069x ➖, 0↑ 3↓)
datafusion / vortex-compact (1.033x ➖, 0↑ 1↓)
datafusion / parquet (0.910x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed (0.988x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.034x ➖, 0↑ 1↓)
duckdb / parquet (1.038x ➖, 0↑ 1↓)
Full attributed analysis
|
Benchmarks: CompressionVortex (geomean): 0.989x ➖ unknown / unknown (0.993x ➖, 3↑ 4↓)
|
Benchmarks: Random AccessVortex (geomean): 0.907x ➖ unknown / unknown (0.942x ➖, 9↑ 0↓)
|
No description provided.