Skip to content

Exclude procedural-string-lights from cached execution ci benchmarks#3351

Merged
TrueDoctor merged 1 commit intomasterfrom
exclude-string-lights-from-benchmarks
Nov 8, 2025
Merged

Exclude procedural-string-lights from cached execution ci benchmarks#3351
TrueDoctor merged 1 commit intomasterfrom
exclude-string-lights-from-benchmarks

Conversation

@TrueDoctor
Copy link
Copy Markdown
Member

@TrueDoctor TrueDoctor commented Nov 8, 2025 •
edited
Loading

This benchmark seem to have been really flaky in our CI so we remove it

@TrueDoctor
Exclude procedural-string-lights from cached execution benchmarks
9094233 
This benchmark seem to have been really flaky in our CI so we remove it
@TrueDoctor TrueDoctor force-pushed the exclude-string-lights-from-benchmarks branch from 3577002 to 9094233 Compare November 8, 2025 09:00
@github-actions github-actions Bot temporarily deployed to graphite-dev (Preview) November 8, 2025 09:05 Inactive
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Nov 8, 2025

Performance Benchmark Results

🔧 Graph Compilation

compile_demo_art_iai::compile_group::compile_to_proto with_setup_0:load_from_name(isometric-fountain)
Instructions: 27,518,858 (master) → 27,520,068 (HEAD) : $$\color{red}+0.00\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     4|          4          -0.03%
D1mr                     350,163|    350,036          -0.04%
D1mw                     108,915|    108,906          -0.01%
DLmr                      31,096|     31,096          +0.00%
DLmw                      51,731|     51,734          +0.01%
Dr                     6,807,675|  6,807,765          +0.00%
Dw                     4,740,483|  4,740,421          -0.00%
EstimatedCycles       43,657,764| 43,658,548          +0.00%
I1MissRate                     0|          0          -0.00%
I1mr                      60,904|     60,904          +0.00%
ILmr                         867|        867          +0.00%
Ir                    27,518,858| 27,520,068          +0.00%
L1HitRate                     99|         99          +0.00%
L1hits                38,547,034| 38,548,408          +0.00%
LLHitRate                      1|          1          -0.04%
LLMissRate                     0|          0          +0.00%
LLdMissRate                    1|          1          +0.00%
LLhits                   436,288|    436,149          -0.03%
LLiMissRate                    0|          0          -0.00%
RamHitRate                     0|          0          +0.00%
RamHits                   83,694|     83,697          +0.00%
TotalRW               39,067,016| 39,068,254          +0.00%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_1:load_from_name(painted-dreams)
Instructions: 13,902,038 (master) → 13,904,786 (HEAD) : $$\color{red}+0.02\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     4|          4          -0.06%
D1mr                     169,291|    169,230          -0.04%
D1mw                      53,583|     53,535          -0.09%
DLmr                         671|        670          -0.15%
DLmw                      12,623|     12,615          -0.06%
Dr                     3,443,779|  3,444,148          +0.01%
Dw                     2,387,000|  2,387,054          +0.00%
EstimatedCycles       21,165,449| 21,167,914          +0.01%
I1MissRate                     0|          0          -0.02%
I1mr                      30,269|     30,269          +0.00%
ILmr                         708|        708          +0.00%
Ir                    13,902,038| 13,904,786          +0.02%
L1HitRate                     99|         99          +0.00%
L1hits                19,479,674| 19,482,954          +0.02%
LLHitRate                      1|          1          -0.06%
LLMissRate                     0|          0          -0.08%
LLdMissRate                    0|          0          -0.07%
LLhits                   239,141|    239,041          -0.04%
LLiMissRate                    0|          0          -0.02%
RamHitRate                     0|          0          -0.08%
RamHits                   14,002|     13,993          -0.06%
TotalRW               19,732,817| 19,735,988          +0.02%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_2:load_from_name(procedural-string-lights)
Instructions: 3,090,133 (master) → 3,089,522 (HEAD) : $$\color{lime}-0.02\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     4|          4          -0.00%
D1mr                      36,446|     36,442          -0.01%
D1mw                      10,665|     10,666          +0.01%
DLmr                          13|         13          +0.00%
DLmw                       2,921|      2,923          +0.07%
Dr                       753,593|    753,538          -0.01%
Dw                       523,947|    523,957          +0.00%
EstimatedCycles        4,689,437|  4,688,829          -0.01%
I1MissRate                     0|          0          +0.02%
I1mr                       6,180|      6,180          +0.00%
ILmr                         686|        686          +0.00%
Ir                     3,090,133|  3,089,522          -0.02%
L1HitRate                     99|         99          -0.00%
L1hits                 4,314,382|  4,313,729          -0.02%
LLHitRate                      1|          1          +0.00%
LLMissRate                     0|          0          +0.07%
LLdMissRate                    0|          0          +0.07%
LLhits                    49,671|     49,666          -0.01%
LLiMissRate                    0|          0          +0.02%
RamHitRate                     0|          0          +0.07%
RamHits                    3,620|      3,622          +0.06%
TotalRW                4,367,673|  4,367,017          -0.02%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_3:load_from_name(parametric-dunescape)
Instructions: 13,608,592 (master) → 13,606,017 (HEAD) : $$\color{lime}-0.02\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     4|          4          -0.01%
D1mr                     169,911|    169,851          -0.04%
D1mw                      49,349|     49,385          +0.07%
DLmr                          65|         65          +0.00%
DLmw                      10,973|     10,974          +0.01%
Dr                     3,330,656|  3,330,362          -0.01%
Dw                     2,336,006|  2,335,983          -0.00%
EstimatedCycles       20,600,504| 20,597,546          -0.01%
I1MissRate                     0|          0          +0.02%
I1mr                      23,125|     23,125          +0.00%
ILmr                         819|        819          +0.00%
Ir                    13,608,592| 13,606,017          -0.02%
L1HitRate                     99|         99          -0.00%
L1hits                19,032,869| 19,030,001          -0.02%
LLHitRate                      1|          1          +0.00%
LLMissRate                     0|          0          +0.02%
LLdMissRate                    0|          0          +0.01%
LLhits                   230,528|    230,503          -0.01%
LLiMissRate                    0|          0          +0.02%
RamHitRate                     0|          0          +0.02%
RamHits                   11,857|     11,858          +0.01%
TotalRW               19,275,254| 19,272,362          -0.02%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_4:load_from_name(red-dress)
Instructions: 32,165,544 (master) → 32,167,084 (HEAD) : $$\color{red}+0.00\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     4|          4          +0.01%
D1mr                     407,822|    407,885          +0.02%
D1mw                     124,405|    124,385          -0.02%
DLmr                      42,417|     42,417          +0.00%
DLmw                      48,622|     48,595          -0.06%
Dr                     7,919,894|  7,920,007          +0.00%
Dw                     5,518,642|  5,518,628          -0.00%
EstimatedCycles       50,759,142| 50,760,143          +0.00%
I1MissRate                     0|          0          -0.00%
I1mr                      67,281|     67,281          +0.00%
ILmr                         862|        862          +0.00%
Ir                    32,165,544| 32,167,084          +0.00%
L1HitRate                     99|         99          -0.00%
L1hits                45,004,572| 45,006,168          +0.00%
LLHitRate                      1|          1          +0.01%
LLMissRate                     0|          0          -0.03%
LLdMissRate                    1|          1          -0.03%
LLhits                   507,607|    507,677          +0.01%
LLiMissRate                    0|          0          -0.00%
RamHitRate                     0|          0          -0.03%
RamHits                   91,901|     91,874          -0.03%
TotalRW               45,604,080| 45,605,719          +0.00%

compile_demo_art_iai::compile_group::compile_to_proto with_setup_5:load_from_name(valley-of-spires)
Instructions: 21,210,504 (master) → 21,211,050 (HEAD) : $$\color{red}+0.00\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     4|          4          -0.00%
D1mr                     262,244|    262,198          -0.02%
D1mw                      77,560|     77,597          +0.05%
DLmr                      15,215|     15,219          +0.03%
DLmw                      33,402|     33,423          +0.06%
Dr                     5,244,838|  5,244,923          +0.00%
Dw                     3,645,657|  3,645,698          +0.00%
EstimatedCycles       33,124,867| 33,126,253          +0.00%
I1MissRate                     0|          0          -0.00%
I1mr                      45,498|     45,498          +0.00%
ILmr                         805|        805          +0.00%
Ir                    21,210,504| 21,211,050          +0.00%
L1HitRate                     99|         99          +0.00%
L1hits                29,715,697| 29,716,378          +0.00%
LLHitRate                      1|          1          -0.01%
LLMissRate                     0|          0          +0.05%
LLdMissRate                    1|          1          +0.05%
LLhits                   335,880|    335,846          -0.01%
LLiMissRate                    0|          0          -0.00%
RamHitRate                     0|          0          +0.05%
RamHits                   49,422|     49,447          +0.05%
TotalRW               30,100,999| 30,101,671          +0.00%

🔄 Executor Update

update_executor_iai::update_group::update_executor with_setup_0:setup_update_executor(isometric-fountain)
Instructions: 46,763,515 (master) → 46,791,078 (HEAD) : $$\color{red}+0.06\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          +0.47%
D1mr                     505,005|    506,932          +0.38%
D1mw                     114,221|    115,507          +1.13%
DLmr                       3,423|      3,393          -0.88%
DLmw                      16,590|     15,661          -5.60%
Dr                    11,714,538| 11,719,963          +0.05%
Dw                     8,025,066|  8,029,028          +0.05%
EstimatedCycles       69,620,379| 69,641,123          +0.03%
I1MissRate                     0|          0          -0.90%
I1mr                       8,589|      8,517          -0.84%
ILmr                         187|        187          +0.00%
Ir                    46,763,515| 46,791,078          +0.06%
L1HitRate                     99|         99          -0.00%
L1hits                65,875,304| 65,909,113          +0.05%
LLHitRate                      1|          1          +0.62%
LLMissRate                     0|          0          -4.80%
LLdMissRate                    0|          0          -4.84%
LLhits                   607,615|    611,715          +0.67%
LLiMissRate                    0|          0          -0.06%
RamHitRate                     0|          0          -4.80%
RamHits                   20,200|     19,241          -4.75%
TotalRW               66,503,119| 66,540,069          +0.06%

update_executor_iai::update_group::update_executor with_setup_1:setup_update_executor(painted-dreams)
Instructions: 24,210,014 (master) → 24,218,314 (HEAD) : $$\color{red}+0.03\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          -0.05%
D1mr                     253,527|    253,107          -0.17%
D1mw                      56,509|     57,131          +1.10%
DLmr                         997|        996          -0.10%
DLmw                       2,970|      2,720          -8.42%
Dr                     6,064,368|  6,067,386          +0.05%
Dw                     4,138,821|  4,147,763          +0.22%
EstimatedCycles       35,797,623| 35,810,961          +0.04%
I1MissRate                     0|          0          -1.03%
I1mr                       5,019|      4,969          -1.00%
ILmr                         173|        173          +0.00%
Ir                    24,210,014| 24,218,314          +0.03%
L1HitRate                     99|         99          +0.00%
L1hits                34,098,148| 34,118,256          +0.06%
LLHitRate                      1|          1          +0.07%
LLMissRate                     0|          0          -6.12%
LLdMissRate                    0|          0          -6.44%
LLhits                   310,915|    311,318          +0.13%
LLiMissRate                    0|          0          -0.03%
RamHitRate                     0|          0          -6.12%
RamHits                    4,140|      3,889          -6.06%
TotalRW               34,413,203| 34,433,463          +0.06%

update_executor_iai::update_group::update_executor with_setup_2:setup_update_executor(procedural-string-lights)
Instructions: 6,260,771 (master) → 6,272,496 (HEAD) : $$\color{red}+0.19\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          +0.72%
D1mr                      65,794|     66,356          +0.85%
D1mw                      14,902|     15,032          +0.87%
DLmr                         NaN|        NaN          +0.00%
DLmw                         442|        444          +0.45%
Dr                     1,593,283|  1,595,479          +0.14%
Dw                     1,095,845|  1,097,239          +0.13%
EstimatedCycles        9,301,567|  9,319,730          +0.20%
I1MissRate                     0|          0          +0.00%
I1mr                       2,631|      2,636          +0.19%
ILmr                         170|        170          +0.00%
Ir                     6,260,771|  6,272,496          +0.19%
L1HitRate                     99|         99          -0.01%
L1hits                 8,866,572|  8,881,190          +0.16%
LLHitRate                      1|          1          +0.67%
LLMissRate                     0|          0          +0.16%
LLdMissRate                    0|          0          +0.32%
LLhits                    82,715|     83,410          +0.84%
LLiMissRate                    0|          0          -0.19%
RamHitRate                     0|          0          +0.16%
RamHits                      612|        614          +0.33%
TotalRW                8,949,899|  8,965,214          +0.17%

update_executor_iai::update_group::update_executor with_setup_3:setup_update_executor(parametric-dunescape)
Instructions: 28,195,221 (master) → 28,192,206 (HEAD) : $$\color{lime}-0.01\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          -0.06%
D1mr                     278,091|    278,087          -0.00%
D1mw                      71,166|     70,945          -0.31%
DLmr                         194|        198          +2.06%
DLmw                       4,588|      4,302          -6.23%
Dr                     7,166,096|  7,165,710          -0.01%
Dw                     5,002,726|  5,002,770          +0.00%
EstimatedCycles       41,928,807| 41,916,400          -0.03%
I1MissRate                     0|          0          +1.15%
I1mr                       4,824|      4,879          +1.14%
ILmr                         166|        169          +1.81%
Ir                    28,195,221| 28,192,206          -0.01%
L1HitRate                     99|         99          +0.00%
L1hits                40,009,962| 40,006,775          -0.01%
LLHitRate                      1|          1          +0.04%
LLMissRate                     0|          0          -5.63%
LLdMissRate                    0|          0          -5.89%
LLhits                   349,133|    349,242          +0.03%
LLiMissRate                    0|          0          +1.82%
RamHitRate                     0|          0          -5.63%
RamHits                    4,948|      4,669          -5.64%
TotalRW               40,364,043| 40,360,686          -0.01%

update_executor_iai::update_group::update_executor with_setup_4:setup_update_executor(red-dress)
Instructions: 64,034,626 (master) → 63,931,033 (HEAD) : $$\color{lime}-0.16\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          -0.16%
D1mr                     663,620|    656,324          -1.10%
D1mw                     157,765|    156,608          -0.73%
DLmr                      14,586|     15,098          +3.51%
DLmw                      30,024|     32,484          +8.19%
Dr                    16,315,686| 16,190,240          -0.77%
Dw                    11,233,819| 11,118,978          -1.02%
EstimatedCycles       96,254,675| 95,966,443          -0.30%
I1MissRate                     0|          0          +0.00%
I1mr                       9,276|      9,261          -0.16%
ILmr                         320|        332          +3.75%
Ir                    64,034,626| 63,931,033          -0.16%
L1HitRate                     99|         99          +0.01%
L1hits                90,753,470| 90,418,058          -0.37%
LLHitRate                      1|          1          -1.09%
LLMissRate                     0|          0          +7.04%
LLdMissRate                    0|          0          +7.60%
LLhits                   785,731|    774,279          -1.46%
LLiMissRate                    0|          0          +3.92%
RamHitRate                     0|          0          +7.04%
RamHits                   44,930|     47,914          +6.64%
TotalRW               91,584,131| 91,240,251          -0.38%

update_executor_iai::update_group::update_executor with_setup_5:setup_update_executor(valley-of-spires)
Instructions: 35,972,239 (master) → 36,136,943 (HEAD) : $$\color{red}+0.46\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          -0.01%
D1mr                     375,366|    380,781          +1.44%
D1mw                      82,760|     85,879          +3.77%
DLmr                       2,668|      2,647          -0.79%
DLmw                       9,501|     10,671         +12.31%
Dr                     8,987,750|  9,127,198          +1.55%
Dw                     6,113,243|  6,256,360          +2.34%
EstimatedCycles       53,302,736| 53,818,367          +0.97%
I1MissRate                     0|          0          -1.37%
I1mr                       6,640|      6,579          -0.92%
ILmr                         179|        179          +0.00%
Ir                    35,972,239| 36,136,943          +0.46%
L1HitRate                     99|         99          -0.01%
L1hits                50,608,466| 51,047,262          +0.87%
LLHitRate                      1|          1          +0.74%
LLMissRate                     0|          0          +8.36%
LLdMissRate                    0|          0          +7.43%
LLhits                   452,418|    459,742          +1.62%
LLiMissRate                    0|          0          -0.46%
RamHitRate                     0|          0          +8.36%
RamHits                   12,348|     13,497          +9.31%
TotalRW               51,073,232| 51,520,501          +0.88%

🚀 Render: Cold Execution

run_once_iai::run_once_group::run_once with_setup_0:setup_run_once(isometric-fountain)
Instructions: 22,556,198 (master) → 22,611,149 (HEAD) : $$\color{red}+0.24\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          -0.65%
D1mr                     276,676|    278,412          +0.63%
D1mw                      60,349|     61,363          +1.68%
DLmr                       7,942|      8,104          +2.04%
DLmw                      10,196|     10,482          +2.81%
Dr                     5,895,874|  5,967,993          +1.22%
Dw                     3,992,550|  4,066,153          +1.84%
EstimatedCycles       35,026,350| 35,252,069          +0.64%
I1MissRate                     1|          1          -0.11%
I1mr                     132,247|    132,421          +0.13%
ILmr                       5,350|      5,347          -0.06%
Ir                    22,556,198| 22,611,149          +0.24%
L1HitRate                     99|         99          -0.00%
L1hits                31,975,350| 32,173,099          +0.62%
LLHitRate                      1|          1          -0.06%
LLMissRate                     0|          0          +1.27%
LLdMissRate                    0|          0          +0.98%
LLhits                   445,784|    448,263          +0.56%
LLiMissRate                    0|          0          -0.30%
RamHitRate                     0|          0          +1.27%
RamHits                   23,488|     23,933          +1.89%
TotalRW               32,444,622| 32,645,295          +0.62%

run_once_iai::run_once_group::run_once with_setup_1:setup_run_once(painted-dreams)
Instructions: 79,614,744 (master) → 79,736,695 (HEAD) : $$\color{red}+0.15\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     2|          2          +0.31%
D1mr                     414,198|    415,371          +0.28%
D1mw                     195,906|    196,957          +0.54%
DLmr                       3,803|      3,830          +0.71%
DLmw                      18,962|     21,347         +12.58%
Dr                    19,120,102| 19,136,116          +0.08%
Dw                    12,727,002| 12,729,133          +0.02%
EstimatedCycles      115,540,426|115,761,652          +0.19%
I1MissRate                     0|          0          -0.18%
I1mr                     183,153|    183,099          -0.03%
ILmr                       7,420|      7,423          +0.04%
Ir                    79,614,744| 79,736,695          +0.15%
L1HitRate                     99|         99          -0.00%
L1hits               110,668,591|110,806,517          +0.12%
LLHitRate                      1|          1          -0.16%
LLMissRate                     0|          0          +7.87%
LLdMissRate                    0|          0         +10.53%
LLhits                   763,072|    762,827          -0.03%
LLiMissRate                    0|          0          -0.11%
RamHitRate                     0|          0          +7.87%
RamHits                   30,185|     32,600          +8.00%
TotalRW              111,461,848|111,601,944          +0.13%

run_once_iai::run_once_group::run_once with_setup_2:setup_run_once(procedural-string-lights)
Instructions: 10,491,911 (master) → 10,500,195 (HEAD) : $$\color{red}+0.08\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     2|          2          +1.92%
D1mr                      58,532|     59,660          +1.93%
D1mw                      23,397|     23,210          -0.80%
DLmr                         476|        478          +0.42%
DLmw                       1,019|      1,018          -0.10%
Dr                     2,615,407|  2,600,407          -0.57%
Dw                     1,891,631|  1,872,431          -1.01%
EstimatedCycles       15,698,151| 15,675,573          -0.14%
I1MissRate                     0|          0          -0.32%
I1mr                      48,239|     48,125          -0.24%
ILmr                       4,456|      4,456          +0.00%
Ir                    10,491,911| 10,500,195          +0.08%
L1HitRate                     99|         99          -0.01%
L1hits                14,868,781| 14,842,038          -0.18%
LLHitRate                      1|          1          +0.84%
LLMissRate                     0|          0          +0.19%
LLdMissRate                    0|          0          +0.83%
LLhits                   124,217|    125,043          +0.66%
LLiMissRate                    0|          0          -0.08%
RamHitRate                     0|          0          +0.19%
RamHits                    5,951|      5,952          +0.02%
TotalRW               14,998,949| 14,973,033          -0.17%

run_once_iai::run_once_group::run_once with_setup_3:setup_run_once(parametric-dunescape)
Instructions: 23,435,661 (master) → 23,499,567 (HEAD) : $$\color{red}+0.27\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     2|          2          +0.74%
D1mr                     159,905|    161,305          +0.88%
D1mw                      63,144|     64,526          +2.19%
DLmr                       2,261|      2,276          +0.66%
DLmw                       5,184|      5,081          -1.99%
Dr                     5,687,177|  5,712,316          +0.44%
Dw                     3,788,626|  3,811,386          +0.60%
EstimatedCycles       34,421,722| 34,542,209          +0.35%
I1MissRate                     0|          0          -0.21%
I1mr                      70,628|     70,669          +0.06%
ILmr                       3,740|      3,741          +0.03%
Ir                    23,435,661| 23,499,567          +0.27%
L1HitRate                     99|         99          -0.01%
L1hits                32,617,787| 32,726,769          +0.33%
LLHitRate                      1|          1          +0.69%
LLMissRate                     0|          0          -1.11%
LLdMissRate                    0|          0          -1.68%
LLhits                   282,492|    285,402          +1.03%
LLiMissRate                    0|          0          -0.25%
RamHitRate                     0|          0          -1.11%
RamHits                   11,185|     11,098          -0.78%
TotalRW               32,911,464| 33,023,269          +0.34%

run_once_iai::run_once_group::run_once with_setup_4:setup_run_once(red-dress)
Instructions: 1,796,902,337 (master) → 1,797,051,970 (HEAD) : $$\color{red}+0.01\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     0|          0          +0.26%
D1mr                   1,908,943|  1,918,475          +0.50%
D1mw                     892,915|    891,017          -0.21%
DLmr                     459,441|    467,407          +1.73%
DLmw                     529,419|    529,642          +0.04%
Dr                   433,397,224|433,451,764          +0.01%
Dw                   279,966,821|280,015,362          +0.02%
EstimatedCycles      2,556,255,028|2,556,785,484          +0.02%
I1MissRate                     0|          0          -0.00%
I1mr                   1,238,271|  1,238,370          +0.01%
ILmr                       5,411|      5,449          +0.70%
Ir                   1,796,902,337|1,797,051,970          +0.01%
L1HitRate                    100|        100          -0.00%
L1hits               2,506,226,253|2,506,471,234          +0.01%
LLHitRate                      0|          0          -0.03%
LLMissRate                     0|          0          +0.82%
LLdMissRate                    0|          0          +0.81%
LLhits                 3,045,858|  3,045,364          -0.02%
LLiMissRate                    0|          0          +0.69%
RamHitRate                     0|          0          +0.82%
RamHits                  994,271|  1,002,498          +0.83%
TotalRW              2,510,266,382|2,510,519,096          +0.01%

run_once_iai::run_once_group::run_once with_setup_5:setup_run_once(valley-of-spires)
Instructions: 20,541,668 (master) → 20,531,426 (HEAD) : $$\color{lime}-0.05\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          3          -1.20%
D1mr                     222,811|    221,822          -0.44%
D1mw                      54,216|     54,134          -0.15%
DLmr                       5,653|      5,667          +0.25%
DLmw                       6,636|      6,952          +4.76%
Dr                     5,177,297|  5,210,285          +0.64%
Dw                     3,484,669|  3,522,572          +1.09%
EstimatedCycles       31,218,366| 31,283,869          +0.21%
I1MissRate                     1|          1          -0.12%
I1mr                     107,931|    107,748          -0.17%
ILmr                       3,541|      3,540          -0.03%
Ir                    20,541,668| 20,531,426          -0.05%
L1HitRate                     99|         99          +0.01%
L1hits                28,818,676| 28,880,579          +0.21%
LLHitRate                      1|          1          -0.64%
LLMissRate                     0|          0          +1.87%
LLdMissRate                    0|          0          +1.85%
LLhits                   369,128|    367,545          -0.43%
LLiMissRate                    0|          0          +0.02%
RamHitRate                     0|          0          +1.87%
RamHits                   15,830|     16,159          +2.08%
TotalRW               29,203,634| 29,264,283          +0.21%

⚡ Render: Cached Execution

run_cached_iai::run_cached_group::run_cached with_setup_0:setup_run_cached(isometric-fountain)
Instructions: 7,932,251 (master) → 7,898,761 (HEAD) : $$\color{lime}-0.42\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     5|          5          +1.82%
D1mr                     195,972|    196,658          +0.35%
D1mw                       7,595|      7,300          -3.88%
DLmr                       1,024|        962          -6.05%
DLmw                         155|        325        +109.68%
Dr                     2,416,394|  2,385,381          -1.28%
Dw                     1,436,709|  1,406,103          -2.13%
EstimatedCycles       12,642,742| 12,552,573          -0.71%
I1MissRate                     0|          0          +0.99%
I1mr                         715|        719          +0.56%
ILmr                         163|        167          +2.45%
Ir                     7,932,251|  7,898,761          -0.42%
L1HitRate                     98|         98          -0.02%
L1hits                11,581,072| 11,485,568          -0.82%
LLHitRate                      2|          2          +0.95%
LLMissRate                     0|          0          +9.23%
LLdMissRate                    0|          0         +10.93%
LLhits                   202,940|    203,223          +0.14%
LLiMissRate                    0|          0          +2.89%
RamHitRate                     0|          0          +9.23%
RamHits                    1,342|      1,454          +8.35%
TotalRW               11,785,354| 11,690,245          -0.81%

run_cached_iai::run_cached_group::run_cached with_setup_1:setup_run_cached(painted-dreams)
Instructions: 5,951,813 (master) → 5,951,012 (HEAD) : $$\color{lime}-0.01\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     5|          5          -0.71%
D1mr                     128,798|    127,831          -0.75%
D1mw                       7,206|      7,155          -0.71%
DLmr                         313|        339          +8.31%
DLmw                          47|         44          -6.38%
Dr                     1,852,625|  1,852,044          -0.03%
Dw                     1,110,039|  1,109,433          -0.05%
EstimatedCycles        9,476,915|  9,471,503          -0.06%
I1MissRate                     0|          0          -0.40%
I1mr                         728|        725          -0.41%
ILmr                         157|        156          -0.64%
Ir                     5,951,813|  5,951,012          -0.01%
L1HitRate                     98|         98          +0.01%
L1hits                 8,777,745|  8,776,778          -0.01%
LLHitRate                      2|          2          -0.74%
LLMissRate                     0|          0          +4.28%
LLdMissRate                    0|          0          +6.43%
LLhits                   136,215|    135,172          -0.77%
LLiMissRate                    0|          0          -0.62%
RamHitRate                     0|          0          +4.28%
RamHits                      517|        539          +4.26%
TotalRW                8,914,477|  8,912,489          -0.02%

run_cached_iai::run_cached_group::run_cached with_setup_2:setup_run_cached(parametric-dunescape)
Instructions: 1,502,209 (master) → 3,754,744 (HEAD) : $$\color{red}+149.95\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     3|          5         +37.55%
D1mr                      26,502|     93,399        +252.42%
D1mw                       3,647|      7,723        +111.76%
DLmr                          12|         51        +325.00%
DLmw                           4|         47       +1075.00%
Dr                       524,947|  1,277,996        +143.45%
Dw                       343,639|    839,951        +144.43%
EstimatedCycles        2,499,067|  6,287,089        +151.58%
I1MissRate                     0|          0         -61.88%
I1mr                         719|        685          -4.73%
ILmr                         144|        141          -2.08%
Ir                     1,502,209|  3,754,744        +149.95%
L1HitRate                     99|         98          -0.44%
L1hits                 2,339,927|  5,770,884        +146.63%
LLHitRate                      1|          2         +33.52%
LLMissRate                     0|          0         -39.70%
LLdMissRate                    0|          0        +151.19%
LLhits                    30,708|    101,568        +230.75%
LLiMissRate                    0|          0         -60.83%
RamHitRate                     0|          0         -39.70%
RamHits                      160|        239         +49.38%
TotalRW                2,370,795|  5,872,691        +147.71%

run_cached_iai::run_cached_group::run_cached with_setup_3:setup_run_cached(red-dress)
Instructions: 3,754,124 (master) → 35,577,087 (HEAD) : $$\color{red}+847.68\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     5|          5          +4.66%
D1mr                      93,685|    683,108        +629.15%
D1mw                       7,699|    101,011       +1212.00%
DLmr                          45|    301,944     +670886.67%
DLmw                          55|     58,931     +107047.27%
Dr                     1,277,796| 10,144,596        +693.91%
Dw                       839,742|  5,503,208        +555.35%
EstimatedCycles        6,287,354| 65,212,475        +937.20%
I1MissRate                     0|          0         -88.44%
I1mr                         709|        777          +9.59%
ILmr                         144|        725        +403.47%
Ir                     3,754,124| 35,577,087        +847.68%
L1HitRate                     98|         98          +0.21%
L1hits                 5,769,569| 50,439,995        +774.24%
LLHitRate                      2|          1         -52.36%
LLMissRate                     0|          1      +16887.07%
LLdMissRate                    0|          2      +48735.38%
LLhits                   101,849|    423,296        +315.61%
LLiMissRate                    0|          0         -46.87%
RamHitRate                     0|          1      +16887.07%
RamHits                      244|    361,600     +148096.72%
TotalRW                5,871,662| 51,224,891        +772.41%

run_cached_iai::run_cached_group::run_cached with_setup_4:setup_run_cached(valley-of-spires)
Instructions: 35,574,784 (master) → 6,306,498 (HEAD) : $$\color{lime}-82.27\%$$

Detailed metrics 
Baselines:                master|       HEAD
D1MissRate                     5|          5          +7.88%
D1mr                     683,254|    155,121         -77.30%
D1mw                     101,193|      6,374         -93.70%
DLmr                     301,364|        241         -99.92%
DLmw                      57,354|        159         -99.72%
Dr                    10,143,822|  1,884,549         -81.42%
Dw                     5,502,760|  1,101,362         -79.99%
EstimatedCycles       65,145,844|  9,957,883         -84.71%
I1MissRate                     0|          0        +417.51%
I1mr                         775|        711          -8.26%
ILmr                         735|        155         -78.91%
Ir                    35,574,784|  6,306,498         -82.27%
L1HitRate                     98|         98          -0.22%
L1hits                50,436,144|  9,130,203         -81.90%
LLHitRate                      1|          2        +109.28%
LLMissRate                     1|          0         -99.15%
LLdMissRate                    2|          0         -99.42%
LLhits                   425,769|    161,651         -62.03%
LLiMissRate                    0|          0         +18.96%
RamHitRate                     1|          0         -99.15%
RamHits                  359,453|        555         -99.85%
TotalRW               51,221,366|  9,292,409         -81.86%

@TrueDoctor TrueDoctor merged commit 7254c47 into master Nov 8, 2025
3 of 4 checks passed
@TrueDoctor TrueDoctor deleted the exclude-string-lights-from-benchmarks branch November 8, 2025 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@TrueDoctor