The following table gives results for the Matrix Vector Multiply function with a wide variety of supported parameters, which are defined in: Matrix Vector Multiply Configuration Parameters.
matrix_vector_mul_benchmark.csv
| Library Element | AIE_VARIANT | TT_DATA_A | TT_DATA_B | TP_DIM_A | TP_DIM_B | TP_DIM_A_LEADING | TP_NUM_FRAMES | TP_CASC_LEN | TP_SSR | Dynamic Power (W) | Latency (ns) | Throughput (MSa/s) | NUM_BANKS | NUM_AIE | DATA_MEMORY | PROGRAM_MEMORY |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| matrix_vector_mul | AIE | cfloat | cfloat | 16 | 32 | 1 | 1 | 1 | 1 | 0.964 | 1375 | 625 | 7 | 1 | 11176 | 2542 |
| matrix_vector_mul | AIE | cfloat | cfloat | 16 | 32 | 1 | 4 | 1 | 1 | 0.994 | 5243 | 624 | 9 | 1 | 38056 | 2626 |
| matrix_vector_mul | AIE | cfloat | cfloat | 32 | 32 | 1 | 1 | 2 | 1 | 1.084 | 1402 | 1250 | 12 | 2 | 21839 | 2416 2516 |
| matrix_vector_mul | AIE | cfloat | cfloat | 32 | 32 | 1 | 1 | 4 | 1 | 1.209 | 820 | 2497 | 19 | 4 | 26269 | 1682 1682 1682 1778 |
| matrix_vector_mul | AIE | cfloat | cfloat | 4 | 512 | 1 | 1 | 1 | 1 | 0.998 | 5005 | 624 | 9 | 1 | 43240 | 1586 |
| matrix_vector_mul | AIE | cfloat | cfloat | 512 | 4 | 1 | 1 | 1 | 1 | 0.999 | 4496 | 624 | 9 | 1 | 43240 | 1698 |
| matrix_vector_mul | AIE | cfloat | cfloat | 64 | 16 | 1 | 1 | 2 | 4 | 1.611 | 445 | 4995 | 45 | 8 | 36156 | 1714 1778 1714 1778 1714 1778 1714 1778 |
| matrix_vector_mul | AIE | cfloat | float | 16 | 32 | 1 | 1 | 1 | 1 | 0.952 | 1128 | 625 | 7 | 1 | 10920 | 2646 |
| matrix_vector_mul | AIE | cfloat | float | 64 | 16 | 1 | 1 | 2 | 4 | 1.586 | 373 | 5019 | 44 | 8 | 35644 | 1570 1682 1570 1682 1570 1682 1570 1682 |
| matrix_vector_mul | AIE | cint16 | cint16 | 16 | 32 | 1 | 1 | 1 | 1 | 0.948 | 566 | 1248 | 7 | 1 | 6697 | 2394 |
| matrix_vector_mul | AIE | cint16 | cint16 | 16 | 32 | 1 | 4 | 1 | 1 | 0.956 | 2000 | 1250 | 7 | 1 | 20137 | 2472 |
| matrix_vector_mul | AIE | cint16 | cint16 | 32 | 32 | 1 | 1 | 4 | 2 | 1.488 | 269 | 10039 | 39 | 8 | 26684 | 1490 1522 1522 1602 1490 1522 1522 1602 |
| matrix_vector_mul | AIE | cint16 | cint16 | 32 | 64 | 1 | 1 | 1 | 1 | 0.954 | 1968 | 1250 | 7 | 1 | 19369 | 1602 |
| matrix_vector_mul | AIE | cint16 | cint16 | 32 | 64 | 1 | 1 | 2 | 1 | 1.062 | 1064 | 2500 | 11 | 2 | 21584 | 2302 2356 |
| matrix_vector_mul | AIE | cint16 | cint16 | 32 | 64 | 1 | 1 | 4 | 1 | 1.194 | 667 | 4995 | 19 | 4 | 26014 | 1872 1872 1872 1956 |
| matrix_vector_mul | AIE | cint16 | cint16 | 48 | 16 | 0 | 1 | 1 | 3 | 1.273 | 352 | 3692 | 20 | 3 | 13563 | 1948 1948 1948 |
| matrix_vector_mul | AIE | cint16 | cint16 | 512 | 8 | 1 | 1 | 1 | 1 | 0.984 | 3778 | 1249 | 9 | 1 | 39145 | 1682 |
| matrix_vector_mul | AIE | cint16 | cint16 | 64 | 16 | 1 | 1 | 2 | 4 | 1.554 | 236 | 9941 | 44 | 8 | 26944 | 1490 1602 1490 1602 1490 1602 1490 1602 |
| matrix_vector_mul | AIE | cint16 | cint16 | 8 | 512 | 1 | 1 | 1 | 1 | 0.984 | 3778 | 1249 | 9 | 1 | 39145 | 1554 |
| matrix_vector_mul | AIE | cint16 | cint32 | 16 | 32 | 1 | 1 | 1 | 1 | 0.934 | 632 | 1248 | 7 | 1 | 7082 | 3022 |
| matrix_vector_mul | AIE | cint16 | int16 | 16 | 32 | 0 | 1 | 1 | 1 | 0.978 | 572 | 1248 | 7 | 1 | 6569 | 2452 |
| matrix_vector_mul | AIE | cint16 | int16 | 16 | 32 | 1 | 1 | 1 | 1 | 0.928 | 569 | 1248 | 7 | 1 | 6569 | 2452 |
| matrix_vector_mul | AIE | cint16 | int32 | 16 | 32 | 1 | 1 | 1 | 1 | 0.247 | 622 | 1248 | 7 | 1 | 6825 | 2470 |
| matrix_vector_mul | AIE | cint32 | cint16 | 16 | 32 | 1 | 1 | 1 | 1 | 0.97 | 1616 | 625 | 7 | 1 | 10922 | 2494 |
| matrix_vector_mul | AIE | cint32 | cint32 | 16 | 32 | 1 | 1 | 1 | 1 | 0.964 | 1343 | 625 | 7 | 1 | 11178 | 1816 |
| matrix_vector_mul | AIE | cint32 | cint32 | 16 | 32 | 1 | 4 | 1 | 1 | 0.994 | 5094 | 624 | 9 | 1 | 38058 | 1852 |
| matrix_vector_mul | AIE | cint32 | cint32 | 32 | 32 | 1 | 1 | 2 | 1 | 1.044 | 1488 | 1250 | 12 | 2 | 21842 | 2596 2768 |
| matrix_vector_mul | AIE | cint32 | cint32 | 32 | 32 | 1 | 1 | 4 | 1 | 1.286 | 923 | 2497 | 19 | 4 | 26274 | 2066 2214 2214 2178 |
| matrix_vector_mul | AIE | cint32 | cint32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.555 | 601 | 4995 | 41 | 8 | 35652 | 2050 2214 2214 2178 2050 2214 2214 2178 |
| matrix_vector_mul | AIE | cint32 | cint32 | 4 | 512 | 1 | 1 | 1 | 1 | 0.996 | 5019 | 624 | 9 | 1 | 43242 | 1780 |
| matrix_vector_mul | AIE | cint32 | cint32 | 48 | 16 | 1 | 1 | 1 | 3 | 1.25 | 736 | 1873 | 20 | 3 | 20478 | 2678 2678 2678 |
| matrix_vector_mul | AIE | cint32 | cint32 | 512 | 4 | 1 | 1 | 1 | 1 | 0.984 | 5126 | 624 | 9 | 1 | 43242 | 1808 |
| matrix_vector_mul | AIE | cint32 | cint32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.634 | 499 | 4995 | 45 | 8 | 36168 | 2050 2178 2050 2178 2050 2178 2050 2178 |
| matrix_vector_mul | AIE | cint32 | int32 | 16 | 32 | 1 | 1 | 1 | 1 | 0.953 | 1026 | 625 | 7 | 1 | 10921 | 2370 |
| matrix_vector_mul | AIE | cint32 | int32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.515 | 397 | 4995 | 41 | 8 | 35132 | 1538 1442 1442 1666 1538 1442 1442 1666 |
| matrix_vector_mul | AIE | cint32 | int32 | 48 | 16 | 1 | 1 | 1 | 3 | 1.211 | 582 | 1873 | 20 | 3 | 20091 | 1940 1940 1940 |
| matrix_vector_mul | AIE | cint32 | int32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.583 | 356 | 5019 | 44 | 8 | 35648 | 1538 1666 1538 1666 1538 1666 1538 1666 |
| matrix_vector_mul | AIE | float | cfloat | 16 | 32 | 1 | 1 | 1 | 1 | 0.955 | 1048 | 1199 | 7 | 1 | 7081 | 1744 |
| matrix_vector_mul | AIE | float | cfloat | 64 | 16 | 1 | 1 | 2 | 4 | 1.574 | 506 | 6872 | 45 | 8 | 27976 | 2136 2208 2136 2208 2136 2208 2136 2208 |
| matrix_vector_mul | AIE | float | float | 16 | 32 | 0 | 1 | 1 | 1 | 0.978 | 617 | 1248 | 7 | 1 | 6696 | 2622 |
| matrix_vector_mul | AIE | float | float | 16 | 32 | 1 | 1 | 1 | 1 | 0.948 | 614 | 1248 | 7 | 1 | 6696 | 2622 |
| matrix_vector_mul | AIE | float | float | 16 | 32 | 1 | 4 | 1 | 1 | 0.955 | 2197 | 1250 | 7 | 1 | 20136 | 2662 |
| matrix_vector_mul | AIE | float | float | 32 | 64 | 1 | 1 | 1 | 1 | 0.954 | 2167 | 1250 | 7 | 1 | 19368 | 1634 |
| matrix_vector_mul | AIE | float | float | 32 | 64 | 1 | 1 | 2 | 1 | 1.03 | 1202 | 2500 | 11 | 2 | 21583 | 2576 2692 |
| matrix_vector_mul | AIE | float | float | 32 | 64 | 1 | 1 | 4 | 1 | 1.251 | 702 | 4995 | 19 | 4 | 26013 | 2002 2016 2016 2086 |
| matrix_vector_mul | AIE | float | float | 512 | 8 | 1 | 1 | 1 | 1 | 0.981 | 4034 | 1249 | 9 | 1 | 39144 | 1714 |
| matrix_vector_mul | AIE | float | float | 64 | 16 | 1 | 1 | 2 | 4 | 1.551 | 249 | 9941 | 44 | 8 | 26940 | 1522 1602 1522 1602 1522 1602 1522 1602 |
| matrix_vector_mul | AIE | float | float | 8 | 512 | 1 | 1 | 1 | 1 | 0.981 | 4189 | 1249 | 9 | 1 | 39144 | 1618 |
| matrix_vector_mul | AIE | int16 | cint16 | 16 | 32 | 1 | 1 | 1 | 1 | 0.953 | 371 | 2497 | 7 | 1 | 4650 | 2052 |
| matrix_vector_mul | AIE | int16 | int16 | 16 | 32 | 1 | 1 | 1 | 1 | 0.941 | 363 | 2497 | 7 | 1 | 4457 | 2344 |
| matrix_vector_mul | AIE | int16 | int16 | 16 | 32 | 1 | 4 | 1 | 1 | 0.952 | 1180 | 2500 | 7 | 1 | 11177 | 2378 |
| matrix_vector_mul | AIE | int16 | int16 | 16 | 512 | 1 | 1 | 1 | 1 | 0.98 | 4203 | 2499 | 9 | 1 | 37097 | 2344 |
| matrix_vector_mul | AIE | int16 | int16 | 512 | 16 | 1 | 1 | 1 | 1 | 0.985 | 4237 | 2499 | 9 | 1 | 37097 | 2768 |
| matrix_vector_mul | AIE | int16 | int16 | 64 | 128 | 1 | 1 | 1 | 1 | 0.979 | 4253 | 2499 | 9 | 1 | 35753 | 2400 |
| matrix_vector_mul | AIE | int16 | int16 | 64 | 128 | 1 | 1 | 2 | 1 | 1.041 | 2311 | 5001 | 11 | 2 | 37968 | 2322 2400 |
| matrix_vector_mul | AIE | int16 | int16 | 64 | 128 | 1 | 1 | 4 | 1 | 1.268 | 1365 | 10002 | 21 | 4 | 42398 | 2328 2312 2312 2406 |
| matrix_vector_mul | AIE | int16 | int32 | 16 | 32 | 1 | 1 | 1 | 1 | 0.951 | 362 | 2497 | 7 | 1 | 4648 | 2406 |
| matrix_vector_mul | AIE | int32 | cint32 | 16 | 32 | 1 | 1 | 1 | 1 | 0.962 | 632 | 1248 | 7 | 1 | 7082 | 3022 |
| matrix_vector_mul | AIE | int32 | cint32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.507 | 326 | 9941 | 39 | 8 | 27460 | 1844 1852 1852 1946 1844 1852 1852 1946 |
| matrix_vector_mul | AIE | int32 | cint32 | 48 | 16 | 1 | 1 | 1 | 3 | 1.235 | 386 | 3746 | 19 | 3 | 14334 | 2302 2302 2302 |
| matrix_vector_mul | AIE | int32 | cint32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.577 | 276 | 10039 | 45 | 8 | 27976 | 1844 1946 1844 1946 1844 1946 1844 1946 |
| matrix_vector_mul | AIE | int32 | int16 | 16 | 32 | 0 | 1 | 1 | 1 | 0.976 | 644 | 1248 | 7 | 1 | 6569 | 2398 |
| matrix_vector_mul | AIE | int32 | int16 | 16 | 32 | 1 | 1 | 1 | 1 | 0.947 | 641 | 1248 | 7 | 1 | 6569 | 2398 |
| matrix_vector_mul | AIE | int32 | int32 | 16 | 32 | 1 | 1 | 1 | 1 | 0.948 | 566 | 1248 | 7 | 1 | 6697 | 2394 |
| matrix_vector_mul | AIE | int32 | int32 | 16 | 32 | 1 | 4 | 1 | 1 | 0.956 | 2000 | 1250 | 7 | 1 | 20137 | 2472 |
| matrix_vector_mul | AIE | int32 | int32 | 32 | 32 | 0 | 1 | 4 | 2 | 1.525 | 273 | 9570 | 39 | 8 | 26684 | 1490 1522 1522 1602 1490 1522 1522 1602 |
| matrix_vector_mul | AIE | int32 | int32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.49 | 269 | 10039 | 39 | 8 | 26684 | 1490 1522 1522 1602 1490 1522 1522 1602 |
| matrix_vector_mul | AIE | int32 | int32 | 32 | 64 | 1 | 1 | 1 | 1 | 0.954 | 1968 | 1250 | 7 | 1 | 19369 | 1602 |
| matrix_vector_mul | AIE | int32 | int32 | 32 | 64 | 1 | 1 | 2 | 1 | 1.062 | 1064 | 2500 | 11 | 2 | 21584 | 2302 2356 |
| matrix_vector_mul | AIE | int32 | int32 | 32 | 64 | 1 | 1 | 4 | 1 | 1.249 | 667 | 4995 | 19 | 4 | 26014 | 1872 1872 1872 1956 |
| matrix_vector_mul | AIE | int32 | int32 | 48 | 16 | 1 | 1 | 1 | 3 | 1.195 | 349 | 3746 | 20 | 3 | 13563 | 1948 1948 1948 |
| matrix_vector_mul | AIE | int32 | int32 | 512 | 8 | 1 | 1 | 1 | 1 | 0.984 | 3778 | 1249 | 9 | 1 | 39145 | 1682 |
| matrix_vector_mul | AIE | int32 | int32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.554 | 236 | 9941 | 44 | 8 | 26944 | 1490 1602 1490 1602 1490 1602 1490 1602 |
| matrix_vector_mul | AIE | int32 | int32 | 8 | 512 | 1 | 1 | 1 | 1 | 0.984 | 3778 | 1249 | 9 | 1 | 39145 | 1554 |
| matrix_vector_mul | AIE-ML | cfloat | cfloat | 32 | 32 | 1 | 1 | 1 | 1 | 0.551 | 27171 | 106 | 6 | 1 | 19854 | 2432 |
| matrix_vector_mul | AIE-ML | cfloat | cfloat | 48 | 16 | 1 | 8 | 1 | 3 | 1.104 | 54908 | 317 | 25 | 3 | 117930 | 2464 2464 2464 |
| matrix_vector_mul | AIE-ML | cfloat | cfloat | 64 | 16 | 1 | 1 | 1 | 4 | 1.175 | 7036 | 416 | 27 | 4 | 28216 | 2432 2432 2432 2432 |
| matrix_vector_mul | AIE-ML | cint16 | cint16 | 32 | 32 | 0 | 1 | 1 | 1 | 0.48 | 1189 | 1242 | 7 | 1 | 10898 | 1616 |
| matrix_vector_mul | AIE-ML | cint16 | cint16 | 32 | 32 | 0 | 1 | 4 | 2 | 1.565 | 366 | 9481 | 42 | 8 | 26454 | 1424 1440 1440 1536 1424 1440 1440 1536 |
| matrix_vector_mul | AIE-ML | cint16 | cint16 | 32 | 32 | 1 | 1 | 1 | 1 | 0.48 | 1184 | 1250 | 7 | 1 | 10898 | 1616 |
| matrix_vector_mul | AIE-ML | cint16 | cint16 | 32 | 32 | 1 | 1 | 4 | 2 | 1.566 | 361 | 9941 | 42 | 8 | 26454 | 1424 1440 1440 1536 1424 1440 1440 1536 |
| matrix_vector_mul | AIE-ML | cint16 | cint16 | 48 | 16 | 1 | 8 | 1 | 3 | 0.897 | 2387 | 3750 | 21 | 3 | 61878 | 1872 1872 1872 |
| matrix_vector_mul | AIE-ML | cint16 | cint16 | 64 | 16 | 1 | 1 | 2 | 4 | 1.574 | 284 | 9941 | 46 | 8 | 26740 | 1424 1536 1424 1536 1424 1536 1424 1536 |
| matrix_vector_mul | AIE-ML | cint32 | cint32 | 32 | 32 | 1 | 1 | 1 | 1 | 0.501 | 2875 | 625 | 6 | 1 | 19602 | 1632 |
| matrix_vector_mul | AIE-ML | cint32 | cint32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.626 | 492 | 5019 | 42 | 8 | 35414 | 1632 1600 1600 1680 1632 1600 1600 1680 |
| matrix_vector_mul | AIE-ML | cint32 | cint32 | 48 | 16 | 1 | 8 | 1 | 3 | 0.962 | 5870 | 1874 | 25 | 3 | 117174 | 1664 1664 1664 |
| matrix_vector_mul | AIE-ML | cint32 | cint32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.624 | 444 | 5019 | 46 | 8 | 35956 | 1632 1680 1632 1680 1632 1680 1632 1680 |
| matrix_vector_mul | AIE-ML | cint32 | int32 | 32 | 32 | 1 | 1 | 1 | 1 | 0.535 | 3122 | 625 | 6 | 1 | 19347 | 2448 |
| matrix_vector_mul | AIE-ML | cint32 | int32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.694 | 1013 | 3436 | 42 | 8 | 34910 | 1728 1712 1712 1808 1728 1712 1712 1808 |
| matrix_vector_mul | AIE-ML | cint32 | int32 | 48 | 16 | 1 | 8 | 1 | 3 | 1.076 | 6456 | 1874 | 27 | 3 | 114105 | 2096 2096 2096 |
| matrix_vector_mul | AIE-ML | cint32 | int32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.702 | 902 | 3436 | 46 | 8 | 35452 | 1728 1808 1728 1808 1728 1808 1728 1808 |
| matrix_vector_mul | AIE-ML | float | float | 32 | 32 | 0 | 1 | 1 | 1 | 0.539 | 5800 | 469 | 6 | 1 | 11150 | 2272 |
| matrix_vector_mul | AIE-ML | float | float | 48 | 16 | 1 | 8 | 1 | 3 | 1.078 | 11887 | 1370 | 21 | 3 | 62634 | 2336 2336 2336 |
| matrix_vector_mul | AIE-ML | float | float | 64 | 16 | 0 | 1 | 2 | 4 | 1.689 | 1851 | 1939 | 44 | 8 | 28772 | 2608 2688 2608 2688 2608 2688 2608 2688 |
| matrix_vector_mul | AIE-ML | int32 | cint32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.63 | 701 | 7111 | 42 | 8 | 27246 | 1632 1616 1616 1728 1632 1616 1616 1728 |
| matrix_vector_mul | AIE-ML | int32 | cint32 | 48 | 16 | 1 | 8 | 1 | 3 | 1.041 | 3340 | 3750 | 20 | 3 | 68031 | 1824 1824 1824 |
| matrix_vector_mul | AIE-ML | int32 | cint32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.636 | 576 | 6282 | 39 | 8 | 27788 | 1632 1728 1632 1728 1632 1728 1632 1728 |
| matrix_vector_mul | AIE-ML | int32 | int32 | 32 | 32 | 0 | 1 | 1 | 1 | 0.521 | 1557 | 1242 | 7 | 1 | 10898 | 2032 |
| matrix_vector_mul | AIE-ML | int32 | int32 | 32 | 32 | 1 | 1 | 1 | 1 | 0.521 | 1552 | 1250 | 7 | 1 | 10898 | 2032 |
| matrix_vector_mul | AIE-ML | int32 | int32 | 32 | 32 | 1 | 1 | 4 | 2 | 1.622 | 792 | 5657 | 42 | 8 | 26454 | 1712 1712 1712 1808 1712 1712 1712 1808 |
| matrix_vector_mul | AIE-ML | int32 | int32 | 48 | 16 | 1 | 8 | 1 | 3 | 1.044 | 3213 | 3750 | 21 | 3 | 61878 | 2048 2048 2048 |
| matrix_vector_mul | AIE-ML | int32 | int32 | 64 | 16 | 1 | 1 | 2 | 4 | 1.636 | 675 | 5657 | 46 | 8 | 26740 | 1712 1808 1712 1808 1712 1808 1712 1808 |