Cascaded Kernels - 2024.1 English

When using the TP_CASC_LEN parameter, the matrix multiply operation is split across TP_DIM_AB and processed in a TP_CASC_LEN number of kernels. The accumulated partial results of each kernel are passed down the cascade port to the next kernel in the cascade chain until the final kernel provides the expected output. Cascade connections are made internally to the matrix multiply graph and external interfaces to the graph remain unchanged.

Each AI Engine kernel in the array is given a sub-matrix, so the interface to the graph is an array of ports for both A and B. The sub matrices are obtained by dividing the input matrices in their common dimension TP_DIM_AB.

Table 57 Input Matrix A (16x16 - 4x4 Tile - Cascade Length 2)

	Cascade 0								Cascade 1
	Tile Col 0				Tile Col 1				Tile Col 2				Tile Col 3
Tile Row 0	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31
	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47
	48	49	50	51	52	53	54	55	56	57	58	59	60	61	62	63
Tile Row 1	64	65	66	67	68	69	70	71	72	73	74	75	76	77	78	79
	80	81	82	83	84	85	86	87	88	89	90	91	92	93	94	95
	96	97	98	99	100	101	102	103	104	105	106	107	108	109	110	111
	112	113	114	115	116	117	118	119	120	121	122	123	124	125	126	127
Tile Row 2	128	129	130	131	132	133	134	135	136	137	138	139	140	141	142	143
	144	145	146	147	148	149	150	151	152	153	154	155	156	157	158	159
	160	161	162	163	164	165	166	167	168	169	170	171	172	173	174	175
	176	177	178	179	180	181	182	183	184	185	186	187	188	189	190	191
Tile Row 3	192	193	194	195	196	197	198	199	200	201	202	203	204	205	206	207
	208	209	210	211	212	213	214	215	216	217	218	219	220	221	222	223
	224	225	226	227	228	229	230	231	232	233	234	235	236	237	238	239
	240	241	242	243	244	245	246	247	248	249	250	251	252	253	254	255

Table 58 Input Matrix B (16x16 - 4x2 Tile - Cascade Length 2)

		Tile Col 0		Tile Col 1		Tile Col 2		Tile Col 3		Tile Col 4		Tile Col 5		Tile Col 6		Tile Col 7
Cascade 0	Tile Row 0	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
		16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31
		32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47
		48	49	50	51	52	53	54	55	56	57	58	59	60	61	62	63
	Tile Row 1	64	65	66	67	68	69	70	71	72	73	74	75	76	77	78	79
		80	81	82	83	84	85	86	87	88	89	90	91	92	93	94	95
		96	97	98	99	100	101	102	103	104	105	106	107	108	109	110	111
		112	113	114	115	116	117	118	119	120	121	122	123	124	125	126	127
Cascade 1	Tile Row 2	128	129	130	131	132	133	134	135	136	137	138	139	140	141	142	143
		144	145	146	147	148	149	150	151	152	153	154	155	156	157	158	159
		160	161	162	163	164	165	166	167	168	169	170	171	172	173	174	175
		176	177	178	179	180	181	182	183	184	185	186	187	188	189	190	191
	Tile Row 3	192	193	194	195	196	197	198	199	200	201	202	203	204	205	206	207
		208	209	210	211	212	213	214	215	216	217	218	219	220	221	222	223
		224	225	226	227	228	229	230	231	232	233	234	235	236	237	238	239
		240	241	242	243	244	245	246	247	248	249	250	251	252	253	254	255

You can find a full list of the descriptions and parameters in the API Reference Overview.

Connections to the cascade ports can be made as follows:

for (int i = 0 ; i < P_CASC_LEN; i++) {
    connect<>(inA[i], mmultGraph.inA[i]);
    connect<>(inB[i], mmultGraph.inB[i]);
}
connect<>(mmultGraph.out, out);