Tiling Parameters Examples - 2024.1 English

AI Engine-ML Kernel and Graph Programming Guide (UG1603)

Document ID
UG1603
Release Date
2024-06-06
Version
2024.1 English
This section uses a graph that contains a memory tile to show how data is written to or read from the memory tile by tiling parameters. The graph code is:
class mygraph: public graph {
  public:
    kernel first;
    input_plio in;
    output_plio out;
    shared_buffer<int32> mtxB;
    mygraph() {
      first=kernel::create(producer);
      source(first) = "producer.cc";
      in=input_plio::create("Datain0", plio_64_bits, "data/input.txt");
      out=output_plio::create("Dataout0", plio_64_bits, "data/output.txt");
      // number of elements, input port=1, output port=1
      mtxB = shared_buffer<int32>::create({256}, 1, 1);
    runtime<ratio>(first)=0.9;
    connect(in.out[0], mtxB.in[0]);
    connect(mtxB.out[0], first.in[0]);
    connect(first.out[0], out.in[0]);
    dimensions(first.in[0])={256};
    dimensions(first.out[0])={16,16};

    location<buffer>(mtxB)=address(6,0,0);
    write_access(mtxB.in[0]) = tiling({.buffer_dimension={256}, .tiling_dimension={256}, .offset={0} });
    read_access(mtxB.out[0]) = tiling({.buffer_dimension={256}, .tiling_dimension={256}, .offset={0} });
  }
};
Following are examples of different tiling configurations. You can see how source data is transferred to destination, how data is stored in row basis in memory, and how the configurations can be applied on write_access or read_access by default.
Note: Zero padding is only valid for read_access.

1D Linear Transferring From Source to Destination

1D Linear transferring from source to destination using three equivalent lines of code (anyone applies):

tiling({.buffer_dimension={256}, .tiling_dimension={256}, .offset={0} });
tiling({.buffer_dimension={256}, .tiling_dimension={256}, .offset={0}, 
      .tile_traversal = {{.dimension=0, .stride=256, .wrap=1}} });
tiling({.buffer_dimension={256}, .tiling_dimension={1}, .offset={0}, 
      .tile_traversal = {{.dimension=0, .stride=1, .wrap=256}} });
Figure 1. Example of 1D Linear Transferring From Source to Destination

1D Linear Zero Pre-padding

Code sample showing how zeros may be pre-padded to the destination:

read_access(mtxB.out[0])=tiling({.buffer_dimension={256}, 
      .tiling_dimension={256}, .offset={-32}, 
      .tile_traversal = {{.dimension=0, .stride=256, .wrap=1}} });
Figure 2. Example of 1D Linear Zero Padding

1D Linear with Zero Pre-padding and Post-padding

Code sample showing how zeros may be pre-padded and appended to the destination:

read_access(mtxB.out[0])=tiling({.buffer_dimension={256}, 
      .tiling_dimension={288}, .offset={-16}, 
      .tile_traversal = {{.dimension=0, .stride=256, .wrap=1}}, 
      .boundary_dimension = {256} });
Figure 3. Example of 1D Linear with Zero Padding

Truncated 1D Linear with Zero Padding

Code sample showing how zeros may be pre-pended and appended to the destination with truncation:

read_access(mtxB.out[0]) = tiling({.buffer_dimension={256}, 
      .tiling_dimension={128}, .offset={-16}, 
      .tile_traversal = {{.dimension=0, .stride=144, .wrap=2}}, 
      .boundary_dimension={96} });
Figure 4. Example of Truncated 1D Linear with Zero Padding

2D Linear Transferring from Source to Destination

2D Linear transferring from source to destination using two equivalent lines of code (anyone applies):

tiling({.buffer_dimension={128,2}, .tiling_dimension={128,2}, 
      .offset={0,0} });
tiling({.buffer_dimension={128,2}, .tiling_dimension={128,1}, .offset={0,0},
      .tile_traversal = {{.dimension=0, .stride=128, .wrap=1}, 
            {.dimension=1, .stride=1, .wrap=2}} });
Figure 5. 2D Linear


2D Linear with Zero Padding

2D Linear with zero padding on the left and right borders:

read_access(mtxB.out[0]) = tiling({.buffer_dimension={128,2}, 
      .tiling_dimension={160,1}, .offset={-16,0},
      .tile_traversal = {{.dimension=0, .stride=128, .wrap=1}, 
            {.dimension=1, .stride=1, .wrap=2}} });
Figure 6. Example of 2D Linear with Zero Padding

2D Matrix Transferring to 4x2 Sub Matrices

2D Matrix transferring to transfer 4x2 sub matrices. The numbers show how data are stored in source and transferred to destination with one such tiling parameter:

read_access(src.out[0]) = tiling({.buffer_dimension={16,4}, 
         .tiling_dimension={4,2}, .offset={0,0}, 
         .tile_traversal = {{.dimension=0, .stride=4, .wrap=4}, 
               {.dimension=1, .stride=2, .wrap=2}} });
Figure 7. Example of 2D Matrix Transferring to 4x2 Sub Matrices

2D Shuffle from Higher Address to Lower Address

Multiple tiling parameters can be chained together to fetch the higher address matrix content first, followed by the lower address, as demonstrated in the following example:

read_access(mtxB.out[0]) = { tiling({.buffer_dimension={32,8}, 
    .tiling_dimension={16,8}, .offset={16,0},
//higher address matrix content    .tile_traversal = {{.dimension=0, .stride=1, .wrap=1}} }), 
tiling({.buffer_dimension={32,8}, .tiling_dimension={16,8}, .offset={0,0},
    //lower address matrix content
.tile_traversal = {{.dimension=0, .stride=1, .wrap=1}} }) }; 
Figure 8. Example of 2D Shuffle from Higher Address to Lower Address

3D Linear Transferring from Source to Destination

3D Linear transferring from source to destination, without changes made:

tiling({.buffer_dimension={32,4,2}, .tiling_dimension={32,4,2}, 
      .offset={0,0,0} });
Figure 9. Example of 3D Linear Transferring from Source to Destination

3D Linear with Zero Padding

3D linear with zero padding.

read_access(mtxB.out[0]) = tiling({.buffer_dimension={32,4,2}, 
    .tiling_dimension={34,6,2}, .offset={-1,-1,0}, 
    .tile_traversal = {{.dimension=0, .stride=128, .wrap=1}, 
            {.dimension=1, .stride=4, .wrap=1}
             , {.dimension=2, .stride=2, .wrap=1}} });
Figure 10. Example of 3D Linear with Zero Padding

4D Data Transfer

Following is an example of four dimensional data transfer. The example demonstrates a tiling dimension of {4,1,1,4}. Tile traversal occurs on dimension 2 first, followed by dimension 1.

tiling({.buffer_dimension={4,4,4,4}, .tiling_dimension={4,1,1,4}, 
      .offset={0,0,0,0}, 
	.tile_traversal = {{.dimension=2, .stride=1, .wrap=4},
            {.dimension=1, .stride=1, .wrap=4}} });

The transmitted data is (assuming the input data is 0~255):

#Each line represents a tiling dimension {4,1,1,4}
0 1 2 3 64 65 66 67 128 129 130 131 192 193 194 195 
16 17 18 19 80 81 82 83 144 145 146 147 208 209 210 211 
32 33 34 35 96 97 98 99 160 161 162 163 224 225 226 227 
48 49 50 51 112 113 114 115 176 177 178 179 240 241 242 243 
4 5 6 7 68 69 70 71 132 133 134 135 196 197 198 199 
20 21 22 23 84 85 86 87 148 149 150 151 212 213 214 215 
36 37 38 39 100 101 102 103 164 165 166 167 228 229 230 231 
52 53 54 55 116 117 118 119 180 181 182 183 244 245 246 247 
8 9 10 11 72 73 74 75 136 137 138 139 200 201 202 203 
24 25 26 27 88 89 90 91 152 153 154 155 216 217 218 219 
40 41 42 43 104 105 106 107 168 169 170 171 232 233 234 235 
56 57 58 59 120 121 122 123 184 185 186 187 248 249 250 251 
12 13 14 15 76 77 78 79 140 141 142 143 204 205 206 207 
28 29 30 31 92 93 94 95 156 157 158 159 220 221 222 223 
44 45 46 47 108 109 110 111 172 173 174 175 236 237 238 239 
60 61 62 63 124 125 126 127 188 189 190 191 252 253 254 255 

Following picture shows how the elements are transmitted:

Figure 11. 4D Data Transfer