The input/output data columns save all raw column data. The statistic information, e.g., row number, column number, is recorded in the meta information. The internal structure of the meta information is shown in the following figure. It is important to make it clear that the meta information is used for both the input port and the output port; the same meta struct should be feed to the input as well as the output meta ports of the 3-in-1 GQE. Moreover, you use only a number of input rows and section ID information before loading the actual table rows for every flows supported in 3-in-1 GQE.
Note
- Join/Aggr/Bloomfilter only uses the first eight rows of meta for the number of input rows and section ID.
- Part flow supports a maximum of 256 partitions; each number of rows in a partition is described by a 32-bit integer, starting from output meta[8], 256/16 = 16 lines are used for partitioning row output.
- The size of each partition is configured through meta row 0, from 104 to 135 bits. You can achieve this by simply calling the
setPartitionAPI in classMetaTableof L2. - When the number of partition is less than 256, the first N parts of meta row 8-23 are used, starting from meta[8][31-0] with little endian.