Example Usage - 2023.2 English

Vitis Libraries

Release Date
2023.2 English

Since programming the FPGA is very time-consuming, we provide the hardware initialization and hardware run processes independently.

For initializing the hardware:

// Initializes FPGA
gqe::FpgaInit init_ocl(xclbin_path);
// Allocates pinned host buffers
// Allocates device buffers

For loading the table columns and building the bloom-filter’s hash-table:

// Please first load data to corresponding table L column buffers
// We assume input table columns is stored in tab_l_col0 & tab_l_col1,
// and the corresponding number of rows is stored in table_l_nrow

const int BUILD_FACTOR = 10;
using namespace xf::database;

// Builds 0 - 1/BUILD_FACTOR of table L into bf1
for (int i = 0; i < table_l_nrow / BUILD_FACTOR; i++) {
    tab_o1_col0[i] = tab_l_col0[i];
    tab_o1_col1[i] = tab_l_col1[i];

// Builds 1/BUILD_FACTOR - 2/BUILD_FACTOR of table L into bf2
for (int i = table_l_nrow / BUILD_FACTOR; i < (table_l_nrow / BUILD_FACTOR) * 2; i++) {
    tab_o2_col0[i - table_l_nrow / BUILD_FACTOR] = tab_l_col0[i];
    tab_o2_col1[i - table_l_nrow / BUILD_FACTOR] = tab_l_col1[i];

// Total number of unique keys at maximum
uint64_t total_num_unique_keys = table_l_nrow / BUILD_FACTOR * 2;

// Creates L table
gqe::Table tab_l("Table L");
tab.addCol("l_orderkey", gqe::TypeEnum::TypeInt64, tab_l_col0, table_l_nrow);
tab.addCol("l_extendedprice", gqe::TypeEnum::TypeInt64, tab_l_col1, table_l_nrow);

// Creates O1 table for building bloom-filter-1
gqe::Table tab_o1("Table O1");
tab_o1.addCol("l_orderkey", gqe::TypeEnum::TypeInt64, tab_o1_col0, table_l_nrow / BUILD_FACTOR);
tab_o1.addCol("l_extendedprice", gqe::TypeEnum::TypeInt64, tab_o1_col1, table_l_nrow/ BUILD_FACTOR);

// Builds bloom-filter-1
gqe::BloomFilter bf1(total_num_unique_keys);
bf1.build(tbl_o1, "l_orderkey");

// Creates C table for stroing filtered results, at worst, we'll get every input key passes through the bloom-filter,
// so the size of the table c should be the same with table L
gqe::Table tab_c("Table C");
tab_c.addCol("c1", gqe::TypeEnum::TypeInt64, tab_c_col0, table_l_nrow);
tab_c.addCol("c2", gqe::TypeEnum::TypeInt64, tab_c_col0, table_l_nrow);

Some use cases may need to merge several bloom-filters before running the filtering process:

// Creates O2 table for building bloom-filter-2
gqe::Table tab_o2("Table O2");
tab_o2.addCol("l_orderkey", gqe::TypeEnum::TypeInt64, tab_o2_col0, table_l_nrow / BUILD_FACTOR);
tab_o2.addCol("l_extendedprice", gqe::TypeEnum::TypeInt64, tab_o2_col1, table_l_nrow/ BUILD_FACTOR);

// Builds bloom-filter-2
gqe::BloomFilter  bf2(total_num_unique_keys);
bf2.build(tab_o2, "l_orderkey");

// Merges bloom-filter-2 into bloom-filter-1

At last, call the run API of gqe::Filter to perform the pipelined bloom-filtering:

// Creates bloom-filter engine
gqe::Filter bigfilter(init_ocl);
// Creates StrategySet object to pass on the number of sections of the table
gqe::StrategySet params;
// if set to 0, the section info should be provided in table L
// if set to n (n > 0), the table L should be divided into n sections evenly
params.sec_l = 1;
gqe::ErrCode err_code;
// Performs the bloom-filtering
err_code = bigfilter.run(tab_l // input talbe
                         "l_orderkey", // selects key from input table
                         bf1, // bloom-filter which provides hash-table
                         "",  // dynamic filter condition, empty for all passes through
                         tab_c, // output table
                         "c1=l_extendedprice, c2=l_orderkey", // output mapping
                         params); // Parameter strcut to provide section info