When you specify Bootgen options on the command line you have many more options than those provided in the Vitis IDE. In the standard install of the Vitis software platform, the XSCT (Xilinx Software Command-Line Tool) is available for use as an interactive command line environment, or to use for creating scripting. In the XSCT, you can run Bootgen commands. XSCT accesses the Bootgen executable, which is a separate tool. This Bootgen executable can be installed standalone as described in Installing Bootgen. This is the same tool as is called from the XSCT, so any scripts developed here or in the XSCT will work in the other tool.
The Xilinx Software Command-Line Tool in the Vitis Embedded Software Development Flow Documentation (UG1400) describes the tool. See the "XSCT Use Cases" chapter for an example of using Bootgen commands in XSCT.