Vivado Hardware Manager supports debugging on DFX designs. To debug such a design successfully, it is necessary to program the full design bitstream before programming the partial bitstream to replace specific reconfigurable modules.
For an example of instantiating debug cores in a DFX design, and functionality within the Vivado Hardware Manager, see the 'Vivado Debug and the DFX Project Flow' section in Vivado Design Suite Tutorial: Dynamic Function eXchange (UG947).