Brady is the son of Isabella Toscano & John Black. Isabella is daughter of Victor, & to Xander, would have been lst cousins. So, technically, Brady would have been lst cousin, once removed. However, they were NOT raised together at all. They once spent a couple weeks during the summer, on a Greek island, along with Daniel, but that was the extent of their contact.
Since it is known that Victor has helped Xander in the past, and is so big on "family", it would seem Brady is being considerate of that factor for Victor, and cares not for any tie to himself.