I faced another issue for frames without detections driving me towards the same solution. If dists is not re-initialized to an empty matrix per iteration, the matrix from the previous frame is reused in the next iteration because the following block does not run when the ground truth or detections are absent. This leads to using the valid_j of the preceding frame to index empty hids.

if len(oids) > 0 and len(hids) > 0:
    ...
    dists = compute_dist(...)
    ...

Edit 1: We see a different exception because the OP has the first frame without detections while I have missing detections in the 72nd frame out of 750.