No need to rebuild, the holes will get used when further inserts occur.
Rest in peace and let the indexes rest too.