This ValueError just breaks the code for no apparent reason. I know that not having this makes things a bit ambiguous but I see no reason to just pick one set of annotations and construct bounding boxes from them.