We introduce Intelligent Annotation Dialogs for bounding box annotation. Wetrain an agent to automatically choose a sequence of actions for a humanannotator to produce a bounding box in a minimal amount of time. Specifically,we consider two actions: box verification [37], where the annotator verifies abox generated by an object detector, and manual box drawing. We explore twokinds of agents, one based on predicting the probability that a box will bepositively verified, and the other based on reinforcement learning. Wedemonstrate that (1) our agents are able to learn efficient annotationstrategies in several scenarios, automatically adapting to the difficulty of aninput image, the desired quality of the boxes, the strength of the detector,and other factors; (2) in all scenarios the resulting annotation dialogs speedup annotation compared to manual box drawing alone and box verification alone,while also out- performing any fixed combination of verification and draw- ingin most scenarios; (3) in a realistic scenario where the detector isiteratively re-trained, our agents evolve a series of strategies that reflectthe shifting trade-off between verification and drawing as the detector growsstronger.
展开▼
