When Will Breakfast be Ready: Temporal Prediction of Food Readiness Using Deep Convolutional Neural Networks on Thermal Videos

摘要

In this paper, we perform prediction of food readiness during cooking by using deep convolutional neural networks on thermal video data. Our work treats readiness prediction as ultra-fine recognition of progression in cooking at a per-frame level. We analyze the performance of readiness prediction for eggs, pancakes, and bacon strips using two types of neural networks: a classifier network that bins a frame into one of five classes depending on how far cooking has progressed at that frame, and a regressor network that predicts percentage of cooking time spent at each frame. Our work provides classification accuracies of 98% and higher within one step of the ground truth class using the classifier, and provides an average error of within 20 seconds for the elapsed time predicted using the regressor when compared to ground truth.