There is an increased interest in solving complex constrained problems where part of the input is not given as facts, but received as raw sensor data such as images or speech. We will use ‘visual sudoku’ as a prototype problem, where the given cell digits are handwritten and provided as an image thereof. In this case, one first has to train and use a classifier to label the images, so that the labels can be used for solving the problem. In this paper, we explore the hybridisation of classifying the images with the reasoning of a constraint solver. We show that pure constraint reasoning on predictions does not give satisfactory results. Instead, we explore the possibilities of a tighter integration, by exposing the probabilistic estimates of the classifier to the constraint solver. This allows joint inference on these probabilistic estimates, where we use the
solver to find the maximum likelihood solution. We explore the trade-off between the power of the classifier and the power of the constraint reasoning, as well as further integration through the additional use of structural knowledge. Furthermore, we investigate the effect of calibration of the probabilistic estimates on the reasoning. Our results show that such hybrid approaches vastly outperform a separate approach, which encourages a further integration of prediction (probabilities) and constraint solving.
Original languageEnglish
Title of host publicationHybrid Classification and Reasoning for Image-based Constraint Solving
PublisherSpringer
Number of pages15
Publication statusAccepted/In press - 27 May 2020
Event17th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research - Vienna, Vienna, Austria
Duration: 26 May 202029 May 2020
https://cpaior2020.dbai.tuwien.ac.at/

Conference

Conference17th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research
Abbreviated titleCPAIOR 2020
CountryAustria
CityVienna
Period26/05/2029/05/20
Internet address

    Research areas

  • Constraint Reasoning, Visual sudoku, Joint Inference, Pre- diction and Optimisation

ID: 49870272