Links

DOI

There is an increased interest in solving complex constrained problems where part of the input is not given as facts, but received as raw sensor data such as images or speech. We will use ‘visual sudoku’ as a prototype problem, where the given cell digits are handwritten and provided as an image thereof. In this case, one first has to train and use a classifier to label the images, so that the labels can be used for solving the problem. In this paper, we explore the hybridisation of classifying the images with the reasoning of a constraint solver. We show that pure constraint reasoning on predictions does not give satisfactory results. Instead, we explore the possibilities of a tighter integration, by exposing the probabilistic estimates of the classifier to the constraint solver. This allows joint inference on these probabilistic estimates, where we use the
solver to find the maximum likelihood solution. We explore the trade-off between the power of the classifier and the power of the constraint reasoning, as well as further integration through the additional use of structural knowledge. Furthermore, we investigate the effect of calibration of the probabilistic estimates on the reasoning. Our results show that such hybrid approaches vastly outperform a separate approach, which encourages a further integration of prediction (probabilities) and constraint solving.
Original languageEnglish
Title of host publicationHybrid Classification and Reasoning for Image-based Constraint Solving
EditorsEmmanuel Hebrard, Nysret Musliu
Place of PublicationCham
PublisherSpringer
Pages364-380
Number of pages16
Volume 12296
ISBN (Electronic)978-3-030-58942-4
ISBN (Print)978-3-030-58941-7
DOIs
Publication statusPublished - 19 Sep 2020
Event17th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research - Online
Duration: 21 Sep 202024 Sep 2020
https://cpaior2020.dbai.tuwien.ac.at/

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
ISSN (Print)0302-9743

Conference

Conference17th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research
Period21/09/2024/09/20
Internet address

    Research areas

  • Constraint Reasoning, Visual sudoku, Joint Inference, Prediction and Optimisation

ID: 49870272