Visual Sudoku App for Android
A visual sudoku solver based on cost function networks
This application solves the sudoku problem from a smartphone by reading the grid using its camera. The cost function network solver toulbar2 is used to deal with the uncertainty on the digit recognition produced by the neural network. This uncertainty, combined with the sudoku logical rules, makes it possible to correct perceptual errors. It is particularly useful in the case of hand-written digits or poor image quality. It is also possible to solve a partially filled-in grid with printed and hand-written digits. The solver will always suggest a valid solution that best adapts to the retrieved digit information. It will naturally detect (a small number of) errors in a partially filled-in grid and could be used later as a diagnosis tool (future work). This software demonstration emphasizes the tight relation between constraint programming, computer vision, and deep learning.
We used the open-source C++ solver toulbar2 in order to find the maximum a posteriori solution of a constrained probabilistic graphical model. With its dedicated numerical (soft) local consistency bounds, toulbar2 outperforms traditional CP solvers on this problem. Grid perception and cell extraction are performed by the computer vision library OpenCV. Digit recognition is done by Keras and TensorFlow. The current android application is written in Python using the Kivy framework. It is inspired from a tutorial by Adrian Rosebrock. It uses the ws RESTful web services in order to run the solver.
See also : Visual Sudoku Application.
Source Code
Download and Install
To install the ‘Visual Sudoku’ application on smartphone :
Download the visualsudoku-release.apk APK file from Github repository :
Click on the downloaded visualsudoku-release.apk APK file to ask for installation (you have to accept to ‘install anyway’ from unknown developer).
In your parameter settings for the app, give permissions to the ‘Visual Sudoku’ application (smartphone menu ‘Parameters’ > ‘Applications’ > ‘Visual Sudoku’) : allow camera (required to capture grids), files and multimedia contents (required to save images as files). Re-run the app.
Warnings :
The application may fail at first start and you may have to launch it twice.
While setting up successfully, the application should have created itself the required ‘VisualSudoku’ folder (under the smartphone ‘Internal storage’ folder) but if not, you will have to create it by yourself manually.
Since the application calls a web service, an internet connection is required.
Description
The ‘SETTINGS’ menu allows to save grids or solutions as image files (‘savinginputfile’, ‘savingoutputfile’ parameters) and to access to some ‘expert’ parameters in order to enhance the resolution process (‘keep’, ‘border’, ‘time’ parameters).
The application allows to capture a grid from its own camera (‘CAMERA’ menu) or to select a grid among the smartphone existing files (‘FILE’ menu), for example files coming from ‘DCIM’, in .jpg or .png formats. The grid image must have been captured in portrait orientation. Once the grid has been chosen, the ‘Solve’ button allows to get the solution.
Fig.1
Fig.2
Fig.3
Fig.1 : Screen of main menu
Fig.2 : Screen of the grid to be solved
Fig.3 : Screen of the solution (in yellow) found by the solver
