Towards explainable orchid flower identification
Diah Apriyanti is a PhD student in the department Datamanagement & Biometrics. (Co)Promotors are prof.dr. P.J.F. Lucas and dr.ir. L.J. Spreeuwers from the faculty Electrical Engineering, Mathematics and Computer Science.
Orchidaceae is one of the largest families in the plant kingdom. It has more than 25,000 orchid species. Compared with other parts of the orchid plant, e.g., leaf, fruit, and root, the orchid flowers have distinct features such as color, shape, texture, etc., that can be used to distinguish between species. Taxonomists and botanists often use these unique characteristics to identify plants.
Automated flower identification systems have been developed in recent years. However, all of these systems operate as blackboxes, making it difficult to understand the reasons behind a decision. Recent studies only focus on image processing techniques and machine learning methods, while taxonomist knowledge is not taken into account.
This thesis presents a new method for explainable plant identification based on flower characteristics by incorporating taxonomist knowledge into artificial intelligence models. The initial solution for realizing the idea is discussed, starting with a decision tree. In order to build specific classifiers that can directly extract flower characteristics from images, we first constructed a new dataset that consists of images, flower characteristics, and names of species. Afterwards, we built a color classifier using deep learning. How to construct classifiers of other features than color is also explained, and we also investigates the classifiers’ effectiveness. Some relatively basic Bayesian-network classification algorithms are applied to study the effect of several combinations of image features. A more knowledge-based approach is made to incorporate uncertain taxonomic knowledge into the classification process, by combining human knowledge with structure learning. The resulting Bayesian networks are combined with an image-based deep neural network to obtain an excellently performing classification method that is also provided with the capability of explanation.
Methods proposed in this thesis can help taxonomists, botanists, and plant enthusiasts to easily and quickly identify plants, while also providing an explanation of solutions with an additional indication of their trustworthiness. The methods can also be used as learning aids for plant enthusiasts, in particular those with an interest in orchids. Furthermore, with respect to ‘Explainable Artificial Intelligence’(XAI), the methods developed in this thesis could be useful in a variety of domains outside plant recognition.
