Normally machine learning systems perform well. However when there is a problem with the trained machine learning system (for example the machine learning system performs worse than random ...) this great "guessing game" is begins. With "guessing game", I allude to my experience. For me it seems, that debugging machine learning systems is most often done by guessing the problem rather then on a methodological way.
And since there are numerous of reasons why a machine learning system may fail, finding the actual bug can be pretty time consuming. For example the bug may be due to:
- biased trainingset
- insufficient training data
- datasets containing errors
- unrepresentative/to many features
- uncareful training (for example in neuronal-networks, when the training data is not presented randomly)
Is there a machine learning system that is easy to debug? (And how can it be debugged?)
Is there a known methodical way of debugging machine learning systems at all?