Microsoft’s MSDN magazine for its development community has been publishing quite a few introductory articles to Machine Learning (ML) over the past few months. January’s issue emphasizes ML with another series of articles, albeit with differing qualities. I liked this quote from the editorial “Advancing AI“: ”ML is a huge graph that requires you to repeatedly examine topics, learning a bit more each time and understanding how topics are interrelated”. An interesting article “Introduction to PyTorch on Windows” is from James McCarthy and only highlights recent rumbles from Microsoft about switching from developing and using its own CNTK library to open source PyTorch, developed mainly by Facebook. In terms of activity, PyTorch statistics on GitHub are about half the activity for TensorFlow and both dwarf CNTK numbers. From this perspective, Microsoft made the right choice abandoning CNTK. For an example of PyTorch staying current, latest version of PyTorch introduces two modes for Python-based ML that enables just in time JIT compilation to improve PyTorch’s adaptability to production environments. Another article “Self-Organizing Maps Using C#” is about an ML technique that is not well known and its usability seems questionable. The third article “Leveraging the Beliefs-Desires-Intentions Agent Architecture” is poorly written and shamelessly plugs the author’s travel agency in the provided sample app.
For most problems in ML, learning is achieved by applying non-linear activation functions to hypotheses. This allows the algorithms to discover non-linearity in data and predict unseen data with greater clarity. Trusted sigmoid activation functions are being replaced by other activation functions and also initialization is being improved. In fact, the “Understanding the difficulty of training deep feedforward neural networks” study details replacing standard random initialization of Deep Learning (DL) networks with other initialization methods. DL networks are neural networks with high number of hidden layers. The study gets rather mathy half way through, but most of the article is digestible. The resulting initialization algorithm is now called Xavier initialization after the first author of the paper and is supported by all leading ML frameworks.