In recent years, we have witnessed immense advances in deep learning, which is often associated with the developments in hardware technologies. Additional to it, one can not deny the impact of deep learning libraries on the boost of deep learning reasearch and developments. They allow researchers and developers to implement and test novel ideas quickly. Probabilistic programming and Bayesian inference is passing through the similar path nowadays. Probabilistic programming libraries are ubiquitous thanks to several novel techniques in Bayesian inference mostly based on automatic differentiation. Black Box Variational Inference (BBVI) by Ranganath et al (https://arxiv.org/pdf/1401.0118.pdf) is among them and you can find my notes on the topic together with some toy examples here.

Bayesian inference in complex models has always been a though task to achieve especially if there are also model parameters to be estimated in the model definition. Although Expectation-Maximization (EM) algorithm, which is based on the alternating optimization of variational lower bound and model parameters, serves a good framework, most of the time we are not capable of evaluating the exact posterior of latent variables. In this case, approximations to the exact posteriors are made either via sampling based Monte Carlo methods or optimization based variational inference. The letter one is faster but mostly limited to the models that are carefully designed by considering conditional conjugacies. This was discouraging researchers to design models that are parameterized with neural networks until VAEs are introduced.

One common way of interpreting Deep Neural Netowrks (DNNs) is that the first layers work as feature extractors and the middle layers map extracted features to a new low dimensional manifold such that a Generalized Linear Model can make the discremenation. Thus for a classification task, we want to map the instances to a manifold that the classes are linearly seperated. From a probabilistic point of view, minimization of negative log-likelihood leads to the loss functions that are being used in Deep Learning. In order to see this connection more detailly, I invite you to take a look at my medium article on Generalized Linear Models.

When I first heard the term deep learning, I thought it is nothing but a feed forward neural network with more than one hidden layers. But these were the times when I was new in machine learning area and machine learning was just a classification, regression, or clustering for me. And again in these times, I was interpretting ANNs as a machine learning methodology to handle the problems which can not be solved by linear models. Therefore it was not clear for me that why Deep Learning is so magical. Throughout the time, I learned different neural network types like Convolutional Neural Networks(CNNs), Recurrent Neural Networks(RNNs), Autoencoders, Deep Generative Models, etc. and reasons why Deep Learning is so popular, started to be more clear to me, gradually.

This is a post which was published in my Medium account before. This very short post basically gives an intuition about what you can do with machine learning. It also mentions learning types very quickly. Here you can find the post.

This is the first post I am creating in Github pages. In this page, I am planning to blog about the topics I am studying and these blogs will be mostly about machine learning and fields which are related with machine learning like probabilistic models, deep learning, natural language processing, and computer vision.