Development of deep learning applications over a variety of new hardware opens many challenges for developers. These challenges include requirements for massive parallelization, runtime optimization and debugging of computation over a large amount of data. Virtual machines can provide solutions to these challenges, by hiding some of the underlying complexity and providing friendly programming models. This talk discusses aspects of the VMs that can be utilized and favoured for development of deep learning applications.