When I was doing research on deep learning structures, I was amazed by convolutional RBM when I read about it because it possess the amazing capability of learning object parts and “recovering” missing parts (see Reference). Although Alex Krizhevsky has an implementation of such network in pure CUDA, I think some (like myself) might want to use a more friendly Matlab version to process smaller amount of data with a bit more time-consumption – so here we are.
|Note (Feb. 7, 2014)||New CPU code updated for new data structure. GPU code will _not_ work. Some updates on help text, code comments, as well as mex option files. New nvmex option file for MacOS added.|
|Important Note||This code is not an official release by the authors and may contain defect resulting in erroneous outcome. Please use with caution and all possible losses is at the user’s own risk. For research / personal purposes only and should be kept public with this note. Feedbacks welcome below this post, and also in the project homepage.|
|Source code||Available at the project homepage|
|Coding status||Under heavy development|
- The CUDA code might crash or produce rubbish results. Although I make the best effort to keep the CPU code of this project clean and usable, I sometimes struggle to do the same for the CUDA code in the same timely manner. If that happens, please accept my deepest apologies and hang tight for my next release.
- The CUDA code is currently not running properly (at all) on Mac OS 10.9.1. This seems to be a general CUDA issue since the Mavericks update. Please sit tight before nVidia releases new drivers to save the world.
- The CUDA code might not compile properly on Linux systems (Ubuntu, etc). I don’t have a Linux system at hand set up with CUDA and everything for the moment to test the building scripts. But anyone’s help on this is very much welcomed.
FAQs & Remarks
- I can’t run the code. Functions not exist?
To run this code properly, run make, and follow the instructions to setup your compiler for MEX to compile necessary MEX files.
- How to use your CUDA-MEX?
See A Guide to Using NVMEX Tool. I’ve modified the nvmexopts.bat file a bit to link the CUDA libs. make will ask you whether you want to compile CUDA MEX files. This feature is limited by the nvmexopts.bat file and currently only work under Win32. Some configuration of the bat file is needed, which is covered in the software documentation (to come).
- How fast can this code run?
I can’t say it runs fast: after all the current (0.1 alpha) backbone code is still implemented in Matlab. But with mex implementation, it’s about 5x faster than a pure Matlab implememtation. However since CUDA-MEX is not quite taken advantage of, the current version (0.1 alpha) using CUDA-MEX runs about 3x slower on my laptop, than the MEX version.
- I found a bug. Where do I report to?
Both commenting below this post and adding an issue at the project homepage will do.
- Future development plans?
This code is still under heavy development before it’s usable in real-world applications instead of toy-sized problems. A full MEX / CUDA-MEX implementation is planned. Besides, a gadget script for compiling MEX / CUDA-MEX files, as well as sample data and usage, are considered.
You may want to cite the following publications when your research uses this code (like I said, this is *not* an official release by the authors so use caution):
- Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, Honglak Lee, Roger Grosse, Rajesh Ranganath and Andrew Y. Ng. In Proceedings of the Twenty-Sixth International Conference on Machine Learning, 2009.
- Stacks of Convolutional Restricted Boltzmann Machines for Shift-Invariant Feature Learning, Mohammad Norouzi, Mani Ranjbar, Greg Mori, IEEE Computer Vision and Pattern Recognition (CVPR), 2009.