When I was doing research on deep learning structures, I was amazed by convolutional RBM when I read about it because it possess the amazing capability of learning object parts and “recovering” missing parts (see Reference). Although Alex Krizhevsky has an implementation of such network in pure CUDA, I think some (like myself) might want to use a more friendly Matlab version to process smaller amount of data with a bit more time-consumption – so here we are.
|Current Version||0.2 alpha|
|Note (Dec. 8, 2012)||A major version is under (heavy) construction, which will include modified MEX interfaces as well as modified function interfaces. A proper version of CUDA MEX source files and setup documentation will also be included in this version.|
|Important Note||This code is not an official release by the authors and may contain defect resulting in erroneous outcome. Please use with caution and all possible losses is at the user’s own risk. For research / personal purposes only and should be kept public with this note. Feedbacks welcome below this post, and also in the project homepage.|
|Source code||Available at the project homepage|
|Coding status||Under heavy development|
FAQs & Remarks
- I can’t run the code. Functions not exist?
To run this code properly, run make, and follow the instructions to setup your compiler for MEX to compile necessary MEX files.
- How to use your CUDA-MEX?
See A Guide to Using NVMEX Tool. I’ve modified the nvmexopts.bat file a bit to link the CUDA libs. make will ask you whether you want to compile CUDA MEX files. This feature is limited by the nvmexopts.bat file and currently only work under Win32. Some configuration of the bat file is needed, which is covered in the software documentation (to come).
- How fast can this code run?
I can’t say it runs fast: after all the current (0.1 alpha) backbone code is still implemented in Matlab. But with mex implementation, it’s about 5x faster than a pure Matlab implememtation. However since CUDA-MEX is not quite taken advantage of, the current version (0.1 alpha) using CUDA-MEX runs about 3x slower on my laptop, than the MEX version.
- I found a bug. Where do I report to?
Both commenting below this post and adding an issue at the project homepage will do.
- Future development plans?
This code is still under heavy development before it’s usable in real-world applications instead of toy-sized problems. A full MEX / CUDA-MEX implementation is planned. Besides, a gadget script for compiling MEX / CUDA-MEX files, as well as sample data and usage, are considered.
You may want to cite the following publications when your research uses this code (like I said, this is *not* an official release by the authors so use caution):
- Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, Honglak Lee, Roger Grosse, Rajesh Ranganath and Andrew Y. Ng. In Proceedings of the Twenty-Sixth International Conference on Machine Learning, 2009.