Limited-Memory Multinomial Logistic Regression Classifier for Matlab
When using Matlab’s
mnrfit to train a multinomial logistic regression classifier recently, I found it rather memory-consuming. Specifically, when training a classifier with thousands of samples and tens of thousands of samples, it used up the 32GB of RAM on a workstation and forced it to maintain considerable virtual memory. What should’ve been a CPU-bound problem then became a HDD-bound problem, and I never received my result (it just took tooooo long).
After reading about the awesome optimization code minFunc, I decided to implement a classifier on my own. Thanks to minFunc and its examples, I can finish this little piece of code and publish it here.
Setup & Usage
mexAll, which might be required the first time you use it. Then, add all minFunc’s subdirectories to your Matlab path (e.g.
addpath(genpath('./minFunc'))) and you’ll be able to run the following test case.
>> x = repmat(eye(3),[10 1]); y = repmat([1;2;3],[10 1]); x = x + 0.1*randn(size(x)); >> model = mnlr_fit(x, y); >> prediction = mnlr_predict(model, x, 1); >> sum(prediction == y) / length(y) ans = 1That is, we just trained our 3-way classifier on a small dataset of 30 samples and achieved 100% training accuracy.
blog comments powered by Disqus