icsiboost

I recently tried using Boosting, a meta-learning approach that combines weak classifiers to create a strong classifier. Specifically, I used Adaptive Boosting (Adaboost), which searches for a linear combination of classifiers by overweighting examples that are misclassified by each classifier. I found that the icsiboost project offered a great implementation of Adaboost over stumps (one-level decision trees) on discrete and continuous attributes (words and real values).

The approach is highly efficient and simple when it comes to combining continuous and nominal values. Moreover, the implementation I used aimed to allow training from millions of examples by hundreds of features within a reasonable time/memory.

Here are some key usage options for anyone interested in trying out icsiboost:

- Use the "-S < stem >" option to define a model/data/names stem
- Use the "-n < iterations >" option to specify the number of boosting iterations
- Use the "-E < smoothing >" option to set a smoothing value (default is 0.5)
- Use the "--jobs < threads >" option to specify the number of threaded weak learners
- Use the "--output-weights" option to output training example weights at each iteration
- Use the "--model < model >" option to save/load the model to/from a file instead of the default stem.shyp file
- Use the "--train < file >" option to specify training examples
- Use the "--test < file >" option to output additional error rate from another file during training

This software release includes several bugfixes in training and test procedures, as well as error rate reports on multi-class problems. In addition, optimization of the most called functions has brought nice training speed improvements. Another great feature is the updated documentation and improved handling of rare cases. Lastly, the F-measure framework has been widely tested on diverse classification problems, making it a robust addition to any data analyst's toolkit.