4. Experiments

General pipeline

We run a series of experiments to measure the classification performance of the pipeline on the natural scene data set. All experiments follow the following pipeline.

1. SIFT feature generation
2. Codebook generation using K-Means or Iterative Codebook Optimization as described in Wang et al
3. Feature encoding using Vector Quantization in SPM, or LLC
4. Feature pooling either sum pooling (i.e. histogram) in SPM, or max pooling in LLC
5. Training SVM classifier this is a kernel SVM with a histogram intersection kernel, or a linear SVM
6. Testing classifier

In all experiments, we randomly choose 100 images per class as training images, and the rest as test images.

##Variations of pipeline

Here we describe variations of the pipeline that were run on the scene category dataset [link].

• Spatial pyramid matching (SPM). This variation has the second highest performance in Lazebnik, Schmid & Ponce on the scene category dataset. The dictionary is learned via K-Means; the dictionary has 200 entries; descriptors are encoded via VQ; the pyramid has three levels; the SVM is a kernel SVM with a histogram intersection kernel.
• Spatial pyramid + linear SVM. This is the same as SPM except that the SVM is linear.
• LLC. This make heavy use of the methods described in Wang et al. The codebook is learned via K-Means; descriptors are encoded via LLC encoding; the pyramid has three levels; features are max-pooled then L2-normalized; the SVM is linear.
• LLC with LLC dictionary. This is the same as vanilla LLC (above), except that the dictionary is learned via Iterative Codebook Iteration.
• Original LLC source code. This is the source code provided by the authors of LLC. The key difference from our implementation is that this code includes a beta parameter, which regularizes the locality term of the LLC objective function. We run this without codebook optimization.

We train the SVM in SPM by calling libSVM [link] with a precomputed kernel matrix. All other variations described above have linear SVMs trained with liblinear [link].

##Grid Search Here we describe the grid search procedure that we have performed on two variations of the classification pipeline - LLC with and without codebook optimization. Grid search on the latter is done on three parameters as given in the following.

1. Dictionary Size, #atoms = {512, 1024, 2048, 4096}
2. Number of Nearest Neighbors, #NN = {2, 3, 4, 5}
3. Number of Pyramid Levels, #PL = {2, 3, 4}

Grid search on LLC with codebook optimization is done on four parameters. The number of nearest neighbors is set at 3. The new parameters, lambda and sigma, are parameters of the codebook optimization algorithm.

1. Dictionary Size, #atoms = {1024, 2048}
2. Number of Pyramid Levels, #PL = {3, 4}
3. Lambda, λ = {0.0005, 0.001, 0.002, 0.003}
4. Sigma, σ = {0.25, 0.5, 1, 2}

Detailed results of the grid search are provided in the Results section.

##Effect of locality parameter beta To understand the impact of the beta parameter on classification performance, We run the original LLC source code for different dictionary sizes and compare it with our implementation on those dictionary sizes. See Results page for results.