Experiments with Real Estate Usage Data

As our first experiment with the Reality data, we compared the hit ratio of the semantically enhanced approach at different combination parameters for both the complete data set and the 40-dimension SVD data set. These results are depicted in Figure 7. In this case we only focused on the Top 10 recommendations generated by the algorithm.

Figure 7: Impact of semantic combination parameter for the top 10 recommendations in the real estate usage data

The results suggest similar conclusions to those observed in the movie data set. First, in general, singular value decomposition has an even more dramatic impact in this case; more so when the focus is shifted to the semantic information ($\alpha$ close to 0) as opposed to usage data. In fact, we see that in this data set, without performing SVD on the semantic attribute matrix, the combined approach does not improve accuracy when compared to pure usage-based recommendations. This may be an indication that many different attributes contribute to the type of property in which visitors show interest. Applying SVD results in a smaller number of latent factors by combining multiple attributes. These factors, individually, may be more predictive in determining user interests than the more fine grained attributes. As can be seen, with SVD the semantic approach results in significant improvements over both usage-only and content-only recommendations, particularly at a combination parameter $\alpha = 0.7$.

Next, we measured the hit ratio improvement achieved by our algorithm (with semantic combination parameter $\alpha = 0.7$), over the two boundary cases when only usage-based similarity ($\alpha = 1$) or only semantic similarity ($\alpha = 0$) are used to generate recommendations. Figure 8 depicts these results. The combined similarity measure achieved between 20% to 37% improvement over the semantic-only recommendations (i.e., over pure content-based filtering). In the case of usage-based recommendations, we observe that with recommendation sets of size less than 20, the combined approach always achieved better Hit Ratio. The improvement is particularly significant for small values of $N$. Indeed, in real situations, we are interested in few, but accurate recommendations, and this is precisely where the semantically enhanced approach seems to provide the most advantage.

Figure 8: Improvement of the semantically enhanced recommendations over content-only and usage-only recommendations