An Experimental Analysis of the Pattern Explosion

Abstract

Although Frequent Itemset Mining is a classical Data Mining technique, the causes of the pattern explosion – one of its major challenges – have never been thoroughly researched. We perform an experimental analysis of the causes of the pattern explosion. Several experiments are performed on five selected datasets. The experiments show that similar transactions usually support similar patterns, similar patterns however do not necessarily describe similar data. In the first case the correlation is strong, yet in the second case only a weak correlation exists. We furthermore show that it is possible in many patterns to swap items for other particular items without influencing the data that is described much. This shows that in many cases, there is little interaction between the items and at least not all of their relations are significant.