A Blocking Framework for Entity Resolution in Highly Heterogeneous Information Spaces

Georgios Papadakis, Ekaterini Ioannou, Themis Palpanas, Claudia Niederée, and Wolfgang Nejdl.
In IEEE Transactions on Knowledge and Data Engineering (TKDE) 25(12), pages 2665-2682, 2013.
pdf

Abstract

In the context of Entity Resolution (ER) in highly heterogeneous, noisy, user-generated entity collections, practically all block building methods employ redundancy in order to achieve high effectiveness. This practice, however, results in a high number of pairwise comparisons, with a negative impact on efficiency. Existing block processing strategies aim at discarding unnecessary comparisons at no cost in effectiveness.

In this paper, we systemize blocking methods for Clean-Clean ER (an inherently quadratic task) over highly heterogeneous information spaces (HHIS) through a novel framework that consists of two orthogonal layers: the effectiveness layer encompasses methods for building overlapping blocks with small likelihood of missed matches; the efficiency layer comprises a rich variety of techniques that significantly restrict the required number of pairwise comparisons, having a controllable impact on the number of detected duplicates. We map to our framework all relevant existing methods for creating and processing blocks in the context of HHIS, and additionally propose two novel techniques: Attribute Clustering Blocking and Comparison Scheduling. We evaluate the performance of each layer and method on two large-scale, real-world data sets and validate the excellent balance between efficiency and effectiveness that they achieve.

Bibtex

@inproceedings{journal/tkde/PapadakisIPNN,
     author = {George Papadakis and Ekaterini Ioannou and Themis Palpanas and Claudia Nieder{\'e}e and Wolfgang Nejdl},
     title ={A Blocking Framework for Entity Resolution in Highly Heterogeneous Information Spaces},
     booktitle = {TKDE},
     volume = {25},
     number = {12},
     year = {2013},
     pages = {2665-2682}
}