ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Automatic Evaluation of Extract Summaries Using Fuzzy F-Score Measure

Ravindra, G and Balakrishnan, N and Ramakrishnan, KR (2004) Automatic Evaluation of Extract Summaries Using Fuzzy F-Score Measure. In: Fifth International Conference on Knowledge Based Computer Systems (KBCS 2004), Dec 19-22, 2004, Hyderabad, India, pp. 487-497.

[img]
Preview
Postscript
paperid-40.ps

Download (205Kb)
[img]
Preview
PDF
Automatic.pdf

Download (229Kb)

Abstract

This paper describes a fuzzy union based approach for automatically evaluating machine generated extract summaries. The proposed method represents every sentence within a machine generated summary as a fuzzy set. Sentences in the reference summary are assigned membership grades in each of these fuzzy sets using cosine distance measure. Finally Fuzzy union (s-norm operation) is used to compute an F-score measure. Max s-norm and Frank's s-norm operators are discussed and the advantages of using a modified Frank's s-norm operator for evaluating summaries is explained. The proposed evaluation method is compared to ROUGE evaluation system with some test cases.

Item Type: Conference Paper
Keywords: summary evaluation;Fuzzy f-score measure;LCS match;Frank's s-norm operator
Department/Centre: Division of Information Sciences > Supercomputer Education & Research Centre
Date Deposited: 03 Jun 2005
Last Modified: 19 Sep 2010 04:19
URI: http://eprints.iisc.ernet.in/id/eprint/3265

Actions (login required)

View Item View Item