ELRA
- HLT Evaluation Workshop
Program
DAY 1: 1st December 2005
Introduction:
Bente Maegaard &
Khalid Choukri: Celebrating ELRA’s 10th anniversary,
1995-2005: workshop dedicated to human language technology (HLT) evaluation...;
14pp. [PDF from PPT, 61KB]
Invited speaker:
Mats
Ljungqvist: Human language technology
initiatives: EU research programmes FP6 and FP7; 23pp. [PDF from PPT, 186KB]
Michael Rosner: Towards HLT evaluation at
the University of Malta; 19pp. [PDF from PPT, 128KB]
Maghi King: Evaluation in human language technology; 32pp.
[PDF from PPT, 107KB]
Keith J. Miller: Evaluation principles and objectives: ISLE, EAGLES
(a play in three acts); 48pp. [PDF from PPT, 901KB]
Carol Peters: Evaluating information retrieval systems: focusing
on systems for cross language information retrieval; 47pp. [PDF from PPT,
561KB]
Christian
Fluhr: Criticisms and proposed evolutions of the
information retrieval evaluation paradigm; 7pp. [PDF from PPT, 32KB]
Patrick Paroubek: Evaluation of text analysis core technologies. Two
successful examples: evaluating POS taggers and parsers for French; 35pp.
[PDF from PPT, 288KB]
Anja
Belz & Ehud Reiter: Evaluation in NLG;
35pp. [PDF from PPT, 2032KB]
Tony Hartley: MT evaluation – the little
we know; 11pp. [PDF from PPT, 74KB]
Andrei Popescu-Belis: Some questions from the discussant [of paper
by Tony Hartley on MT evaluation]; 8pp. [PDF from PPT, 48KB]
Nick
Campbell: Speech synthesis evaluation;
14pp. [PDF from PPT, 57KB]
Harald Höge: Evaluation of speech synthesis technology; 2pp.
[PDF from PPT, 16KB]
Edouard
Geoffrois: Speech recognition evaluation:
practices and issues; 20pp. [PDF from PPT, 148KB]
DAY 2: 2nd December 2005
Gianni Lazzari: Speech to speech translation; 110pp. [PDF of
PPT presentation, 1091KB]
Jean Carletta: Some problems in evaluating multimodal systems;
27pp. [PDF from PPT, 436KB]
Gregor
Thurmair: Automatic means of MT evaluation;
40pp. [PDF from PPT, 2974KB]
Evaluation initiatives:
CELCT -- Amedeo Cappelli:
CELCT: Center for the Evaluation of Language
and Communication Technologies; 13pp. [PDF from PPT, 58KB]
CST -- Bente Maegaard: Evaluation and validation activities at CST;
6pp. [PDF from PPT, 93KB]
ELRA
-- Khalid Choukri: An evaluation infrastructure
for HLT in Europe – part 1; 8pp. [PDF from PPT, 34KB]
DFKI
-- Anthony Jameson: The DFKI evaluation center
for language technology systems: from metric optimization to useful systems;
3pp. [PDF from PPT, 708KB]
French Ministry of Research -- Joseph Mariani: From basic research to innovative applications
through evaluation.; 15pp. [PDF from PPT, 95KB]
ISSCO
-- M.King: ISSCO and evaluation 1988-2005; 8pp.
[PDF from PPT, 94KB]
Christopher
Cieri: Evaluation initiatives: US activities in
HLT evaluation; 7pp. [PDF from PPT, 237KB]
Khalid
Choukri: An evaluation infrastructure for HLT
in Europe – part 2; 23pp. [PDF from PPT, 100KB]
Christopher
Cieri: HLT evaluation: role of data centers;
6pp. [PDF from PPT, 195KB]
Wrap up and further actions
Panel [not available]