MT EVALUATION: BASIS FOR FUTURE DIRECTIONS
AN NSF-SPONSORED WORKSHOP
(San Diego, 2-3 November 1992)
PROGRAM and Proceedings
Preface—M.Vasconcellos
Monday, 2 November
Announcements and Greetings
Association for Machine
Translation in the
European Association for
Machine Translation—M. King
National Science Foundation—Y.T. Chien
Traditions in the Evaluation of MT—Y. Wilks
International Coordination
International Working Group on MT Evaluation—M. King
EC Evaluation Activities—L. Rolling
Evaluation Surveys
The JEIDA Methodology and Survey—H. Nomura, H. Isahara
The MITRE Methodology and Survey—P. Jordan, J. Benoit, B. Dorr
The DARPA Methodology
Objective—G.Doddington
Presentation—J. White
Participants—P. Brown, L. Carlson, G. Doddington, D. Farwell, E. Hovy, E. Lange,
M. Leon, M. Mandel, S. Nirenburg
Outside commentators—R. Berwick, L. Rolling, M. Vasconcellos
New Directions
It’s Lucky If Concatenation Does Anything—R.
Berwick
Automatic Evaluation of Translation Quality—H. Thompson
Evaluation of the Machine-Aided Voice
Translation System—
Evaluation of Related NLP Research
Message Understanding:
The MUC Methodology—B. Sundheim
Semi-Automated Evaluation
of Japanese and English Output—L. Carlson
The Murasaki Project:
Multilingual Natural Language Understanding—R.
Doerr
Measurement of Human
Performance for Information Extraction—C.
Will, B. Onyshkevych
Tuesday, 3 November
Discussion:
"Apples,
M. Vasconcellos (moderator), R. Dessau, E. Hovy, V. Lawson, C. Miller, M. Nagao, B. Scott, M.Tacelosky
Practical Use and Effect of
the JICST Japanese-English Machine Translation System—K. Tamura,
LOGOS— B. Scott
Methodology for the Computing Research Laboratory’s Multilingual MT System—D. Farwell
Quality Analysis, an
Internal Evaluation Tool at SYSTRAN—E. Lange, J. Homer, L. Gerber
METAL—W. S. Bennett
Approaches to the Evaluation of Japanese-English MT
Criteria for Evaluating the Linguistic Quality of
Japanese-English MT—S. Ikehara
Application of the TOEFL Test to the Evaluation of Japanese-English MT—M. Tomita [abstract]
Evaluation of Lexical Coverage— G. Magnusdottir
Discussion: Assessing the Labor-intensive Elements of MT—M. León (moderator), D. Albisser, M. Baena, L. Childs, G. Magnusdottir
Results of the Warmup Exercises—M. León
Discussion: Future Directions in MT—S. Nirenburg (moderator), W.S. Bennett, B. Dorr, E. Hovy, S. Ikehara, M. King, A. Melby, M. Nagao, M. Tomita, V. Teller
Closing Messages
IAMT—M.
Nagao
AMTA—M. Vasconcellos
Annex A:
Machine
Translation Evaluation Criteria—R.Dessau
Annex B:
Warmup exercises [introduction]
Photocopy of: Machine Translation Evaluation Methodology [
Photocopy of: Appendix 10 of the ALPAC report: An experiment in evaluating the quality of translations