20Y-CHEC: 20 Years of Confusion in Human Evaluation

This is the website relating to our INLG 2020 paper:

Howcroft, Belz, Clinciu, Gkatzia, Hasan, Mahamood, Mille, van Miltenburg, Santhanam, & Rieser. (2020). “Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definitions”. ACL Anthology

PDF

You will currently find the annotations and our analysis scripts as well as the highlighted PDFs associated with our annotations in the GitHub repo.

Please open an issue on the GitHub repo if you have difficulty reproducing our analyses or if you find problems with our code or data.