June 19th, 2022: camera-ready paper submission.June 19th, 2022: notification of acceptance.June 1st, 2022: publication of results.May 19th, 2022: test set and open for submissions.Determinants and punctuations are excluded in the evaluation. (2019), the macro-average F1 score of word overlap: we compare each individual prediction against the different human gold standard answers and select the maximum value as system F1 score for that instance the system performance is the macro-average of all those F1 scores. We will also report, following Reddy et al. Q2: ❼uándo vuelven las clases presenciales a todas las escuelas?Īs one of our evaluation metrics, we will measure average Exact Match for all the dataset instances, following the approach of SQuAD (Rajpurkar, 2016). Q1: ❼uántas escuelas rurales hay en Uruguay?Ī1: De las escuelas habilitadas abrieron 344, confirmó a la diaria Limber Santos, director del departamento de Educación Rural del CEIP. Given these possible questions, the expected answers that the system should find are the following: La asistencia, por tanto, llegó a 36% en el primer día. De esas escuelas, cerca de 90 no recibieron alumnos Santos estimó que en la mañana del miércoles 1.030 niños concurrieron a las escuelas, de un total de 3.900 que concurren a las 547 habilitadas y de 2.838 alumnos que tienen matriculadas las 344 escuelas que abrieron. De las 547 escuelas habilitadas abrieron 344, confirmó a la diaria Limber Santos, director del departamento de Educación Rural del CEIP. La situación de La Macana se repitió en varias de las escuelas que abrieron este miércoles. A las 9.00, cuando debían comenzar las clases en la escuela de La Macana, no había ningún niño. De los 28 alumnos que asisten regularmente, 14 habían dicho que no iban a ir y los otros no habían confirmado. Ya estaba instalado el micrófono y el parlante en el patio, habían llegado los inspectores regionales junto con la directora general del Consejo de Educación Inicial y Primaria (CEIP), Irupé Buzzetti, que junto a la prensa local esperaban a los niños. Dos maestras con túnicas blancas y tapabocas esperaban a los alumnos que reanudarían las clases presenciales luego de cinco semanas de conexión virtual. For example, consider the following news text:Ĭomenzaron las clases presenciales en 344 escuelas rurales, con baja asistenciaĪ las 8.45 dos perros paseaban por el patio de la escuela rural 27 de La Macana, en Florida. The expected results are the shortest spans of text that contain the answer, taking into account that some questions could not be answered using the information in the text. Most of the questions in the dataset are about Covid-19 matters, but some of them are also about other topics. Originally, we planned to have two separate corpora for evaluation, but seeing that the texts often contain Covid-19 related news mixed with other topics, we decided to annotate only one set. The training, development and test datasets are a based on a corpus of news in Spanish related to the Covid-19 domain. The systems will get a full news article and a question, and must find the shortest spans of text in the article (if they exist) that answer the question. We propose a task for developing question answering systems that can answer questions based on news articles written in Spanish. In this task we address the problem of answering questions by extracting answers from a set of documents. Each of these stages has its own challenges, and the whole task requires a successful outcome in each of them and in their integration. Open domain question answering involves two main stages: a) obtaining the relevant documents, generally using methods from the Information Retrieval field (IR) (Manning, 2008), possibly one of the most widely studied topics in NLP, with web search engines as their most noticeable product, b) extracting the answer from those documents. Question Answering (QA) is a classical Natural Language Processing task (Jurafsky, 2021), and can be divided into two main categories: semantic analysis, where the question is transformed to a query to a knowledge database and open domain question answering, where, starting from a question written in natural language and a set of documents, the answer to the question is obtained using information retrieval and information extraction techniques. The Codalab page for the competition is available. The competition is over, the evaluation results have been published. This task is part of IberLEF 2022, and is organized by Grupo PLN-UdelaR News Welcome to the shared task QuALES - Question Answering Learning from Examples in Spanish, a task to automatically find answers to questions in Spanish from news text.
0 Comments
Leave a Reply. |