WikiKGQA: Wiki-Based Knowledge Graph Question Answering Challenge

News

April 13, 2026 Codabench competition opened: https://www.codabench.org/competitions/15358

March 20, 2026 Call for Papers published.

Training data already online via QAWiki. For the training data corresponding to the four challenge tracks see codabench.org, a snapshot of the full dataset is available at https://doi.org/10.5281/zenodo.19474009.
Competition hosted at Codabench
Discussion Forum at Google Group
Challenge solution submission: June 19, 2026 (11:59pm, AoE)
Challenge results: July 10, 2026
Paper submissions: July 17, 2026 (11:59pm, AoE)
Paper acceptance notification: August 21, 2026
Workshop days: October 25 OR 26, 2026

Contact

Please contact the Google Mailing list for general questions or email the organizers directly for specific questions.

Call for Papers

WikiKGQA: Wiki-Based Knowledge Graph Question Answering Challenge

Co-located with the 25th International Semantic Web Conference (ISWC 2026), Bari, Italy

Motivation

KGQA has a long tradition in the Semantic Web community, particularly under the Question Answering over Linked Data (QALD) moniker and challenge series. With the rise of LLMs, the performance of state-of-the-art systems has improved dramatically. However, benchmark datasets in the space exhibit well-known limitations, primarily due to the manual cost of creating parallel corpora of natural language questions and structured queries. To address these issues, QAWiki proposes a collaboratively-edited community resource of hand-crafted question-query pairs, focusing currently on English & Spanish questions answered over Wikidata's SPARQL query service. The motivation is two-fold: (1) to evaluate KGQA systems over a hand-crafted collection of multilingual question-query pairs over Wikidata, and (2) to build a community centered around QAWiki, to expand and improve it, and thus expand and improve the resources available for training and benchmarking of KGQA systems.

Task Description

Knowledge Graph Question Answering (KGQA) offers a promising alternative to end-to-end QA systems by obtaining truthful answers from trusted and well-maintained knowledge sources, such as Wikidata, without losing the flexibility of natural language input.
Nevertheless, the creation of large high-quality KGQA benchmarks remains a challenge, with many benchmarks being small, containing errors or repetitive questions that are rarely fixed. Therefore, building on QAWiki, this challenge aims to make the creation of KGQA benchmarks a collaborative effort, just like Wikidata itself. The training data for this challenge is publicly available through QAWiki, consisting of English and Spanish questions, SPARQL queries as well as the relevant metadata.
For evaluation, a private test dataset will be created, which would be integrated with QAWiki after the challenge. Over the years, we aim to build an evolving high-quality KGQA dataset of increasing size.
English and Spanish queries will be ranked separately, each participant can choose to participate only for one or for both languages. The task itself consists of two sub-tasks for each language:

KGQA using only the input question
KGQA using the input question together with the metadata QAWiki provides, i.e., entity and property annotations

In both sub-tasks, the goal is to generate a single SPARQL query to answer that question for the version of Wikidata accessible via the WikiKGQA SPARQL endpoint. As an evaluation metric, Macro QALD F1 scores will be used to evaluate the results of that SPARQL query. The full training data is available under https://doi.org/10.5281/zenodo.19474009 as an extended version of QALD_JSON. For the specific training data corresponding to the four challenge tracks see codabench.org.

Submission Guidelines

We invite submissions of English challenge papers (up to 8 pages excluding references). All submissions should be formatted in the CEUR layout.
Submission link: https://easychair.org/conferences/?conf=wikikgqa2026.
A prerequisite for submitting a challenge paper is to submit challenge solutions of your approach to at least one of the tracks on codabench.org.