Half Day Event

This special session aims to catalyse progress in Generative AI and Semantic Web Technologies by fostering in-depth discussions and collaborative problem-solving among researchers, practitioners, and industry leaders. The session will feature three dedicated slots, each focusing on a specific challenge or theme within the broader topic area. Through a combination of short paper presentations, focused discussions, and working groups, participants will share insights, identify open problems, and initiate collaborative projects. We encourage participation from researchers at all career stages.

Session Format

The session is organised in three slots, each addressing a specific research agenda:

• Slot 1: Benchmarking & Reproducibility in Gen AI for Knowledge Engineering
• Slot 2: Bridging the Gap: Building Knowledge-Enhanced Gen AI
• Slot 3: Spotlight on Applications and Success Stories in Gen AI + Semantic Web

Each of the three slots will follow a similar structure designed to maximise engagement and facilitate tangible outcomes. This will include a combination of the following activities (please note that timings are purely indicative).

1. Slot kickstarter: short paper overview (15 minutes): A selection of short papers, submitted in advance, will be briefly pitched by the panellist to provide a concise overview of relevant research, tools, or datasets, that will kickstart the discussion.
2. Focused discussion (60 minutes): Moderated discussions will delve deeper into the presented topics, allowing participants to share perspectives, raise questions, and identify key challenges. Depending on the slot, these may include hands-on activities, brainstorming sessions, or collaborative exercises.
3. Follow-up (15 minutes): The idea of follow-up is for members to form and coordinate groups (with a leader) that meet up later in the conference (at coffee, lunch, dinner, etc.) and follow up on concrete actions. This is expected to generate concrete outputs, such as white papers, research agendas, or project proposals.

Expected Outcomes

We anticipate the following outcomes from the special session:

• White Papers: Each slot will lay the groundwork for a collaboratively authored white paper summarising key findings, open problems, and potential research directions. We anticipate publishing the white papers as part of a special issue in a journal (to be defined).
• Research Agendas: The discussions and workshops are expected to generate research agendas outlining priority areas for future investigation.
• Collaborative projects and open repositories: We hope the session will spark new collaborations and initiatives to address the identified challenges.

Submission Method

We welcome short papers (4 pages max) that define 1) one or more benchmark proposals for some specific tasks, preferably if accompanied by a GitHub repository with additional material, 2) describe methods for the enrichment of LLMs with ontologies and KGs, or 3) present successful deployments of Generative AI with Semantic web technologies. More information is available in the Call for Papers below.

Slot Topics

Slot 1: Benchmarking & Reproducibility in Gen AI for Knowledge Engineering
This slot addresses the critical need for standardised benchmarks and evaluation methodologies in the rapidly evolving landscape of GenAI-enhanced Knowledge Engineering (KE). We will explore challenges in measuring the effectiveness of LLM-driven KE workflows, discuss best practices for ensuring reproducibility, and collectively identify datasets and tasks that can serve as community resources.

In this slot, we focus on the proposal of benchmarks for validating Knowledge Engineering approaches, including the use of LLMs as part of their definition. These may include, but are not limited to:

• Tasks,
• Datasets and benchmarks,
• Methods (automated, semi-automated, or human in the loop) with associated measures,
• Frameworks and libraries for evaluation.

Of particular interest (but not exclusively) are benchmarks for the requirement definition phase of KG and OE lifecycles, e.g., definition or re-engineering of competency questions and/or use cases.

Slot 2: Bridging the Gap: Building Knowledge-Enhanced Gen AI
The integration of knowledge graphs and language models holds promise for enhancing AI capabilities, but significant challenges remain. In particular, there have been a number of proposals for enriching LLMs with curated knowledge in the form of KGs or ontologies for:

• KG-based Retrieval augmented generation:
  • Fine-tuning of LLMs;
  • Pre-training; and
  • Inference computation;
• Limiting or preventing hallucinations;
• Establishing safety measures or guardrails for LLMs;
• Editing LLM models.

This slot will delve into open problems including (but not exclusively): efficient knowledge injection, reasoning over structured and unstructured information, handling knowledge conflicts, and ensuring explainability in KG-augmented LLM outputs. We aim to foster a collaborative environment where researchers can share their approaches, identify promising directions, and collectively chart a path toward more knowledgeable and robust AI systems.

Slot 3: Spotlight on Applications and Success Stories in Gen AI + Semantic Web
This slot will showcase applications of Generative AI and Semantic Web Technologies in real-world scenarios. We will explore the current state of the art, identify ongoing challenges, and discuss the future research directions that will likely drive innovation and adoption. The session aims to foster collaboration between academia and industry, potentially leading to the definition of new challenges for future ISWC editions. We encourage participation from researchers and industry practitioners working on innovative applications of these technologies.

Topics for the panel discussion in Slot 3 include:

• Successful applications and use cases of Generative AI and Semantic Web Technologies. We particularly welcome discussions on the following questions: What challenges were encountered,will likely and how were they overcome? How was the interaction with the stakeholders coordinated?
• Industry needs and perspectives: Which specific areas of research and development are most important for industry adoption? What kind of support or collaboration is needed from academia?
• Future trends and challenges: What emerging trends are likely to shape the future of Generative AI and Semantic Web Technologies? What are the key challenges that need to be addressed to realise their full potential?

To kickstart each slot, we invite the submission of short papers (4 pages max, excluding references) addressing relevant topics within the scope of the special session. Papers should specifically address one of the specific slots and focus on presenting novel research, tools, and datasets or identifying open problems and challenges. We highly welcome papers that are accompanied by a GitHub repository with further relevant materials. Papers can be submitted via Easychair and are single-anonymous.

Submissions should be formatted using the CEURART 1-column style. The title should use the emphasising capitalised style, and the paper should have no page numbers.

Papers will be assessed for relevance, clarity, novelty, and potential impact. With respect to novelty, we will prioritise papers that present a novel perspective or insight rather than necessarily presenting novel research contributions.

Organising Committee

Reham Alharbi, University of Liverpool

Jacopo de Berardinis, University of Liverpool

Paul Groth, University of Amsterdam

Albert Meroño-Peñuela, King’s College London

Elena Simperl, King’s College London

Valentina Tamma, University of Liverpool