We’ve written earlier about the first round of our Enterprise RAG Challenge. In this friendly challenge different AI Assistants competed in answering questions based on the annual reports of public companies. Teams from the different countries competed in building such assistants. There were even a few commercial solutions in the mix.
Datum
04.10.2024
content.autor.writtenBy
You can find code for the solutions marked with “TTA” (TimeToAct) on Github. Solutions include description of the approach, code itself, sometimes even the log of failed experiments.
Code is real, without any cleanups and beautification.
In short:
Daniel - winning and surprisingly simple solution that used checklist pattern with structured outputs. First place.
→ Check out Daniel's solution
Felix - multi-agent solution with ChatGPT-4o. 12th place.
→ Check out Felix' solution
Maria - solution using OpenAI Assistants API. 13th place.
→ Check out Maria's solution
Pedro - locally-capable solution using openchat-3.5-0106. ninth place.
→ Check out Pedro's solution
What's next?
Next round of Enterprise RAG Challenge will take place later this fall, with a bigger audience. The exact time depends on the organisation process within the TIMEOTACT GROUP. Perhaps, around November.
In the next round, question generator will be rebalanced, so that:
There are less questions that don't have an answer (agents must respond N/A to these)
There is more variability in the questions, so that “brute force” approach with checklists+structured outputs will not be able to win the competition so easily.
Questionnaires for the participants will also be reworked, so that we all together could learn more about approaches for Enterprise AI that work well in practice.