Based on real benchmark data from our own software products, we re-evaluate each month the performance of different LLM models in addressing specific challenges. We examine specific categories such as document processing, CRM integration, external integration, marketing support, and code generation.
![](/TTA_AT/Blog/AI/31408/image-thumb__31408__ImageSlider-modern-bg-default/die-besten-sprachmodelle-llm.jpg)
The best language models for digital products
LLM Benchmarks | June 2024
You can expect the following insights & highlights:
The elephant in the room - Claude 3.5 Sonnet and artifacts feature
Confidential computing - how it can make AI more secure and cost-effective for companies
The trend towards small and powerful LLMs that can be operated locally
How well can the model work with large documents and knowledge bases?
How well does the model support work with product catalogs and marketplaces?
Can the model easily interact with external APIs, services and plugins?
How well can the model support marketing activities, e.g. brainstorming, idea generation and text generation?
How well can the model reason and draw conclusions in a given context?
Can the model generate code and help with programming?
The estimated cost of running the workload. For cloud-based models, we calculate the cost according to the pricing. For on-premises models, we estimate the cost based on GPU requirements for each model, GPU rental cost, model speed, and operational overhead.
The "Speed" column indicates the estimated speed of the model in requests per second (without batching). The higher the speed, the better.
Archive
Curious about how the scores have evolved? Here you can find all links to previously published leaderboards
Transform your digital projects with the best AI language models!
Discover the transformative power of the best LLM and revolutionize your digital products with AI! Stay future-oriented, increase efficiency and secure a clear competitive advantage. We support you in taking your business value to the next level.
![](/GROUP_Share/Personen/Mitarbeitende/TIMETOACT%20%C3%96sterreich/31352/image-thumb__31352__ContactformTextContact-contact-person/ch.ha.jpg)
Christoph Hasenzagl
![Christoph Hasenzagl](/GROUP_Share/Personen/Mitarbeitende/TIMETOACT%20%C3%96sterreich/31352/image-thumb__31352__MediumTileGrid-item-author/ch.ha.jpg)
![](/TTA_AT/Blog/AI/31380/image-thumb__31380__MediumTileGrid-item-main-s1-3/fehler-ki-entwicklung.png)
Common Mistakes in the Development of AI Assistants
![Christoph Hasenzagl](/GROUP_Share/Personen/Mitarbeitende/TIMETOACT%20%C3%96sterreich/31352/image-thumb__31352__MediumTileGrid-item-author/ch.ha.jpg)
![](/TTA_AT/Blog/AI/31411/image-thumb__31411__MediumTileGrid-item-main-s1-3/tipps-zur-entwicklung-von-ki-assistenten.jpg)
8 tips for developing AI assistants
![](/TTA/Referenzen/TRUMPF/30095/image-thumb__30095__MediumTileGrid-item-main-s1-3/Trumpf_Logo_breiter.png)
Standardized data management creates basis for reporting
![Martin Lange](/GROUP_Share/Personen/Mitarbeitende/TIMETOACT%20Deutschland/19113/image-thumb__19113__MediumTileGrid-item-author/Martin%20Lange%20-%20TTA.jpg)
![Lizenzmanagement hat nicht nur Relevanz für das Thema Compliance, sondern kann auch Kosten und Risiken minimieren. Checkliste als Symbol für die verschiedenen To Dos im Bereich Lizenzmanagement](/TTA/Blog/SAM/Lizenzmanagement%20Wissen/19102/image-thumb__19102__MediumTileGrid-item-main-s1-3/Lizenzmanagement-im-griff-teaser.jpg)
License Management – Everything you need to know
![Cloud Pak for Data – Test-Drive für einen ersten Eindruck Headerbild zu Cloud Pak for Data – Test-Drive](/TTA/Technologien%20neu/IBM/IBM%20CP4D%20PoV%20Landing%20Page/16729/image-thumb__16729__MediumTileGrid-item-main-s1-3/Header-Test-Drive-IBM-Cloud-Pak.jpg)
IBM Cloud Pak for Data – Test-Drive
![Mit IBM Cloud Pak for Data Accelerator schnelle Umsetzung von Anwendungsfällen Headerbild zu IBM Cloud Pak for Data Accelerator](/TTA/Technologien%20neu/IBM/IBM%20CP4D%20Accelerator/17004/image-thumb__17004__MediumTileGrid-item-main-s1-3/Header%20Accelerator.jpg)
IBM Cloud Pak for Data Accelerator
![](/TTA/Referenzen/Funke/28909/image-thumb__28909__MediumTileGrid-item-main-s1-3/200316_FUNKE_Logo_cmyk_red-_1_.jpg)
Managed service support for optimal license management
![IBM Watson Knowledge Studio Headerbild zu IBM Watson Knowledge Studio](/TTA/Technologien%20neu/IBM/Natural%20Language%20Processing/18636/image-thumb__18636__MediumTileGrid-item-main-s1-3/IBM%20Watson%20Knowledge%20Studio.png)
IBM Watson Knowledge Studio
![IBM Watson Assistant Headerbild zu IBM Watson Assistant](/TTA/Technologien%20neu/IBM/Natural%20Language%20Processing/18632/image-thumb__18632__MediumTileGrid-item-main-s1-3/IBM%20Watson%20Assistant.jpg)
IBM Watson Assistant
![IBM Watson Discovery Headerbild zu IBM Watson Discovery](/TTA/Technologien%20neu/IBM/Natural%20Language%20Processing/18635/image-thumb__18635__MediumTileGrid-item-main-s1-3/IBM%20Watson%20Discovery.png)
IBM Watson Discovery
![](/TTA/Referenzen/KI.TEST/23128/image-thumb__23128__MediumTileGrid-item-main-s1-3/KI-TEST_Logo_mit_Schriftzug_rechts.png)
Interactive online portal identifies suitable employees
![](/TTA/Referenzen/AGAPLESION/21560/image-thumb__21560__MediumTileGrid-item-main-s1-3/AGAPLESION_Dachmarke_RGB.png)
Modernes Business Intelligence und Data Warehouse System
![](/TTA/Referenzen/GdP/14806/image-thumb__14806__MediumTileGrid-item-main-s1-3/Header-Freizeitpark.png)
Flexibility in the data evaluation of a theme park
![Talend Application Integration Headerbild Talend Application Integration](/TTA/Technologien%20neu/Talend/19857/image-thumb__19857__MediumTileGrid-item-main-s1-3/Talend-Application-Integration.jpg)
Talend Application Integration / ESB
![](/TTA_AT/Stock/Adobe/30659/image-thumb__30659__MediumTileGrid-item-main-s1-3/AdobeStock_211443771.jpg)
Managed Service: Mailroom
![Digitale Planung, Forecasting und Optimierung zur Planung und Analyse von Geschäftsprozessen. Headerbild zu Digitale Planung, Forecasting und Optimierung](/TTA/Kompetenzen/Business%20Intelligence/Digitale%20Planung/9004/image-thumb__9004__MediumTileGrid-item-main-s1-3/header-digitale-planung.jpg)
Demand Planning, Forecasting and Optimization
![](/GROUP_Share/Stockbilder/Gekauft/6361/image-thumb__6361__MediumTileGrid-item-main-s1-3/AdobeStock_283492262.jpg)
Application Integration & Process Automation
![](/TTA_AT/Servicebeschreibungen/27963/image-thumb__27963__MediumTileGrid-item-main-s1-3/Cloud-Banner.jpg)
Cloud Transformation & Container Technologies
![IBM Cloud Pak for Data als integrierte Lösung für die Implementierung einer vollständigen Analyticsumgebung Headerbild IBM Cloud Pak for Data](/TTA/Technologien%20neu/IBM/Cloud%20Pak/17509/image-thumb__17509__MediumTileGrid-item-main-s1-3/Header-Cloud-Pak-for-Data.jpg)
IBM Cloud Pak for Data
![IBM Cloud Pak for Automation zur Automatisierung von Unternehmensschritten Headerbild zu IBM Cloud Pak for Automation](/TTA/Technologien%20neu/IBM/Cloud%20Pak/17510/image-thumb__17510__MediumTileGrid-item-main-s1-3/Header-Cloud-pak-for-automation.jpg)
IBM Cloud Pak for Automation
![IBM Cloud Pak for Application für flexible und kostengünstige Modernisierung von Anwendungen Haderbild zu IBM Cloud Pak for Application](/TTA/Technologien%20neu/IBM/Cloud%20Pak/17511/image-thumb__17511__MediumTileGrid-item-main-s1-3/Header-Cloud-pak-for-application.jpg)
IBM Cloud Pak for Application
![IBM Cloud Pak for Data System zur Bereitstellung der Hardware für Cp4D Headerbild IBM Cloud Pak for Data System](/TTA/Technologien%20neu/IBM/Cloud%20Pak/18440/image-thumb__18440__MediumTileGrid-item-main-s1-3/Header-Cloud-pak-for-data-system.jpg)
IBM Cloud Pak for Data System
![Mit Business Intelligence sind Unternehmensdaten fest im Griff. Navigationsbild zu Business Intelligence](/TTA/Kompetenzen/Business%20Intelligence/17541/image-thumb__17541__MediumTileGrid-item-main-s1-3/Nav-Business-Intelligence.jpg)
Business Intelligence
![Mit Data Governance Consulting stellen Sie Nachvollziehbarkeit, Qualität und Schutz Ihrer Daten sicher. Headerbild zu Data Governance Consulting](/TTA/Kompetenzen/Business%20Intelligence/3%20Governance/15226/image-thumb__15226__MediumTileGrid-item-main-s1-3/Header-Data-Governance-Consulting.jpg)
Data Governance
![Visualisierung gewonnener Daten mit Dashboards und Reports Headerbild zu Dashboards und Reports](/TTA/Kompetenzen/Business%20Intelligence/6%20Dashboards%20und%20Reports/13955/image-thumb__13955__MediumTileGrid-item-main-s1-3/header-dashboards-reports.jpg)
Dashboards & Reports
![Mit Data Integration Service und Consulting Daten in ein einheitliches und einfach abfragbares Schema bringen. Teaserbild zu Data Integration Service und Consulting](/TTA/Kompetenzen/Business%20Intelligence/2%20Data%20Integration/15204/image-thumb__15204__MediumTileGrid-item-main-s1-3/Header-Data-Integration.jpg)
Data Integration, ETL and Data Virtualization
![Datenmanagement mit IBM DB2 Headerbild zu IBM DB2](/TTA/Technologien%20neu/IBM/Produkte/17357/image-thumb__17357__MediumTileGrid-item-main-s1-3/Header-IBM-DB2.jpg)
IBM Db2
![Informationsintegration mit IBM DataStage Headerbild zu IBM DataStage](/TTA/Technologien%20neu/IBM/Produkte/17398/image-thumb__17398__MediumTileGrid-item-main-s1-3/Header-InfoSpere%20Information%20Server.jpg)
IBM InfoSphere Information Server
![Bessere Entscheidungen mit IBM Decision Optimization treffen Headerbild zu IBM Decision Optimization](/TTA/Technologien%20neu/IBM/Produkte/4148/image-thumb__4148__MediumTileGrid-item-main-s1-3/Header-IBM-Desicion-Optimization.jpg)
Decision Optimization
![](/TTA_AT/Servicebeschreibungen/27968/image-thumb__27968__MediumTileGrid-item-main-s1-3/Banner-Softwareentwicklung.jpg)
Software, Mobile and Web App Development
![Talend Data Integration Headerbild Talend Data Integration](/TTA/Technologien%20neu/Talend/19859/image-thumb__19859__MediumTileGrid-item-main-s1-3/Talend-Data-Integration.jpg)
Talend Data Integration
![Mit Data Science wertvolle versteckte Informationen extrahieren. Navigationsbild zu Data Science](/TTA/Kompetenzen/Data%20Science/17544/image-thumb__17544__MediumTileGrid-item-main-s1-3/Nav-Data-Science.jpg)
Data Science, Artificial Intelligence and Machine Learning
![Operationalisierung von Data Science (MLOps): Integation von KI-Prozesse Headerbild zu Operationalisierung von Data Science (MLOps)](/TTA/Kompetenzen/Data%20Science/18869/image-thumb__18869__MediumTileGrid-item-main-s1-3/Header-MLOps.jpg)
Operationalization of Data Science (MLOps)
![Zur Konzeption individueller BI-Lösungen nutzen wir Workshops. Header Konnzeption individueller Business Intelligence Lösungen](/TTA/Kompetenzen/Business%20Intelligence/1%20Architecture/13891/image-thumb__13891__MediumTileGrid-item-main-s1-3/Header-Konzeption-BI-L%C3%B6sungen.jpg)
Conception of individual Analytics and Big Data solutions
![Mit Microsoft Azure in die Cloud Headerbild zu Microsoft Azure](/TTA/Technologien%20neu/Microsoft/20089/image-thumb__20089__MediumTileGrid-item-main-s1-3/header-azure.jpg)
Microsoft Azure
![Mehr analytische Möglichkeiten mit Microsoft Power BI Headerbild zu Microsoft Power BI](/TTA/Technologien%20neu/Microsoft/20845/image-thumb__20845__MediumTileGrid-item-main-s1-3/header-power-bi-verlauf.jpg)
Microsoft Power BI
![IBM Netezza Performance Server als optimierte Datenbank Headerbild zu IBM Netezza Performance Server](/TTA/Technologien%20neu/IBM/Produkte/17403/image-thumb__17403__MediumTileGrid-item-main-s1-3/Header-IBM%20Netezza.jpg)
IBM Netezza Performance Server
![Einfache Implementierung mit IBM Watson Studio Headerbild zu IBM Watson Studio](/TTA/Technologien%20neu/IBM/Produkte/6495/image-thumb__6495__MediumTileGrid-item-main-s1-3/header-ibm-watson-studio.jpg)
IBM Watson Studio
![Talend Data Fabric Headerbild zu Talend Data Fabric](/TTA/Technologien%20neu/Talend/19860/image-thumb__19860__MediumTileGrid-item-main-s1-3/Talend-Data-Fabric.jpg)
Talend Data Fabric
![Talend Real-Time Big Data Platform Headerbild zu Talend Real-Time Big Data Platform](/TTA/Technologien%20neu/Talend/19858/image-thumb__19858__MediumTileGrid-item-main-s1-3/Talend-Real-Time-Big-Data.jpg)
Talend Real-Time Big Data Platform
![Datenanalyse mit IBM SPSS Headerbild für IBM SPSS](/TTA/Technologien%20neu/IBM/Produkte/17437/image-thumb__17437__MediumTileGrid-item-main-s1-3/Header-IBM%20SPSS.jpg)
IBM SPSS Modeler
![Datenverwaltung mit IBM Watson® Knowledge Catalog Headerbild zu IBM Watson® Knowledge Catalog](/TTA/Technologien%20neu/IBM/Produkte/17393/image-thumb__17393__MediumTileGrid-item-main-s1-3/Header-Information%20Governance%20Catalog.jpg)
IBM Watson® Knowledge Catalog/Information Governance Catalog
![Reporting und Analyse mit IBM Cognos Analytics 11 Headerbild zu IBM Cognos Analytics 11](/TTA/Technologien%20neu/IBM/Produkte/17214/image-thumb__17214__MediumTileGrid-item-main-s1-3/Header-IBM-Cognos-Analytics-Titel.jpg)
IBM Cognos Analytics 11
![Analyse und Planung mit IBM Planning Analytics mit Watson Headerbild zu IBM Planning Analytics mit Watson](/TTA/Technologien%20neu/IBM/Produkte/17432/image-thumb__17432__MediumTileGrid-item-main-s1-3/Header-IBM%20Planning%20Analytics%20mit%20Watson.jpg)
IBM Planning Analytics mit Watson
![](/TTA/Technologien%20neu/Microsoft/21293/image-thumb__21293__MediumTileGrid-item-main-s1-3/header-microsoft-azure-synapse-analytics.jpg)
Microsoft Azure Synapse Analytics
![Analytische Lösungen mit Microsoft SQL Server Headerbild zu Microsoft SQL Server](/TTA/Technologien%20neu/Microsoft/20073/image-thumb__20073__MediumTileGrid-item-main-s1-3/header-sql-server.jpg)
Microsoft SQL Server
![](/TTA/Referenzen/Schw%C3%B6rerHaus/11787/image-thumb__11787__MediumTileGrid-item-main-s1-3/schwoererhaus_teaser.png)
The digital customer file with IBM Content Manager
![](/TTA/Referenzen/Rottendorf/19134/image-thumb__19134__MediumTileGrid-item-main-s1-3/Rottendorf_Pharma_Logo.svg.png)
Automation lays the foundation for smooth archive changeover
![](/TTA/Referenzen/GLS/16218/image-thumb__16218__MediumTileGrid-item-main-s1-3/logo-gls.png)
Support for IBM software licenses and ILMT for GLS
![](/TTA/Referenzen/Schweizerische%20Bundesbahnen/25374/image-thumb__25374__MediumTileGrid-item-main-s1-3/SBB%20Logo%20Teaser.png)
Interactive stock map for Schweizerischen Bundesbahnen
![](/TTA/Referenzen/Pfalzwerke/28601/image-thumb__28601__MediumTileGrid-item-main-s1-3/Pfalzwerke%20Aktiengesellschaft%20Logo.png)
Managing sensitive data through digital personnel files
![](/TTA/Referenzen/Sunrise/24867/image-thumb__24867__MediumTileGrid-item-main-s3-3/Sunrise%20Header.jpg)