The monthly TIMETOACT GROUP Language Model (LLM) Benchmarks help you choose the best AI models for digital product development.
Based on real benchmark data from our own software products, we re-evaluate each month the performance of different LLM models in addressing specific challenges. We examine specific categories such as document processing, CRM integration, external integration, marketing support, and code generation.
LLM Benchmarks | January 2025
In the first issue of TIMETOACT GROUP LLM Benchmarks V2 we present our new benchmarks and talk about DeepSeek r1.
Highlights:
- Second Generation Benchmark - Early Preview
- DeepSeek r1
- Cost and price development of DeepSeek r1
The benchmark categories in detail
Docs
How well can the model work with large documents and knowledge bases?
CRM
How well does the model support work with product catalogs and marketplaces?
Integrate
Can the model easily interact with external APIs, services and plugins?
Marketing
How well can the model support marketing activities, e.g. brainstorming, idea generation and text generation?
Reason
How well can the model reason and draw conclusions in a given context?
Code
Can the model generate code and help with programming?
Cost
The estimated cost of running the workload. For cloud-based models, we calculate the cost according to the pricing. For on-premises models, we estimate the cost based on GPU requirements for each model, GPU rental cost, model speed, and operational overhead.
Speed
The "Speed" column indicates the estimated speed of the model in requests per second (without batching). The higher the speed, the better.
Archive
Curious about how the scores have evolved? Here you can find all links to previously published leaderboards
Discover our AI workshops for businesses
Whether it's AI fundamentals, Prompt Engineering training, or potential analysis – we offer tailored solutions for every need.
Transform your digital projects with the best AI language models!
Discover the transformative power of the best Large Language Models and revolutionize your business with AI! Stay future-oriented, increase efficiency and secure a clear competitive advantage. We support you in taking your business value to the next level.
