Do we find value in LLMs?
Or is it more about their integration, platforms and systems that help to utilize them right and create the value?
The past month was fast over. I had the opportunity to speak at DevOpsDays Ukraine: AI Chapter, Container Days Hamburg, and gave a talk at the DKB Tech Conference. I also attended Bits & Pretzels, soaking up some inspiration and finding some reality. For me, it was interesting to see how those topics I’m talking about, like AIDPs and Platform Engineering, are coming together with the business demands of companies and what leading AI companies are stating at the stages.
Platform Engineering, AIDPs and the future of AI Engineering
The future of AI is unpredictable. While celebrated as the next industrial revolution, the reality check shows that most AI (primarily GenAI) introductions are failing, for now. As companies are hunting for the sexy use cases, the successful projects are telling a different story: replacing and automating the boring back-office tasks is the way to win.
And while we are waiting for the next awesome LLM taking over LinkedIn and YouTube, before we will forget about it within two weeks, this trend pushes us to new limits. Systems are reaching new levels of complexity and scale, and require running applications in practically auto mode.
This is a good point to talk about a recent observation, paired with the actual statements of the big AI companies: It will become harder and harder to improve the performance of an AI model. LLMs are reaching their limits, tied up in a shortage of quantitative and qualitative data, computing power, and the current state of technology. As Marjorie Janiewicz from Mistral says: you get more out of your LLMs through the systems and ecosystem it runs in, than to wait for the next 1% improvement by training. Aleph Alpha directly shows this by becoming more and more a platform provider rather than having its own good models. And surprisingly, Thinking Machine Labs, the new company founded by the former CTO of OpenAI, Mira Murati, has launched a product called Tinker that provides an API for fine-tuning open models.
Since I have heard this once, I also don’t get tired of repeating that: It’s not about the model, it is about the platform. The next best model might perform 1-2% better in the evaluation. However, to make it useful and valuable for you and your company, everything depends on the platform —the system it runs on.
An AIDP, an Internal Development Platform for AI Engineering, is the way to go for building environments that can meet any business demand, while providing a solid tooling to avoid high cognitive load from the very rare individuals who are actual AI experts.
Entrepreneurship & Pretzels
The Bits & Pretzels is for me a yearly reminder about the number of young people working on their ideas, products and dreams. It is an engaging atmosphere that calls for action, also thanks to the MCs who are moderating the stages.
I liked the insights and learning more about their history with stable diffusion, particularly from German pioneers like Black Forest Labs. While the LLM companies are close to hitting the end of available data, BFL has a practically unlimited source of relevant data. Images and videos come every time with new perspectives, focus, and details, even if it is a hundred or a thousand times the exact same animal. Paired with their research, they are just standing at the beginning of their journey.
With Mistral and Hugging Face, two industry leaders shared their perspectives on the market and their current journeys. Again, it’s the platform, not the model ;)
However, I also sometimes just wonder why. On the stages, the big “startups” of the continent, in the hallways 40 ways to get some money, and in the caves new startups presenting their products. For me missing are the end users, the people who use those products. Why? A startup without users typically doesn’t receive funding.
On the other hand, nothing new to learn from the big startups; those are enterprises with millions and millions in debt. And the same questions from the moderator always lead to very similar answers: What does AI do to your business? How do you see AI? What are your challenges with AI? …
I think the bits need to change the format. It’s just about celebrating yourself.
Sovereign Cloud Community
I can’t stop making advertisements for our sovereign cloud community. A regional yet international community for sovereignty in data, cloud technology and AI. We just planned the next session until the end of the year, and we are lucky to have found hosts for Poland and France.
We aim to provide a place where we can have a valuable exchange about sovereignty, without the political or commercial bullshit. What works, what's your experience, what are your learnings? That’s what we want to talk about. Join us at https://www.sovereigncloud.community. We now have a Zulip server, and our latest videos are up on YouTube.
Sources of the week
Stanford Free Online Courses - Overview of Transformers
From IDP to AIDP: Evolving Your Platform for the Machine Learning Age - my talk from Container Days
Tools to check out
TmuxAI - AI-Powered, Non-Intrusive Terminal Assistant
Mammouth AI - The “all AI Models in One” solution
Presidio Anonymizer - Python based module for anonymizing detected PII text entities with desired values