Pre and Post-LLM Software

The recent RSA conference has left me concerned for the many companies in attendance. It seems we are at a turning point in software history, divided into two epochs: Pre-LLM and Post-LLM.

Pre-LLM software is limited in scope, only aware of its specific database and rigid schema. It operates through narrow, brittle queries. In contrast, Post-LLM software is based on understanding. It deals with knowledge and wisdom rather than data and information. Instead of requesting specific rows of data, you can simply ask for the insight itself.

Pre-LLM software is characterized by rigidity, narrowness, and self-centeredness, while Post-LLM software is flexible, cohesive, and powered by context. LLMs have access to a vast array of knowledge, with large models like GPT-4 containing a significant snapshot of human knowledge. The interface is language, allowing for better questions and insights through natural language.

The value of Pre-LLM software is capped due to its lack of context and inability to integrate it. Consider an Incident Response/SOC software that now has access to real-time knowledge of people’s locations, backgrounds, relationships, and cloud infrastructure details. This information is crucial for determining whether a connection is malicious or not.

LLMs excel in this environment, using context to connect the dots when answering questions. Instead of a Tier 1 analyst spending hours researching, we can simply ask the LLM if a connection is malicious and why. The LLM can provide a detailed response based on context, saving time and effort.

In this new world, software becomes a combination of context and questions. The more context and better questions we have, the more we can improve. This is the realm of LLMs, and the transformation is beginning now.

Initially, companies will use vector embedding databases to bring context to LLMs. Eventually, as prices decrease, we’ll see custom models built on top of massive, general models. Companies will train their own LLMs on their data, making them queryable and capable of not only identifying problems but also helping to fix them.

We are at a critical juncture in software history, with Spring 2023 marking the transition point. It’s time to prepare ourselves and our companies for the Post-LLM reality, where databases and queries are replaced by custom LLMs trained on our data, answering our questions, and taking action towards our desired outcomes.

Don’t wait. Start your transition now.