Three Myths of LLM-based Application Limitations

The emergence of large language models (LLMs) such as GPT/ChatGPT has revolutionized the way we process information. However, myths surrounding the use of LLMs in applications can discourage people from fully utilizing their capabilities. It's important to distinguish between limitations of the applications that can be built with LLMs versus the properties of the models themselves. In this blog post, we will debunk three of the most common myths surrounding LLM applications and show you the incredible potential they offer.

Myth #1: Lack of Recency

One common myth about LLM applications is that they are not useful for applications requiring information on current events after 2021. While it is true that current OpenAI models are trained with data up to 2021, there are tools available that allow for combining semantic search with LLM response generation. For example, Langchain provides a way to access frequently updated vector databases and incorporate them into LLM applications. This means that LLMs can still be used effectively for applications that require up-to-date information.

Myth #2: Lack of Privacy

Another myth surrounding LLM applications is that they are not useful for private data. While it is true that the OpenAI models are limited to content that is available on the web, we can add private knowledge stored on our own servers through indexing and semantic search. This means that LLMs can be used for applications that require the processing of private data, as long as appropriate security measures are in place.

This example video shows a real world use case using information from the current week as well as information that is managed privately in an owned vector database. This still does require transmitting documents once to an LLM provider to generate vector embeddings, but they are only stored temporarily as per their data retention policy.

Myth #3: Lack of Traceability

A last myth about LLMs is that they are not useful for applications where you need to understand where the response originated from to verify facts. However, this is not entirely accurate. Question answering with indexed semantic search allows for the construction of custom "source" references that can be supplied with the response. This means that LLMs can still be used effectively for applications that require traceability and verification of information.

Receive a LLM (GPT3 in this example) generated response for a direct answer

Drill down to root source content for confident validation

As seen in this example - the language model can restructure, summarize and make more consumable the facts that are discoverable by semantic search in the unedited source documents.

In conclusion, while there are several myths surrounding applications built with LLMs, it is clear that they are a powerful tool for processing and analyzing information. By understanding the full potential of LLMs and the tools available to work with them, we can develop effective applications that meet the needs of modern businesses and organizations.