Chat software: a neat wrapper around your LLM

#6.1Infrastructure

In general, you communicate through a chat interface with AI systems that contain Large Language Models (LLMs). This technology seems simple—you type a question and get an answer—but behind the scenes it requires a complex technical infrastructure. Many people think they are talking directly to an LLM and that their data is processed only by the model, and that the model automatically learns from it. In reality, all communication passes through the chat interface, which is managed by another party. This leads to misunderstandings, for example about:

• who has access to your data;
• whether your data is used for model training;
• where your data is stored and for how long.

It is therefore essential to understand the difference between the LLM file (the model) and the provider of the chat interface (the intermediary layer, in other words: the wrapper, the packaging).

#6.2The LLM is a large file

An LLM is a single file that you could, in principle, put on your own computer. The file contains billions of parameters trained on large amounts of data. This file does not include built-in chat functionality: it is a file that, given a particular input (prompt), generates a single response based on the GPT method (Generative Pretrained Transformer, see Box 1.1 - What do we mean by ‘GPT’?). The model itself remembers nothing from previous interactions; it is what is called stateless. The file also has no knowledge of, for example, the current date (and platforms offer additional techniques and functions that do allow it to ‘know’ the current date or other sources).

#6.3Chat interface and chat history

The experience of an ongoing conversation with a generative AI is created by an intermediary layer: the chat interface (for example, a web app or app—think of Copilot, ChatGPT, Gemini, or Claude).

This chat interface stores the full chat history and, with each new question, sends the relevant chat history back to the LLM file. This is called conversational memory. This principle emerged alongside the rise of LLMs (Pinecone, 2025) and is essential for creating coherent conversations with AI.

So the LLM keeps seeing the entire conversation (or the part that fits within the context window), which makes it seem as though the model has a memory and understands well what you want to ask. Without sending the first question and the answer back along with it, the LLM does not know what the conversation is about.

#6.4Preconfigured chatbots and projects

Using general chat environments such as Copilot and Gemini, you can also create preconfigured bots. For example, you can create so-called Custom GPTs: a fixed prompt, optionally with an added knowledge source, that you can reuse again and again. This saves time because you do not have to re-enter those instructions every time, and it leads to more consistent responses from the language model.

In addition, in platforms such as Gemini , you can also create a project (Gemini calls them Gems). A project is a kind of workspace in which you bring together files, sources, and rules. Within that project, the AI responds specifically to that context each time, so you do not have to re-enter it every time in a new chat within the project. A bot mainly defines a role or way of working (such as “always answer in an academic style”), whereas a project is mainly about organizing your own material and agreements.

For example: if you want to query one document in different ways, you can turn it into a Gemini Gem and get started right away. As a researcher, you can specify in the instructions which domain the model should primarily draw knowledge from and that the answers should always include references. Or, in image generation, you can describe a fixed style so that every new image has the same look. In a project, you can also store your files and context on top of that, so you do not have to upload them again and again.

For clarification: Google calls this mechanism of preconfigured chatbots Gems, Anthropic calls it Projects, Copilot calls them Agents and EduGenAI calls this a Persona.

You also increasingly see such built-in, preconfigured bots in ordinary programs such as Word or Google Docs. Often you do not even notice it: you simply use a menu option to summarize text, translate it, or automatically create slides based on a web page.

Based on the technology, you can also imagine that you—or companies themselves—create and offer such an intermediary layer. Below you see a number of examples.

#6.5Agentic AI and Artificial General Intelligence

AI application systems continue to develop: in 2025 and 2026 especially around so-called Agentic AI. In addition, there is a great deal of discussion about the development of so-called Artificial General Intelligence.

#6.5.1What is Agentic AI?

Agentic AI refers to combinations of prompts and search mechanisms that can think through problems, make decisions, and take actions with minimal human guidance. Unlike regular generative AI, which simply responds to prompts, Agentic AI behaves more like an autonomous assistant. These systems use LLMs to reason, have access to external tools (such as search engines or databases—or your own files), and have memory to learn from previous experiences. Think of it as the difference between a calculator that only computes what you enter and a study partner who can break your complex assignment into steps, look up information online, remember what worked last time, and adjust the approach if something does not work.

If, for example, you asked an agentic AI to help solve errors in your programming assignment, it would not only propose a solution but also analyze the error, test possible solutions, learn from failures, write additional code for helper functions along the way, and improve iteratively until the code works (Herrman, 2026) (and it automatically keeps all the documentation up to date). But agentic AI can also be linked to your email program, electronic calendar, and files on your computer, so that it can also act on them directly such as automatically scheduling appointments or replying to emails. Another example is how AI-driven scientific literature search engines work. Although this makes them incredibly powerful for tackling complex, multi-step solution paths, they are not always necessary. For simple tasks, traditional AI chat methods are faster and cheaper.

#6.5.2Artificial General Intelligence (AGI)

Contemporary AI excels at specific tasks such as language processing, image recognition, or playing games. But Artificial General Intelligence (AGI) would mainly possess flexible, generalizable intelligence capable of reasoning about unfamiliar problems, transferring knowledge across domains, and adapting to entirely new situations without additional training. Whether AGI will become reality is a much-discussed question in technology and science. Some researchers believe we may achieve it within decades by scaling up current approaches or through fundamental breakthroughs, while others argue that it requires discoveries we have not yet imagined, or that it may not be possible at all. The timeline and feasibility depend on unresolved questions about consciousness, reasoning, and whether intelligence can arise purely from computational processes, making AGI at once a technical challenge, a philosophical question, and a topic that requires careful consideration of its societal implications.

#6.6Data and privacy: who decides what happens to your data?

Providers of chat interfaces determine what happens to your data and the generated responses. The provider may, for example, choose to:

• store the chat history locally or in the cloud;
• use your data for analysis, product improvement, or even for training new models;
• share data with third parties or explicitly protect it.

These policies differ from provider to provider and are often not fully transparent. Some interfaces offer privacy settings; others do not. That is why it is important to understand the technology described above and, based on that, make judgments such as:

• Are you working with a provider that has unclear terms and processes? Then be very careful with the data you upload and the questions you ask. Make sure you never share sensitive data such as personal information or confidential business information. Examples of this kind of provider include ChatGPT, Google, Anthropic, and many smaller providers. In other cases as well, the terms are often unclear, such as with a paid account or accounts that supposedly work based on signing in with your organization account.
• Does the university or your employer provide a chat environment, and are there clear agreements that the data will never be shared with parties outside the institution? Then you can use the chat environment much more freely, but still do not share sensitive data if it is not strictly necessary. Examples include the Copilot environment used by universities, at VU Amsterdam the Gemini funtions, the SURF EduGenAI environment, or environments offered and managed by research institutes themselves. Note: login with your VU Amsterdam credentials.
• If the university or employer does not provide an environment, or if you doubt the agreements with the provider, then see whether you can work with an LLM that you can install on your own (powerful) PC or laptop and use through a local program (Local LLMs). Examples of programs include Jan.ai, LM Studio (Google), and Ollama.

Even so, a great deal often remains unclear about how your data is processed, and many legal and technical issues play a role. For more background on the complexity surrounding ownership of user-generated data in AI interactions, we refer you to the work of Metcalf & Crawford (2016).