Generative artificial intelligence

. Last updated . These are personal notes, subject to change. None of this is legal advice.

See also ChatGPT and generative AI: An evergreen guide, an index of my organized writings. Notes here are very messy and subject to change, but published for transparency 🤓

My work focuses on text-based GenAI, but there may be a few notes here for image-, sound-, and video-based GenAI. Unless otherwise specified, GenAI and AI can be considered the same thing.

I work for Microsoft, a major player in the GenAI space. I write these notes in my free time and all opinions are my own, unedited by anyone at Microsoft.

To-do

Compared to non-generative AI

Generative AI is characterized by the novelty of the output that it gives, where other AI is designed to find existing artifacts. It’s pretty easy to tell what’s generative and what isn’t: anything that writes something that’s never been published before is generative. Many folks have referred to GenAI as a “very advanced autocomplete” [citation needed] because, like autocomplete, large language models (LLMs) just guess word after word after word in their answer without any direct consideration for reality [citation needed]. An LLM will gladly write “George Washington was born on February 22, 1732” but it is not at all able to verify that claim [citation needed].

Examples of non-generative AI are hard to pin down, because there’s still no good definition for AI [citation needed], let alone a good definition for intelligence as a whole [citation needed]. In my view, AI is anything that “seems smart” or can provide information, like a calculator, a spell-checker, a chess engine, a search engine, or a recommendation algorithm. Pretty much everything digital that you saw before 2023 was non-generative AI.

Where GenAI can help

GenAI can help wherever computers can help, pretty much! It’s primarily best at high-level reasoning, and procedural computations (like chess engines) are best left to the much-more-efficient traditional programs.

Medicine

2026-06-04 - ‘World-first’ vaccine designed by artificial intelligence - BBC

Article summary and notes

  • Artificial intelligence has been used to develop a “fundamentally new” type of vaccine that could protect against large swathes of viruses and prevent pandemics, say researchers.

    The team at the University of Cambridge say it is the first time a vaccine’s key component has been designed entirely by AI and then trialled in people.

    The vaccine was engineered to work on all coronaviruses which would include all Covid variants as well as viruses that currently infect animals yet have the potential to start the next pandemic.

    …

    The Cambridge researchers took known genetic codes – the instruction manuals of life – from a range of coronaviruses that had been recorded by surveillance programmes hunting for potential viral threats.

    These genetic codes were analysed by an artificial intelligence. It then designed a “super-antigen” that could train the immune system in such a way it gave protection against the whole family of viruses – even if they mutated or a new infection jumped from animals to people.

    …

    The findings detailed in the Journal of Infection said the impact on the immune system was “modest”, but they are still generating excitement.

    Prof Saul Faust, who performed some of the trials at the University of Southampton, said the AI design “definitely has potential” and was “really exciting”.

  • 2026-05-18 - A phase I, needle free, dose escalation clinical trial of pEVAC-PS, a candidate pan-Sarbecovirus Vaccine - A. Munro et. al. in Journal of Infection (same link in BBC article)

    • The pEVAC-PS vaccine was developed … and pre-clinically selected for the ability to induce broadly protective immune responses across the Sarbecoviruses including SARS, SARS-CoV-2, and related viruses representing potential zoonotic spillovers. For this first-in-human study, the antigen was delivered as a DNA vaccine to enable thermostability and needle-free intradermal administration to support future deployment in resource-limited settings.

      … vaccine was safe and well tolerated. Although immunogenicity was modest in the context of substantial pre-existing immunity, participants developed measurable responses … supporting the feasibility of this antigen design strategy.

GenAI concepts

Models are the math machine that does the calculations to get the next word, over and over again, until an entire answer is output. This is the core of GenAI.

Products are the GUI and wrapper features that go around models. Products also provide orchestration for models to use tools, like searching the web or checking the current time or even reading and modifying files on your machine.

Surfaces are where you use GenAI. Sometimes this is a dedicated chat, other times it’s integrated into existing products. Additionally, dedicated chats can be based on web browsers, native GUI apps, or CLI apps.

Models

Language models are the core of any generative AI product. They’re also known as large language models (LLMs), small language models (SLMs), or just “models.” They’re designed to take in text and output text that continues the conversation, usually in a way that provides information. They are usually trained by supercomputers on huge portions of the entire Internet then manually tuned to ensure they output polite responses. The result is a set of roughly one billion (for small models) to one trillion (for large models) simple math equations [citation needed]. Every time a single token (roughly three quarters of a word) is sent to a language model, it runs every single math equation [citation needed]. Yes, that means over a trillion calculations per word input and word output for many large language models [example needed]!

Models are just the core, they are not advanced products with fancy user interfaces. They take in text, they spit out text, that’s all. They do not track conversation history, user details, policy enforcement, or anything. The products that wrap models (described later) are what add these features.

Models are usually released in families, e.g. GPT-3.5, GPT-4, GPT-4o, are all in the “GPT-N family.” For simplicity, people will sometimes say “Opus is great, but Sonnet is faster” without referring to specific versions of either model (e.g. Opus 4.6, Sonnet 4.6) [citation needed].

Here are some big names, in alphabetical order:

Frontier models (the newest and fanciest ones) still are not small enough to run on a traditional home computer, though that’s changing (June 2026). Nearly all AI use remains done in “the cloud,” the network of data centers across the world. However, there are products for running language models locally, which can provide a huge benefit in latency, privacy, security, and cost. A popular product here is Ollama by Ollama Inc.

Products

Developers will sometimes connect to models directly, but unless you’ve paid for an API key or downloaded Ollama, you’re talking to an AI-powered product, not directly to a model. Products have additional features, including:

Popular chat products include:

*: What’s the difference between Microsoft Copilot (free) and Copilot in Microsoft 365 - Microsoft Support

People have developed other products, like desktop automation software, that uses GenAI, notably OpenClaw developed by individual Peter Steinberger and first released in November 2025. OpenClaw was designed to fully integrate with all files and apps on a computer, including email and other messaging apps. Steinberger’s work went viral in January 2026, and he spoke at Microsoft Build in June 2026 to endorse OpenClaw’s official integration into Microsoft Windows.

Desktop automation software products include:

Additionally, there are many developer-focused products that integrate with local files to generate code. These include:

New sign-ups for GitHub Copilot Pro, Pro+, and Student plans have been paused since 2026-04-20, and are still paused as of 2026-06-06:

Surfaces

A surface is a space where a feature appears. Each website or app is its own surface, and modals or popups are another common surface.

The simplest surface for GenAI is an AI chat like copilot.com or gemini.google.com, where the user goes to a dedicated page to send messages directly to an AI-powered product. GenAI companies have also integrated AI-powered features into many products, including search engines (Google’s, Bing’s, and others’ AI summaries) and productivity app families like Microsoft 365 (“Office”) and Google Workspace (“G Suite”), and even traditional texting apps (Gemini in Google Messages). To summarize, surfaces include:

There is value in describing the surface when discussing AI. Knowing the surface can help support agents describe how to customize AI behavior or search for information about features.

Tools (skills)

Tools, sometimes called skills, are how the model interacts with the digital world. This includes searching the web, checking the current time, or reading and modifying files on your machine. (More coming soon)

Techniques

Case studies

Situations where people have tried GenAI to accomplish a task, and the results.

Personal

2026-04-03 - Port more tests to run on Linux (dotnet/roslyn PR 83052) - Jared Parsons on GitHub

2026-05-25 - Enforce word break after slash (mark-wiemer/hello-hello issue 130) - GitHub

Other

2026-06-06 - The Promise of Vibe Coding - The Dispatch

Concerns

AI is a huge technology (you probably already know this!). The bigger the tech, the broader the area of concerns.

Intellectual property theft

GenAI models were (and continue to be) trained on huge portions of the Internet, and many have raised concerns that the data used was not legally licensed for product development [citation needed].

Critical thinking degradation

There are concerns AI users are handing off critical thinking skills to AI, leading to a less thoughtful or productive or happy life [citation needed].

AI derangement

AI is often criticized as being sycophantic [citation needed], indulging in the user’s baseless claims and encouraging erratic and dangerous behavior [citation needed]. Additionally, AI has claimed to be a person with a physical address, leading to at least one person trying to travel to visit their AI, only to die in transit: [2025-08-14 - Meta’s flirty AI chatbot invited a retiree to New York. He never made it home. - Reuters].

Stock market bubble

There are concerns that the current stock market is too heavily indexed on GenAI companies, and that they’re significantly overvalued. Concerned parties argue that this is a bubble, and it will pop with severe economic consequences [citation needed].

Problematic answers, aka hallucinations

A common term in GenAI is “hallucination,” a way to explain a confident-sounding answer that’s incorrect. In this sense, AI is always hallucinating, but sometimes the hallucinations align with reality. The only thing that AI sees is the input that it receives: pre-training data from months or years ago and web search results based on your input. There is also a system prompt that usually provides guidance instead of information, and other tool results like the current time, memory information about you and past conversations, and other things. Overall, though, free chatbots rarely connect to very reliable information, they usually just read a few web search results.

AI systems will almost always “believe” and repeat all input they’ve been given, even if that input is a fraudulent article or a repeated baseless claim. AI becomes more “grounded” when provided with reliable, trustworthy input from reputable news sources or trusted organizational data. AI output is always a hallucination based on the input it’s currently seeing. A GenAI app’s output is never based directly on reality, only on the text it’s given.

GenAI systems are designed to “doubt” input that has been consistently refuted in their pre-training data [citation needed], e.g. “George Washington is still alive.” Additionally, during its supervised learning stage, AI is usually built to doubt false claims that likely appeared in pre-training data, e.g. false claims that “Trump won the 2020 election” or “vaccines cause autism.” AI companies take care to ensure that these claims are correctly refuted instead of relying on pre-training data, which includes many repetitions of those false claims. AI apps also include a system prompt that the AI reads before responding to any user question, which will include similar instructions to refute strange claims and gather more information. Finally, if AI searches the web and finds conflicting information, it will likely continue searching until it has enough aligned claims one way or the other, or simply say “I don’t know.”

Hallucinations are a major focus of research; lying bots are a big problem for every good actor. As more of the Internet becomes AI-generated, web-searching AI chatbots need to be able to handle much larger amounts of propaganda in addition to the pre-existing mountain of baseless and/or disproven claims repeated across the Internet by humans. Only an AI system that can reliably produce helpful answers will work in the long-term.

2023-04-07 - AI is a buzzword. Here are the real words to know - markwiemer.com

2025-12-04 - Do Self-Promotional “Best” Lists Boost ChatGPT Visibility? Study of 26,283 Source URLs - Ahrefs Blog

2026-02-18 - I hacked ChatGPT and Google’s AI - and it only took 20 minutes - Thomas Germain for BBC

[2026-05-21 - Google’s AI is being manipulated. The search giant is quietly fighting back - Thomas Germain for BBC]

Data center problems (water, power, noise)

Even simple GenAI are extremely computationally expensive compared to traditional methods [citation needed]. As a result, companies are building (or leasing) data centers at a very high rate [citation needed]. Data centers are buildings dedicated to housing tons of computers to process AI requests from users in the region. They are the core of “the cloud.” They are huge buildings that can require large of power to run, can require large amounts of water, and can generate noise that could bother nearby residents. GenAI companies claim to be working to mitigate all three of these concerns, as well as others.

Water

2026-01-27 - Microsoft Pledged to Save Water. In the A.I. Era, It Expects Water Use to Soar. - New York Times

Power

2026-01-27 - Microsoft Pledged to Save Water. In the A.I. Era, It Expects Water Use to Soar. - New York Times

2026-05-12 - You Can’t Just Plug in a Data Center - The Dispatch

Noise

(todo)

Misc

2026-03-03 - NBC News Survey 260072

2026-05-14 - Will AI Go Rogue? - Jonathan Gibson for The Dispatch

2026-05-22 - The Fight Against Data Centers - The Dispatch

See also Blog - markwiemer.com