Curious about the functioning of an LLM? Software engineer Brendan Bycroft created a visualization and walkthrough of the LLM algorithm underlying OpenAI’s ChatGPT.
Explore the algorithm’s every step: observing the process in action.
Here are some simplified explanations of the terms used in this visualization:
- Embedding: It’s like translating words or other things into a secret code that computers can understand better. Each word or item gets its unique code.
- Layer Norm: Imagine you’re editing a photo to make the colors look better. Layer normalization does something similar for the data in a neural network, making it easier for the network to learn.
- Self Attention: It’s like when you’re reading a story and pay more attention to the important parts that help you understand it better. In a neural network, this helps it focus on the important parts of the data.
- Projection: Think of it as changing the appearance of something to fit into a new space, like adjusting a picture to look good in a different-shaped frame.
- MLP (Multi-Layer Perceptron): It’s a simple type of brain-like system in a computer. It has layers of small processing units that work together to solve problems, like recognizing if a photo is of a cat or a dog.
- Transformer: This is a more advanced type of computer brain system, really good at understanding sequences, like sentences in a conversation. It’s like having a super smart pen pal who remembers and understands everything you’ve said.
- Softmax: Imagine you have a group of runners, and you want to figure out how likely each one is to win the race. Softmax helps turn their speeds into these winning chances.
- Output: This is the final answer or result that comes out of a computer system after it has done its calculations, like the end result of a math problem.