If you ever wonder how AI systems learn from seemingly unstructured data, I have condensed it all in this PDF. As I showed with the image reconstruction - an AI system can learn an objective by synthetically creating "problems" and fixing them. It is possible to extend the same approach to almost any kind of data. Autonomous learning systems are the cutting edge of AI systems today and it is already possible to put AI agent + local model into an environment, define a learning objective and have it then invent tasks that can be used to generate trajectories which can then be used to teach the model that learning objective. Data is basically the playground. This week I'm building the first version of a system that can learn autonomously from arbitrary source material by inventing its own tasks, attempting them, verifying the results, keeping the useful trajectories, and then training on what worked. More info in the attached PDF. The first playground will likely be some software repository because code gives us unusually strong feedback: we can deliberately break it, ask the agent to recover them, verify the result with tests, and turn successful and repaired trajectories into training data. The same architecture can later generalize beyond code to things like manuals, technical documentation, stories, datasets, and other corpora with very different learning objectives.

Full sample source is part of Brain: https://github.com/swedishembedded/brain/tree/main/samples/decision/doom This is a two part model: - Encoder - MiniLM-L6-v2 (6-layer BERT, 384-d), imported pretrained and frozen by default. It encodes the observation text and each candidate option's text. - Scoring head- a small trainable head on top that produces one scalar per option, softmaxed into a distribution. PPO is used to learn a continuously improving policy.

The most exciting way to train AI is to let it learn real skills from synthetically generated objectives. Suppose we want to learn photo decluttering. A problem with base model is that it will invent things - we don't want that. We want a very clear objective of ONLY removing the clutter. We define what's important: the model must preserve the room but remove the clutter. Meaning: no invented walls, no deleted walls, no reshaped walls. We can generate all of them with AI for training and then apply the model on real images and it will transfer the skill of decluttering. Step 1: generate using image model (or get) clean, well lit, room images. Step 2: generate using the same model exactly the same room, with exact same camera angle but cluttered Step 3: train lora adapter to go from cluttered to decluttered image. Here are all the prompts I actually used: Cluttering prompt: ""Preserve the room exactly but add clutter, bad lighting, photo noise and make it into simply a badly taken photo. Be VERY careful NOT to change the room architecture. Keep it exactly the same as original photo. Do NOT change camera angle. Keep exact same camera angle. Do NOT add any text or camera date to the photo." Training - 3 prompt variants, round-robin'd across 54 training images: - A: "Declutter this photo. Remove all garbage and clutter. Keep the room's shape, architecture and camera angle exactly as it is. Fix the lighting so it looks bright and clean. Realphoto" - B: "Remove all clutter, trash, and mess from this room. Do not change the walls, floor, windows, or camera angle. Make the lighting bright and even, like a professional real estate photo. Realphoto" - C: "Make this into a clean, professionally lit real estate photo. Remove all clutter and personal items. Preserve the room's exact architecture and camera angle - do not change them. Realphoto" Realphoto is just used as a lora activation keyword and is present in ever prompt. You don't have to use 3 different prompts since embeddings will be close but I found it gives slightly better result to vary the input prompts because it makes it a little more robust to different wordings.

The most valuable skill in the modern age of AI for you as a developer is how well you can connected different inputs, models and effects together to produce the final result. It's how much you can control what comes out that matters. If you are skilled, you can control it completely and produce anything you want. If you are not skilled you will get what everybody else is getting. That's the differentiator.

Learn how to work with AI in a way that accelerates your work from plan to finished MR instead of slowing you down!