Activity
Mon
Wed
Fri
Sun
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
What is this?
Less
More

Owned by Aurel

AI narovinu bez zbytočného hypu

28 contributions to Clief Notes
Need help with evaluation
Hey Clief Notes fam! 👋 I'm Andre Izarra and I'm stoked to be here. I've been learning the past month about ICM created some simple workspace and projects and find this like an Eureka moment. 🙋 A little about me: I've been working in software development the last 5+ years, most of the time as freelance and this last year and a half I've been dedicating a lot of time to IA, learning the tools, ways to use it better and to integrate well LLMs in products. 🎯 My current goal: Learn a way to actually evaluate the accuracy of the ICM 💪 What I'm currently building/working on: The biggest use I have for ICM is a project for my whole work pipeline. I have a workspace for every mayor part of my process, manage conversations to clients until a deal, actually build the software, create the study cases with metrics, create content (starting with this thanks to this community) and repeat. 🤔 My biggest struggle or question right now: I notice the improvements working with agents following ICM but I can't tell exactly how to improve the structure, sometimes I notice problems with the results and I kinda can spot the problems try something different and usually improves, there were few times that got worse but I want as I said a way to actually evaluate the results to find a clear path to improve the structure. Let's get it! 🚀
0 likes • just now
The few times it got worse are the useful part, and you can only read them with fixed inputs. Pick ten real tasks from one workspace, say past client threads that ended in a deal, and keep the outputs you accepted as the reference. Before changing the structure, write down what a good output must contain for each one, as yes or no checks rather than a score out of ten. Then change one file at a time, rerun the same ten, and count the checks. When a run fails, note which CONTEXT.md or reference file the agent was reading at that point. Over a few weeks the failures pile up on a couple of files, and those are the ones to restructure.
𝗛𝗼𝘄 𝗜𝗖𝗠 𝗮𝗻𝗱 𝗛𝗲𝗿𝗺𝗲𝘀 𝗞𝗲𝗲𝗽 𝗬𝗼𝘂𝗿 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 𝗠𝗲𝗺𝗼𝗿𝘆 𝗟𝗼𝗰𝗮𝗹
Thank you Jake Van Clief and David McDermott for having me, and David Vogel for the Hermes assist. This one is ICM in practice. I open my own Hermes and walk every layer of its memory, live: the built-in memory files, session history, skills and the curator, the memory provider I host myself, and the ICM workspace that holds the job itself. The part I think this room will care about: in my workspace, every folder carries a small AGENTS.md router that points to a CONTEXT.md contract. Hermes loads AGENTS.md on its own. It doesn't load CONTEXT.md, so the router is what makes sure the contract gets read. Small router, real instructions 1 step away. Watch it here: https://www.youtube.com/watch?v=GtROa4PfvY8 ICM is why this system holds together, and Hermes is what walks it. Jordan The read through https://agentsasfolders.ai/blog/ai-agent-memory-hermes/
1 like • 2d
Of the five layers, session history is the one I would look at hardest, because nothing curates it. The memory files and the curator decide what to keep. Session history keeps the conversation with the tool results in it, so whatever the agent opened is in there, including files the router never meant to load. Keeping it local decides where all of that lives and says nothing about how long it stays. Retire a client's folder in the ICM workspace and its contents can still sit in the transcripts and maybe in the self-hosted provider. A quick check is to take a name from a folder you already archived and search the history for it. How long does Hermes keep session history by default?
1 like • 1d
@Jordan Shaw Pruning by age clears the stale ones. The case it misses is retiring a client whose sessions are still recent, since an age rule keeps those for the whole window. And pruning the session store does not touch whatever the curator already copied into memory from them. Can the prune target the sessions that worked in one folder, or only go by date?
Our files were built for people, not for AI... So I'm rebuilding them, and teaching as I go.
Who I am: Jeff van Leenen, Calgary, Alberta. What I actually do: I'm an Executive at a residential home builder. My job is how the work gets done as the company grows. I was a high school science teacher before this, and it still shows. What I'm building with AI right now: Four things... 1. Our company file system. I'm restructuring it into ICM so it's ready for our staff and AI to actually work from. 2. Teaching our staff about AI and how to use it safely on their own jobs. 3. Folder systems for trade businesses. The first one is for a painter who works for builders. The purchase orders he gets paid on are often wrong and nobody checks them, so I'm building a tool that checks each PO against his price list and what he measured on site. 4. The Idea Engine. It plugs into Claude, saves the ideas I float while I'm working on something else, and brings an old one back when it fits what I'm doing. It's in testing on my own machine. ...plus more... What I'm looking for connection-wise: - Someone who's solved discovery with trades owners. I want to get better at finding the real pain quickly, before anyone talks about tools. - Trading workflows and systems, especially anything built for builders or trades. Best way to reach me: DM here, or comment below.
1 like • 2d
Before the restructure is finished I would settle who reads what once AI is working from it. A home builder's files hold payroll, homeowner contracts with names and addresses, supplier pricing and warranty disputes, and today the drive's folder permissions probably keep most staff out of a good part of that. An AI session reads with the access of whoever opened it, so if the new ICM tree sits in one shared folder that everyone opens, nothing in the routing stops a site super's question from pulling in a payroll file. Sorting the folders by who is allowed to see them comes first, with the restricted ones in their own tree rather than a subfolder. That also gives the safe-use training one concrete rule to teach. I work on VaultGuard, which adds per-file permissions and an audit trail to a shared Obsidian vault for this kind of company tree.
Solving Context Before Adding Infrastructure
I keep seeing the same technologies in AI engineering roles: LangChain, LangGraph, RAG, vector databases and agent orchestration. They solve real problems. After building extensively with ICM, I've started wondering how often we introduce that infrastructure before deciding whether the problem actually requires it. ICM approaches the problem from the context side. A workflow is divided into stages. Each stage has one job, known inputs, expected outputs and defined context. The agent reads what that stage requires and produces an artifact that the next stage can work from. Context selection becomes part of the design. That distinction has changed how I think about RAG. RAG is extremely useful when the system cannot know in advance which information will be relevant. If someone can ask arbitrary questions across thousands of changing documents, retrieval has to determine what information should reach the model. That creates a second problem alongside the original AI task: retrieval itself has to work well. Documents need to be chunked. Search and ranking need to return the right material. Retrieval quality has to be evaluated. If an answer is wrong, you may need to determine whether the model reasoned poorly or whether it never received the right information in the first place. There are many applications where that complexity is justified. But consider a structured workflow where the current stage already tells you which references are relevant. If I'm performing a security review and the workflow already defines the security requirements, architecture references and files that should be inspected, I'm not convinced semantic retrieval should automatically sit between the agent and that information. The architecture already knows what context is required. In that situation, retrieving the "most relevant" context at runtime may be less useful than explicitly providing the correct context by design. The failure modes change too. With retrieval, a poor result can come from chunking, indexing, query formulation, ranking or reasoning.
0 likes • 4d
The part I would push on is the gap between what a stage is told to read and what it can read. The stage file lists the references, but an agent with file tools will still grep around the workspace when it gets stuck, and in a security review that is exactly when it wanders into .env or another client's folder. Checking this is cheap. Compare the files each stage lists with the Read and Grep calls in its session log, and any file that shows up only in the log is context nobody designed. For a stage that handles sensitive material the list has to become a boundary, a deny rule or a folder the session cannot reach, because routing on its own is a suggestion.
0 likes • 3d
@Ignacio Sandi I would keep those rules in layer 0, they state the boundary for every stage. They just do not enforce it. CLAUDE.md is text the model reads, so it holds while the model follows it, and an agent stuck mid task can still decide the file it needs is worth opening. A deny rule in settings.json, say Read on .env or on another client's folder, is checked by the harness before the tool runs, so the model does not get a vote on that read. A Read rule does not stop cat in Bash, though, so fence that too. ICM for the routing and CLAUDE.md for the intent, with deny rules only on the few paths that must never be read. Have you seen a stage step outside its list in the logs yet?
Anyone in the due diligence space?
I’m building out an ICM architecture for a PE firm catered towards making due diligence easier and more efficient, I wanted to reach out and see if anyone here is in that space or worked close to it and even any tips you’d have for the correct folder structure/processes. I know a bit but knowing more on your guys’s process would be unreal thanks!
1 like • 4d
I would put a hard wall around each deal from day one. A PE team can be looking at two companies in the same sector at once, each under its own NDA, and if the analysis stage can read a shared precedents folder or last month's memo on the competitor, the agent will use one target's numbers to sharpen the other's write-up. Nobody catches it because the output reads better. The other case to plan for is the dead deal. NDAs usually say return or destroy, so every AI note made from that data room has to sit inside that deal's folder and leave with it, not in a shared lessons file. I work on VaultGuard, which adds per-file permissions and an audit trail to an Obsidian vault for this kind of who-can-read-which-deal problem. https://vaultguard.cloud
1-10 of 28
Aurel Babiš
3
23 points to level up
@aurel-babis-9673
Finance & Automation Engineer

Active 1h ago
Joined Jun 9, 2026
Powered by