WRITING / hermes
Raj OS: Giving Hermes the Context to Be Useful
I keep a small operating manual in Obsidian so Hermes can work with my priorities, preferences, and constraints. Here is how I organize that context and keep it current.
I have been thinking about how much of an AI agent’s usefulness depends on knowing the person asking for help. A capable model can produce a reasonable plan. Whether that plan belongs in my life is a different question.
My response has been a folder in my personal Obsidian vault called 0 Raj OS. It holds the context I want my agents to work with: my priorities, operating preferences, and expectations for how they should help. I have been using it with Hermes, and it has been working pretty well.
This is the next piece of the setup I described in my Hermes desktop article and Obsidian publishing workflow. Those posts were about giving an agent tools and a place to put its work. Raj OS is about giving it a useful starting point for deciding what work makes sense.
The first file tells Hermes where to look
The entry point is 00_READ_ME_FIRST.md. Its opening description calls the folder a small operating manual. That keeps the purpose manageable: it needs to be accurate enough to support decisions, without becoming another project to finish.
The file routes the agent to different notes depending on the task. Personal decisions point toward values and priorities. Getting work done points toward operating style and support needs. Agent behavior has its own instructions. Temporary priorities live in 05_Current_Season.md.
That last distinction matters to me. A long-term goal can remain valid while being the wrong thing to pursue this week. Keeping a current-season note gives me somewhere to explain that sequencing without rewriting my entire profile.
Here is a shortened map of the parts relevant to this article. The private notes themselves stay in my vault:
0 Raj OS/
├── 00_READ_ME_FIRST.md
├── 05_Current_Season.md
├── 06_How_I_Operate.md
├── 10_AI_Agents_and_Automation_Instructions.md
├── 11_What_Hermes_Knows_About_Me.md
├── 15_Context_Questions_and_Decisions.md
└── 18_Personal_Context_System.mdThe numbered files give the system a visible structure. The read-first note supplies the routing. I can open either in Obsidian and see what I am asking an agent to rely on.
Some of the most useful context is very ordinary
One instruction in my agent notes is to give me at most two clear steps at a time when troubleshooting. Another asks the agent to consider the maintenance burden and second-order effects before rushing into a recommendation.
These are specific preferences about the help I want. A technically correct answer can still arrive in a form that makes it harder to use. Writing those preferences down gives Hermes something concrete to follow, and gives me something concrete to correct when the interaction misses the mark.
I also tell agents to push back when my priorities or the evidence warrant it. That instruction sits alongside a requirement to avoid manufactured disagreement. I want a recommendation connected to my circumstances, with a reason I can examine.
I do not have a measured before-and-after result for Raj OS. My assessment so far is that it is working well for me. The folder makes the context explicit enough that I can inspect what is missing or stale, rather than treating every unhelpful answer as a prompting problem.
I keep different kinds of memory in different places
One of the decisions in Raj OS is to separate personal context from the changing details of a task.
| Information | Where I put it |
|---|---|
| Personal priorities, preferences, and agent expectations | Raj OS |
| Compact, stable facts needed across sessions | Hermes durable memory |
| Current project state, blockers, and next actions | Project notes |
| A procedure worth repeating | A reusable skill |
| Temporary details from one task | That session |
There is also a note named 11_What_Hermes_Knows_About_Me.md. Its purpose is to mirror durable memory in a form I can inspect. The intended arrangement is that the small set of stable facts can travel with Hermes while the fuller context remains readable in my own notes.
The newer architecture note extends that separation to other systems. Calendar is authoritative for appointments. Tasks holds commitments and deadlines. Original documents stay with their source. A summary in Obsidian should not silently become a replacement for a live record.
This is a design rule I am building around. It does not mean every integration is already reliable or every memory copy stays synchronized automatically. Those are things that need verification.
The whole vault is too much context
My context-selection rule is to retrieve the smallest sufficient set of current, relevant information for the task.
For example, the read-first note sends an agent to family context only when the work materially involves that area. An unrelated technical question should not pull in personal details simply because the files are available.
The architecture note spells out what a useful task packet should contain: the requested outcome, relevant sources, applicable constraints, current state, and action boundaries. It also asks for uncertainties and freshness to be made visible. If a record could have changed, the agent needs to check the authoritative source.
Stored context is supplementary. My current instruction takes precedence, and a conflict should be surfaced. Otherwise an operating manual can turn into a way of repeatedly applying an old decision after I have changed my mind.
Maintenance needs a place in the design
The read-first note includes questions I asked while shaping this system: how the information would reach Hermes memory, how often I should revisit it, and how to track the evolution of my thinking.
Those questions led to distinct places for distinct updates. Context-changing questions and decisions have their own note. Review history has another. Ordinary task questions stay out of that decision record.
The maintenance rule is small: update a note when something important changes, keep the update short, and remove stale context. The documented review plan is a short weekly check for what changed, what became stale, and where a correction belongs.
I want that last question to remain part of the review. A new project deadline belongs with the project or task. A preference I expect agents to respect repeatedly belongs in personal context. A fix I will need again may deserve a skill. Saving everything into a single memory file would erase those distinctions.
A correction from the daily-brief pilot
The supervised pilot now combines Google Calendar, Google Tasks, Raj OS, and Hermes. Its handoff contains a useful correction: an action was marked complete too early, then reopened. The operating rule recorded from that mistake is simple: preparation being ready does not mean the action happened.
That is exactly the kind of detail I want this system to retain. A summary that says a task is moving along can sound positive while hiding a bad state change. The correction says to mark an action complete only when I confirm it happened or directly instruct the agent to do so.
The same handoff records my approval of the brief’s length. That approval applies to the format; it does not authorize a recurring automation. Those are separate decisions, and the record keeps them separate.
This is a more useful account of progress than saying Hermes now knows me. The system has a specific correction to apply, a format I have accepted, and a boundary on what happens next.
Keeping the context mine
Raj OS lives in Markdown because I already work in Obsidian and can inspect the files directly. The architecture treats vendor memory and retrieval indexes as copies that can be rebuilt from authoritative sources.
That does not give every model automatic access to my vault. Access still has to be configured, and a task still needs a deliberate selection of context. It does give me a readable source I can maintain without depending entirely on what one agent happens to remember.
The folder is private. This article describes its structure and a few working preferences; it does not publish the underlying personal records. Context about me also does not grant an agent permission to make commitments on my behalf. Those boundaries belong alongside the preferences.
The next step is another supervised brief, using fresh Calendar and Tasks data, followed by one piece of feedback at a time. I am keeping the workflow manual while deciding whether it deserves a schedule. It gives Raj OS a practical test: can Hermes use what I have written to help with the day in front of me?