I Keep Hermes Agent's Self-Improvement OFF For the First 14 Days — Here's What Happens When I Don't

I have published two articles praising Hermes Agent this week. This one is different. Both of those articles are accurate — Hermes is genuinely excellent at self-directed skill acquisition, and the compounding effect of the memory system is real. But the self-improvement loop is too powerful to enable on day one. Three weeks ago I ignored my own rule on a fresh VPS deployment. By day 14, eleven auto-generated skills had silently degraded my agent’s output quality across three production workflows. I lost an entire evening reverting to a clean state and hand-auditing every skill file. That evening is the reason this article exists.

This is the contrarian Hermes Agent setup guide. The thesis is simple: disable auto_create and auto_refine for the first 14 days, observe how the agent actually behaves on your real workload, hand-author your first generation of skills with intention, build the rollback infrastructure you will need later, and then enable self-improvement deliberately. The compounding still happens. It just compounds toward your actual workflow rather than toward the agent’s confused first-week impressions of what you do.

The Default Is a Footgun

Hermes Agent ships with self-improvement enabled. That is the right default for a demo environment, a low-stakes personal assistant, or a proof-of-concept. It is the wrong default for a production workload that touches customer data, deploys code, or handles anything with financial consequences. The default configuration assumes the agent will quickly learn what good looks like on your workload. It will learn something — but in the first week, what it observes is your onboarding confusion, your exploratory prompts, your experiments with workflow patterns you will ultimately discard. It encodes all of that as skills.

There is no single flag that cleanly disables self-improvement in the default Hermes config. You have to set it explicitly. Here is the configuration I call “training wheels on” — the starting point for every new Hermes deployment I run:

# config.yaml — Hermes Agent "training wheels on" baseline
# Use this for the first 14 days on any new workload

agent:
  name: "hermes"
  version: "2.4"

skills:
  # Disable autonomous skill creation entirely
  auto_create: false

  # Disable autonomous skill refinement (agent cannot improve its own skills)
  auto_refine: false

  # Memory stays ON — you want observation data, just not autonomous action on it
  memory_enabled: true

  # External dirs for read-only hand-written skills
  # The agent can USE these skills but cannot modify or create new ones
  external_dirs:
    - path: "/home/hermes/skills/hand-authored"
      mode: "read-only"
    - path: "/home/hermes/skills/pinned"
      mode: "read-only"

  # Auto-created skills dir exists but nothing writes to it yet
  managed_dir: "/home/hermes/skills/managed"
  managed_dir_writable: false

memory:
  observation_log: "/home/hermes/logs/observations.jsonl"
  session_summaries: true
  cross_session_retention: true

logging:
  level: "info"
  skill_invocations: true
  skill_outcomes: true

The key decisions in this config: memory stays on because you want the observation data accumulating, you just do not want the agent acting autonomously on those observations. The external_dirs section gives the agent access to your hand-authored skills through a read-only mount so it can use them immediately, but it cannot modify them. The managed directory exists but managed_dir_writable: false ensures nothing auto-generates there even if the agent tries.

Why “Delete Bad Skills Later” Doesn’t Work

The most common objection I hear to this approach is: “just let it run and prune the bad skills afterward.” This sounds reasonable. It does not work, for three distinct reasons.

Reason 1: Self-Congratulation Bias

The agent grades its own homework. When Hermes evaluates whether a skill it created is good, it uses the same reasoning model that created the skill in the first place. A skill that encodes a subtly wrong heuristic will generate outputs that look correct to the model that wrote the heuristic. The skill gets a high self-evaluation score. You have no external signal that something is wrong until the degradation shows up in actual output quality — which is a lagging indicator, often by days. By the time you notice the degradation, you have a week of accumulated sessions that all reinforced the bad skill.

Reason 2: The Skill Overwrite Problem

If you do write hand-authored skills and enable auto-refinement simultaneously, the agent can and will degrade your hand-authored skills. The refinement loop identifies “opportunities for improvement” based on observed usage. If the agent observes that your hand-authored skill for “deploy to production” is invoked frequently but not always successfully (because some failures are user errors, not skill errors), it may “improve” the skill by adding conditionals that change the behavior for the common-failure cases. Your carefully tested skill is now different from what you wrote. The diff is subtle. You will not notice until a deployment fails in a new way.

Reason 3: The Accumulation Rate

On a busy workload, Hermes generates approximately two to four skills per day on default settings. By day 14, you have 28 to 56 auto-generated skills. Auditing 50 skills is not a quick afternoon task — each skill requires you to understand what it encodes, test whether that encoding matches your actual intent, and decide whether to keep, modify, or delete it. In practice, most people do not do this audit. The skills accumulate. The agent’s behavior drifts from what you intended, in small increments, over weeks. The drift is invisible until it is significant.

The math on why starting clean is easier: zero skills to audit on day 14, versus 28–56. The 14-day observation period also gives you the data to write skills that reflect your real workflow, not your first-week exploration of the tool.