Skip to content
Docs

Export your data

Download everything Estelle holds for your workspace as JSONL files, including training files for a local or custom model.

View as Markdown

Estelle can export everything it holds for your workspace: memories, session transcripts, the code written in those sessions, Monitor issues, traces and logs, QA, security, testing, on-call and Atlas data. You can keep the files, move them, or use them to train a model.

Start an export from Settings → Export data. Owners and admins can start one. A solo account owns its workspace.

#What you get

An export runs in the background and keeps going if you close the page. It writes gzipped JSONL files, at most 8 MB each before compression, and keeps them for seven days. Only one export runs at a time for an account, and only one that finishes complete replaces the last complete one you kept. Past 1 GB stored, the remaining sections say could_not_export. An account can export with no credits left.

GroupWhat it holds
SessionsEvery session event: messages, thinking, tool calls and results. Also titles, diary entries, prompts from Ask and Agent, Rooms and plans.
Trainingeden.agent_examples.jsonl.v1 and eden.trace_records.jsonl.v1 files built from your sessions.
MemoriesFacts, decisions, checkpoints, conversations, Plates and knowledge cards.
CodeEvery edit, workspace diff and commit recorded in a session, code graph revisions, and the source chunks Estelle indexed.
ProductionMonitor issues with stacks and repair patches, per-minute error counts, spans and logs, alert rules, uptime checks and usage events.
On-callAwake incidents, pages and digests.
QA and testingTwice campaigns, results, findings and text evidence, plus test runs recorded in sessions.
SecurityScans, findings and conversations about them.
AtlasSources, indexes, chunks and published answers.
OtherOrchestra runs, Silo running time, credit use, Intake documents and settings.

#The manifest

Every export has a manifest.json.gz. It lists every section with its row count and schema version, and everything that was not exported and why.

A section ends in one of three states:

  • exported: rows shipped.
  • empty: the section was read to the end and held nothing.
  • could_not_export: the read failed. reason says how. Rows it already wrote are real, but the section is not whole.
  • withheld: your team's session rule keeps it from you. reason says so. You do not hold it, so the export is not complete.

complete is true only when you hold everything you asked for: every section was read to the end, none shipped fewer rows than it counted when the export reached it, no session was too large to read whole, and your team's session rule kept nothing from you. A credential taken out or withheld does not change it. An export is not a snapshot: sections are read one after another, so a row written after its section was read is not included.

#Keys and credentials

The stores that hold provider keys, API keys, tokens, password hashes and connection secrets are not read at all.

A credential someone pasted into a transcript is different, because it is just text. It is removed by field name, by exact value for your own account key and namespaces, by named values in source, YAML, JSON and URLs, and by known shapes. Each row is then scanned again, and a row where anything still matches is withheld and counted rather than shipped.

This cannot catch everything. It can miss:

  • a credential with no recognised prefix and no telling name beside it;
  • a credential split across two fields, two events or two lines;
  • a credential that is hex-encoded, URL-encoded, compressed, or base64-encoded twice (plain base64 is read, once);
  • a password written in ordinary words.

Personal data is not removed. Names and email addresses in your own transcripts ship, because they are your data. Read the files before you share them.

Images and recordings are listed with their size but not attached.

#Training a model

The training files fit Eden's import formats.

eden.agent_examples.jsonl.v1 pairs each request with the answer that finished it, or with the diffs written for it. Each file is at most 1 MiB and 1,000 examples, and a session stays in one file where it fits. An example Eden would refuse is left out and counted, never trimmed. The manifest gives the count per reason, such as too_long, reserved_marker or harness_message.

eden.trace_records.jsonl.v1 keeps whole sessions, tool calls and results included, with the same file limits.

Each file fits one upload to Eden. Eden holds 32 datasets and 16 MiB per project, and its current training recipe takes 64 examples, so a large export will need choosing from.

An answer in a training file is the last thing the agent said in that turn. That does not prove it was right. Rights are not_assessed and no train or test split is assigned. Example ids start with their session's hash, so you can keep one session wholly on one side of a split.

#From a terminal or an agent

Use an API key with full access.

bash
curl -X POST -H "Authorization: Bearer $ESTELLE_API_KEY" -H "Content-Type: application/json" \
  -d '{}' https://api.fatelabs.ca/export/jobs
curl -H "Authorization: Bearer $ESTELLE_API_KEY" https://api.fatelabs.ca/export/jobs/$JOB_ID
curl -H "Authorization: Bearer $ESTELLE_API_KEY" -o estelle-export.tar \
  https://api.fatelabs.ca/export/jobs/$JOB_ID/archive

GET /export/jobs lists your exports and every section name. To export only some sections, send {"sections": ["session_events", "memory_notes"]}. One file is at /export/jobs/{id}/files/{n}, and POST /export/jobs/cancel with {"job_id": "..."} stops an export.

An agent can do the same with the estelle_export MCP tool.

#What is not in an export

These are also listed on every manifest.

  • Traces and logs older than the archive. Monitor now keeps spans and logs for 30 days. Before that it kept them for an hour.
  • Metrics. Metric points are kept for an hour and are not archived.
  • Span attributes outside a safe list. A span keeps its timing, status, GenAI usage and a fixed list such as the route and the database system. Other values are dropped when they arrive, and their names are kept. Monitor does not store prompts or completions from your spans.
  • Every occurrence of an error. An issue keeps its latest sample, its stack and a count.
  • Stacks and releases an error never sent. An error sent without a stack is counted but has no code location.
  • CI job records. Test results recorded in sessions are included.
  • Files in a Silo. A sandbox's files go when it goes. The diffs and artifacts a session recorded remain.
  • Sessions with no capture. A session on a host without the Estelle hook never reached Estelle.
  • Embeddings. They are rebuilt from the text.

On a team, Monitor, security and credit rows belong to each member's own account, so each member exports their own. Leave a team and its exports stop being yours to download. If your team restricts who can see whose sessions, that rule applies to an export too: hidden sessions, and the checkpoints and cards drawn from them, are left out and counted. Sections that cannot be tied to one session, such as Rooms, ship if the rule hides nothing from you, and otherwise say withheld. A rule set after an export finished does not change files already written.

The older GET /export still returns the code graph and learned state as one document, in pages if it is large.