cockpit doctor

This reference is generated from the same canonical source distributed as cockpit CLI help. See Cockpit CLI for concepts and safe operating guidance.

Doctor inspects only the local instance; --host returns host_not_supported.

Usage: cockpit doctor [--heap-profile start|stop|status]

Cross-checks cli-runtime.json and AGI_COCKPIT_*_PORT overrides against the local process. The result includes instance, the local Cockpit this command resolved as its target (production, dev, or dev-task-<hash>), derived from the runtime file or inherited IPC directory actually used. pidVisibility is alive, dead, or unknown; a sandbox can make a running process unknown. pidAlive remains for compatibility and is true only for alive.

For each local service (taskPort, autorunPort, cockpitPort, askPort), the result keeps the existing runtime/environment/effective port and listening fields and adds directlyReachable, which describes what this CLI caller can reach over loopback rather than claiming the port is stopped. At the report level, transports.loopback and transports.fileIpc show each task API authentication attempt and its failure reason, directlyReachable summarizes direct task API reachability, and effectiveTransport is loopback, file, or null. The task check requires a successful request with the advertised token and a 401/403 response with an invalid token. A loopback failure is followed by the same check through file IPC.

Each service also reports runtimeReason, envReason, and the effective reason: a connection error such as ECONNREFUSED, a timeout, authentication_failed, or not_advertised; a successful direct check has a null reason. For the task API, listenerState is the app's recorded taskServer state, including status (starting, listening, recovering, or stopped), listening, port, retryAt, and lastFailure (timestamp, phase — bind, error, or close — port, code, and message). It is null for an older runtime without listener diagnostics. Bind and listener failures also remain in main-process-errors.log after recovery. The caller's direct reachability and the app's recorded listener state are separate evidence: a sandbox or a delayed HTTP response can prevent direct authentication while the app is still listening.

Task and talk commands recover an injected stale connection only when its IPC directory proves that the replacement runtime belongs to the same Cockpit instance. Other explicit connections never redirect to the runtime file or its file bridge. A CLI bound to one local instance ignores inherited connection settings that point at another; if it would still send there, it fails with code: "instance_mismatch" before any request is sent. Local JSON results include the same instance field. Commands that target another machine with --host omit it.

From the PowerShell launcher, the report also includes nodeRuntime: the path of the runtime that ran doctor and its source — environment (AGI_COCKPIT_NODE_PATH), cockpit (the AGI Cockpit executable: the running app, the path recorded in the runtime file, or the installed Microsoft Store package, so doctor also runs while the app is closed), or path (node on PATH). Without any runtime, doctor returns code: "node_runtime_unavailable"; a runtime that prints no JSON returns code: "node_runtime_failed".

mainProcessStall is null while the main process is advancing. An independent worker watches a heartbeat sent every second. After 30 seconds without progress, it writes a marker next to the runtime file, appends a main-process-stalled record to main-process-errors.log, and shows a native notification. The report includes status, pid, instanceId, heartbeatAt, detectedAt, stalledForSeconds, and thresholdSeconds. A matching marker makes doctor report Main process stalled for Ns and exit 7 without waiting for the stalled servers. Markers from another process instance are ignored. The marker clears on recovery, which appends main-process-recovered. A system sleep grants a fresh detection interval. The watchdog never restarts or kills the app.

crashDumps summarizes the local crash dumps of that Cockpit: how many came from Cockpit itself, how many came from unrelated child processes that inherited the crash handler, how many could not be identified, how many dumps are retained, and how many were deleted as external or over the retention limit. It is null until Cockpit has scanned its crash directory once.

fileWatchers summarizes the file watchers the main process holds: the total, the split between chokidar and fs watchers, per-purpose counters (task-status, file-watch, creative-studio-workspaces, …) with how many directories each purpose watches and how many entries those directories actually hold on disk, and the largest watched directories by entry count. Entry counts are read from the directory itself, so a watcher that fell back to a huge parent directory is visible even before it admits any child. At most 256 directories per watcher are measured; measuredDirectories reports how many of watchedDirectories were counted. Directory paths are hashed. It comes from the latest main-process-metrics.jsonl sample and is null until Cockpit has written one. A watcher whose directory holds tens of thousands of entries rescans that directory on every change and is the shape of a main-process memory leak.

mainProcessMemory reports the main-process memory from the latest main-process-metrics.jsonl sample: sampledAt, uptimeSeconds, privateBytes, rssBytes, heapUsedBytes, heapTotalBytes, workerHeaps, and privateGrowth. It is null until Cockpit has written a sample. privateBytes is the memory owned by the main process alone (process.getProcessMemoryInfo().private; on macOS it is about the same as phys_footprint), and it is null in samples written before it was recorded or when it could not be read within one second. workerHeaps holds the V8 heaps of the worker threads, which the main-process heap figures do not include: goalUpdate (the goal-update worker) and watchdog (the stall watchdog), each with usedHeapBytes, totalHeapBytes, physicalBytes, externalBytes, and mallocedBytes, or null while that worker is not running or does not answer within one second. privateGrowth is how much privateBytes changed across the samples of the current process (each sample records the main-process pid) in the last 256 KB of the log: windowSeconds, bytes, and bytesPerHour, or null with fewer than two such samples. Judge main-process memory growth by privateBytes and privateGrowth, not by RSS minus heapTotalBytes: RSS still counts freed pages the system has not reclaimed, a shrinking JS heap makes that difference grow while RSS stays flat, and worker heaps fall into it too.

accounts summarizes the agent accounts of that Cockpit as reported by cockpit accounts list, fetched through the effective transport after the task API authenticates: fetched, error, total, ok, an expired list and a signedOut list of <agentType>:<name> labels (followed by (<email>) when the account's email address is known), and the entries with agentType, profileId, name, emailAddress, authState, loggedIn, and usageState. An account whose stored credential no longer works is listed under expired and must be signed in again before tasks or Fleet Runs can use it. accounts is null when no transport authenticates. The fetch waits up to 60 seconds; on a cold start, when every provider's usage cache is empty, it can still time out, in which case fetched is false and error says to run cockpit accounts list first and then cockpit doctor again.

grokTurnFailures summarizes the local Grok turn-failure log of that Cockpit: available, entryCount, and the recent entries, read from grok-turn-failures.jsonl and its one archive in the logs/ directory next to the runtime file. AGI Cockpit appends one line per failed Grok turn and rotates the file at 128 KB, so it cannot grow without limit. Each entry holds failurePath (prompt_response, prompt_exception, or provider_initiated), the resolved stopReason, retained — the rate-limit classification still held from earlier in the turn and where it came from (retry_state) — retryState with the retry state's own state, is_rate_limited flag, and reason, provider with the provider's failure reason, code, own stopReason, and rate-limit flag, model, usageLimitKnown (whether a reset time was known when the turn failed), and messageSource, the branch that produced the displayed text (usage_limit, provider_detail, or generic_fallback). It exists to tell a real usage-limit failure apart from a generic one when a turn shows the generic Grok failure message. The most recent entries carrying a rate-limit signal are kept in recent even when later ordinary failures would otherwise push them out. reason is the provider's own failure text and is the only free-text field kept: up to 200 characters of it, after URLs are reduced to their host and e-mail addresses, credentials, cookie headers, file paths, and opaque 32-character-and-longer tokens are redacted. A provider message that quotes the request back is therefore recorded as the provider wrote it — that text is the failure reason, which is the point of the log — so treat reason as provider-authored prose rather than as a machine field. Nothing else is taken from the payload: only the fields listed above are read, the payload itself is never serialized, and no tool output, token, key, header, or cookie is written. Identifier fields — model, code, stopReason — are redacted by the same rules and bounded to 64 characters, so an identifier that looks like a key or a UUID is recorded as <redacted>. Each string is also bounded before redaction, so a long provider string cannot stall the recorder. The same summary is part of the diagnostics export written by the app's Help menu and by its --export-diagnostics=<path> startup argument — that argument belongs to the AGI Cockpit application, not to cockpit.

This is the app-reachability diagnostic. It is not cockpit app doctor, which inspects Android/iOS App Surface targets.

--heap-profile start begins a V8 sampling heap profile of the main process, --heap-profile stop ends it and writes a .heapprofile file under heap-profiles/ in the Cockpit diagnostics directory, and --heap-profile status reports whether one is running. The JSON result includes directory and, after a stop, filePath. The main process also starts a sampling profile on its own when its JS heap passes 1.5 GB or grows by more than 300 MB in one minute, and stops it at about 2.75 GB or after 60 seconds. An automatic profile is taken once per episode: it does not start again until the heap has fallen back below 1 GB, so a process that stays large writes one profile rather than one per minute. Sampling is used rather than a full heap snapshot because taking a full snapshot near the heap limit can itself exhaust memory. Profiles record allocation stacks only; task IDs are hashed in metrics and no prompt, message, or tool output is written.

Exit 0 when either loopback or file IPC authenticates with the task API, including when PID visibility is unknown. Exit 7 when a current stall marker is present or neither transport authenticates, including when the runtime file is unavailable and no transport can be selected.

When the task listener is starting or recovering without an advertised port, doctor and task commands wait within the same 20-second authentication deadline for the same PID, token, and IPC directory to advertise a listening port. It does not send credentials to the pending or previous port.

After the same instance migrates to a new port, doctor uses the current advertised port and reports effectiveSource and effectiveTokenSource as runtime. It does not probe the inherited previous port, whose envListening is false and envReason is not_advertised.