Skip to main content

Jobs and deployments

Jobs​

A job is one unit of work for one agent or connector. Deployments fan out into many jobs; scripts, inventory refreshes and reboots are single jobs.

  • sent is set when the job is written to the WebSocket. Without an ack within 60 s the job goes back to created and is sent again on the next connection. The agent's hello lists the jobs it still holds so both sides reconcile after a reconnect.
  • Agents execute jobs serially per device (no concurrent installers); the server may still queue many.
  • Every job has expires_at; the offline sweep marks expired jobs timeout.
  • A started job (running; agents report a progress message when a job starts) times out timeout_seconds + 5 minutes after it started. The agent runs a device's jobs one after another, so a job it acknowledged but has not started waits while an earlier job of the device is unfinished, and is then timed from when the last of those finished (remote desktop sessions do not count).
  • Results are idempotent: a duplicate job.result for a finished job is ignored.
  • Secrets a job needs (LDAP password, push credential, enrollment token, download links) are added at delivery by job enrichers. The jobs table stores only references or sealed ciphertext.

Deployment scheduler​

Key invariants:

  • deployment_targets is materialised once at start. Devices enrolled later are not added automatically; Add new devices re-runs the filter.
  • A pass picks pending targets of online devices only, respecting max_concurrency (targets in queued, downloading or installing) and the maintenance window in the device's site time zone.
  • The window is evaluated in SQL: deployment_in_window(now, coalesce(site tz, tenant default tz), start, end).
  • One pass per deployment at a time (SELECT … FOR UPDATE on the deployment row; target results lock the deployment first, in the same order, and deadlocks are retried), so several replicas can tick safely.
  • Passes run every 30 s, on every agent hello that has pending targets, and after every target result (coalesced per second). A pass also settles targets whose job ended without a result.
  • A failed or timed-out target with retries left returns to pending with next_attempt_at = now + retry_backoff × 2^(attempt-1).
  • When the deployment expires, pending and queued targets time out (queued jobs are cancelled on the agent).
  • The deployment's status and counters are recomputed from its targets on every change and cached on the row.
  • Jobs store no download link. The package enricher presigns a fresh one-hour URL each time the job is delivered; agents refresh an expired link with GET /api/agent/v1/packages/{id}/download-url.

Target resolution​

all, devices (explicit list) or filter (the device filter plus software_present / software_absent conditions with an optional version comparison using version_cmp). Devices without an agent and decommissioned devices are excluded and counted in excluded_count. At most 20,000 targets (too_many_targets).

The scheduler is tested by entrosity-axis.backend/internal/deploy/scheduler_integration_test.go, which simulates a fleet across time zones, windows, retries and reconnects.