Scheduled jobs
Plonex uses Huey with a persistent SQLite queue by default. No Redis server or system crontab is required for the default setup. Jobs can be run manually, queued, or scheduled; all execution paths share history, logs, timeouts, and file locks. The scheduler runs under Supervisor. See the Huey guide for the underlying periodic-task model.
When cron_jobs contains any jobs, dependency generation adds plonex and
huey[stats] to the target project’s requirements. dependencies install,
dependencies sync, and Supervisor startup preparation install them with the
project’s interpreter and constraints. The Plonex requirement is unpinned; specify
an editable source in the project’s requirements or source configuration if it
must be preserved during dependency synchronization. The worker protocol must
match the CLI protocol.
plonex cron daemon replaces its launcher with the target interpreter running
plonex.services.cron.runtime. Supervisor uses that entry point directly. The
worker and each Plone job therefore use the target environment. The worker itself
does not need a permanent ZODB connection; each Plone callable still runs through
a fresh target-venv zconsole subprocess.
cron run, cron enqueue, and cron daemon prepare missing runtime dependencies.
Read-only inspection commands report a missing runtime without installing it.
Use plonex dependencies install to prepare explicitly. Constraints must permit
Huey 3.5 or newer, below 4; incompatible pins cause a normal resolver error.
Queue backend
Configure cron_backend and cron_backend_options to select a Huey storage
backend and pass its storage options. For example:
cron_backend: redis
cron_backend_options:
url: redis://localhost:6379/0
Available values are sqlite (the default), cy-sqlite, redis,
redis-expire, priority-redis, priority-redis-expire, postgres, file,
memory, and blackhole. Options are passed to the corresponding Huey storage
class; consult the Huey storage documentation
for supported arguments. For SQLite, filename defaults to
var/cron/queue.sqlite3 and relative paths are relative to the project root.
For file storage, path defaults to var/cron/queue. Redis, PostgreSQL, and
Cython SQLite drivers are installed as needed when dependencies are prepared.
The memory backend is local to each process, while blackhole discards tasks;
neither provides a shared persistent queue. History and statistics remain in
local SQLite databases regardless of the queue backend. Switching backends does
not migrate existing queued tasks. All producers and workers for a project must
use the same settings. Restart the worker after changing them.
A Plone job without HTTP credentials
Move the work into a trusted Python function installed in the project virtualenv:
# my.addon/jobs.py
import logging
logger = logging.getLogger(__name__)
def store_session_stats(context, days=1):
# Perform the application's work here, using context and normal Plone APIs.
# Do not commit here: the job runner owns the transaction.
count = update_statistics(context, days=days)
logger.info("Stored statistics for %s sessions", count)
return {"sessions": count}
update_statistics represents your existing domain implementation. The runner
calls the configured function with the traversed context and keyword arguments.
The function must not require a browser request or an HTTP authenticator.
Declare the job:
cron_timezone: Europe/Rome
cron_workers: 1
cron_jobs:
StoreSessionStats:
schedule: "15 2 * * *"
callable: my.addon.jobs:store_session_stats
site: /Plone
context: sectors
user: scheduler
runtime: cron
kwargs:
days: 1
timeout: 900
lock: zope
lock_timeout: 300
enabled: false
Create the named scheduler user in Plone and give it the roles the work requires.
The job does not need that user’s password. It opens a dedicated ZEO client through
zconsole, sets the Plone site hook, adopts the named user using plone.api, and
commits on success. Exceptions abort the transaction; the user and site context
are restored. ZODB connections are isolated in a fresh subprocess for each run.
The project must have Plone, its add-on, and zconsole installed, and ZEO must be
available. Normal Zope configuration, ZCML additions, blob paths, and environment
settings apply. runtime: cron selects etc/plonex-cron.yml overrides.
To set the public URL for a callable job, optionally provide a Zope virtual-host traversal path, for example:
vhm: /VirtualHostBase/https/example.com:443/Plone/client/
The runner traverses this path on the job’s synthetic request before invoking
the callable. Use the same VirtualHostRoot and _vh_ directives as in your
proxy configuration when the public path differs from the physical Zope path.
The path must resolve in the target Zope instance. site still identifies the
physical Plone site and context remains relative to that site.
This is trusted server-side code running as the deployment OS account. Naming a Plone user provides the application security context; it is not a sandbox for untrusted Python code.
Keep the browser view protected
The browser view can call the same domain function after checking its normal permissions, HTTP method, and authenticator:
from Products.Five.browser import BrowserView
from plone.protect import CheckAuthenticator, PostOnly
from my.addon.jobs import store_session_stats
class StoreSessionStats(BrowserView):
# Register with an appropriate permission in ZCML.
def __call__(self):
PostOnly(self.request)
CheckAuthenticator(self.request)
return str(store_session_stats(self.context))
Test it from an authenticated browser form using POST and an authenticator token.
The scheduled job invokes the function, not the view. It does not disable CSRF,
store Basic Auth passwords, or manufacture authentication cookies. Plone accepts
an authenticator form field or X-CSRF-TOKEN header for browser requests; see
Plone’s CSRF documentation.
Test, enable, and deploy
enabled: false disables the periodic schedule while allowing explicit manual
runs and enqueues. It does not cancel work already stored in the queue:
plonex cron list
plonex cron run StoreSessionStats
plonex cron logs StoreSessionStats
plonex cron history StoreSessionStats
After verifying the result, set enabled: true and run:
plonex cron configure
plonex supervisor start
If Supervisor is already running, use plonex supervisor update to apply the
new program. The worker program is rendered whenever Supervisor configuration is
prepared and jobs are configured. Manual-only jobs (enabled: false) still need a
consumer for explicitly queued work, so they keep the Supervisor worker. Removing
all job declarations removes plonex-cron.conf; apply that change with
supervisor update.
Job definition changes require restarting the daemon, for example with
supervisor restart. This restarts all managed programs, so plan it accordingly.
For foreground development:
plonex cron daemon --workers 1
To queue an immediate run for the daemon:
plonex cron enqueue StoreSessionStats
Enqueuing does not start a daemon. The persistent queue waits until a consumer is running. Only one scheduler daemon may own a project’s daemon lock. Workers run job subprocesses; each job’s timeout bounds its execution. Normal daemon shutdown terminates active subprocess groups and records their outcomes.
Queue monitoring and controls
plonex cron status
plonex cron queue
plonex cron inspect TASK_ID
plonex cron retry TASK_ID
plonex cron revoke TASK_ID
plonex cron restore TASK_ID
status shows consumer state, queue depths, and execution statistics. Consumer
state is checked through the daemon lock rather than assuming a recorded PID is
still alive. queue lists pending and scheduled work, including its task IDs,
due times, revocation flags, and whether the job definition still exists.
inspect joins queue metadata, Huey events, and execution attempts. Huey’s task ID
is retained across automatic retries; each attempt has its own run ID, attempt
number, exit status, and private log. A manual retry creates a new task ID and uses
the current job definition. An automatic retry already waiting in the queue or
schedule must finish or be handled before a manual retry is accepted.
revoke prevents future execution of that task ID. It does not terminate a running
job or immediately remove the queue entry. restore removes that revocation.
Pausing a periodic job uses enabled: false and a daemon restart. Keep a job’s
definition until its queued work has drained; removing it leaves unregistered tasks
that the consumer cannot execute.
All monitoring/control commands accept --format json; status and queue accept
--limit up to 1000. Full execution history is also available as JSON:
plonex cron status --format json
plonex cron queue --limit 100 --format json
plonex cron inspect TASK_ID --format json
plonex cron history --format json
plonex project describe queue
The same project-runtime API backs these commands, the report, and the optional Plone control panel described below. No separate web service is needed.
Plone queue control panel
Set cron_control_panel: true in the project YAML, install dependencies, and
restart the Plone instance. The option enables the queue view and supplies the
project path to the instance. It also requests plonex and Huey even if no jobs have
been declared yet.
Users with the Manage portal permission can visit /Plone/@@plonex-queue
to inspect the worker, queue and recent attempts, and enqueue, retry, revoke, or
restore tasks. Changes use POST with Plone’s authenticator check. Revocation
prevents future execution and does not terminate a running task. The page does not
install runtime dependencies; run plonex dependencies install explicitly if
the runtime is unavailable.
To add a link in Site Setup, apply the GenericSetup profile
profile-plonex.services.cron:queue-control-panel to each site. For example, run
a trusted Zope script with plonex instance run -- SCRIPT.py:
import transaction
from zope.component.hooks import setSite
site = app["Plone"]
setSite(site)
site.portal_setup.runAllImportStepsFromProfile(
"profile-plonex.services.cron:queue-control-panel"
)
transaction.commit()
Monitoring uses Huey’s statistics extension, the recorder behind its Django and Flask dashboards. It is enabled in producers and consumers. Arguments are not captured. Events are buffered briefly and retained for 48 hours, up to 2000 events per queue, using Huey’s defaults. In-flight records are last-observed execution state: an abruptly killed worker can leave them until retention cleanup. They are shown separately from whether a worker is actually running. Job logs retain the program’s original output and may contain values the job prints.
Producing work from application code
The target runtime exposes QueueRuntime for application producers:
from pathlib import Path
from plonex.services.cron import CronService
from plonex.services.cron.runtime import QueueRuntime
with QueueRuntime(CronService(target=Path("/path/to/project"))) as queue:
receipt = queue.enqueue("StoreSessionStats")
task_id = receipt["task_id"]
Use one project queue per process and close short-lived producer instances so buffered statistics are flushed. In a Plone request, publish only after a successful ZODB commit, for example through a transaction after-commit hook that checks its success argument. Publishing and the ZODB commit are separate operations; applications needing durable publication should use a persistent outbox and idempotent processing. Queue arguments should contain identifiers or paths rather than persistent objects. The configured job controls callable and execution user.
Other commands and execution controls
A job must choose exactly one of callable or an argv-list command:
cron_jobs:
maintenance:
schedule: "*/15 * * * *"
command:
- /absolute/path/to/.venv/bin/python
- -m
- mypackage.maintenance
environment:
MODE: development
timeout: 120
Commands run from the project root without a shell and with stdin closed. The
target virtualenv’s bin directory leads PATH, and VIRTUAL_ENV identifies it.
Configured runtime environment values apply, with per-job environment values
taking precedence. Shell operators, redirects, pipes, and variable expansion are
not interpreted. Output goes to a private log.
Plone jobs default to the shared plone lock; command jobs default to a lock named
after the job. Set a common lock to serialize related jobs. A busy lock skips the
run after lock_timeout seconds; 0 means skip immediately. Locks release when
the owning process exits, including after crashes.
Schedules have five fields: minute, hour, day of month, month, weekday. Timezones are IANA names; the default is UTC, independent of the machine’s timezone. Huey’s cron matcher requires all specified fields to match, including day and weekday. Schedules are checked once per minute. Missed schedules are not replayed after an outage. Local DST transitions can skip or repeat wall-clock times.
retries defaults to zero. Optional retries and retry_delay configure queue
retries on failed runs; manual cron run executes once. Write idempotent jobs,
especially when they send mail or call external systems. ZODB rollback cannot
undo external side effects, and abrupt process termination is not an exactly-once
execution guarantee.
State is under the configured var_folder:
cron/queue.sqlite3: default SQLite Huey queue.cron/history.sqlite3: task IDs, execution attempts, exit codes, timestamps, and log paths.cron/stats.sqlite3: Huey lifecycle events and in-flight statistics.cron/daemon.json: identity of the lock-owning worker while it runs.cron/logs/: per-run stdout/stderr logs.cron/locks/: process-safe lock files.log/cron.log: the Supervisor daemon log, with rotation.
Queue, history, payloads, and per-run logs are created with owner-only file
permissions. Temporary Plone payloads and runtime directories are removed after
the run. Logs retain the program’s original output; avoid printing secrets in
job code. Reports show job declarations and recent log links without displaying
job output. Use project describe jobs --browse to inspect scheduling state.
Execution history and per-run logs have no automatic retention policy yet; include
them in your deployment’s storage monitoring and archive policy.