Instead of storing the cached manager info in the Blender preferences,
store the info in a JSON file. The file is located in the user prefs
folder (`~/.config/blender/{version}/config` on Linux).
This also reduces the number of 'refresh' operators to a single one, which
then fetches all necessary info from the Manager.
This fixes an issue (reported via chat) where worker tags were sometimes
not retained across file saves.
Fix the database migration that adds `NOT NULL` clauses. It used
`INSERT INTO temp_x SELECT * from x;`, and the `*` returns the fields in
the order they are defined on the table. Since this might be different from
the order that the `INSERT INTO temp_x` expects, strange problems can
happen where columns get swapped (or constraints can fail on columns that
they should not fail for, because they got fed data from a different
column).
This makes it easier to later also create `query_workesr.sql`,
`query_meta.sql` etc. so that the sqlc-generated code can follow the
same subdivision as the persistence service code itself.
No functional changes.
GORM has certain downsides:
- Code-first approach, where queries have to be translated to the Go code
required to execute them.
- GORM comes with its own SQLite implementation, which doesn't provide an
on-connect callback. This means that new connections cannot correctly
enable foreign key constraints, causing database consistency issues.
[SQLC](https://sqlc.dev/) solves these issues for us.
This commit doesn't fully replace GORM with SQLC, but introduces it for
a few queries. Once all queries have been converted, GORM can be removed
completely.
Split the header into two or three parts, depending on the number of
columns shown. The farm status indicator will be above the middle column
(in 3 col mode) or at the right edge of the left column (in 2 col mode).
Also I reverted the hiding of the farm status when SocketIO has
disconnected, as that disconnect happens when navigation between tabs.
That created a too 'blinky' interface, so now it just shows the last-known
farm status.
The database is polled every 30 seconds to determine the farm status; at
startup the first poll is done after 1 second to get a faster status.
Note that when jobs and workers change their status, the farm status is
always updated.
There are still issues with foreign keys getting disabled, so enable them
in the periodic database consistency check.
A more permanent solution is likely to drop GORM and switch to something
else that gives us an on-connect-callback, which can then be used to
turn on foreign key constraints for every connection made.
This introduces the concept of 'event listener', which is now used by
the farm status service to respond to events on the event bus.
This makes it possible to reduce the regular poll period from 5 to 30
seconds. That's now only necessary as backup, just in case events are
missed or otherwise things change without the event bus logic noticing.
SocketIO has 'rooms' and 'event types'. The 'event type' is set via
reflection of the OpenAPI type of the event payload. This has to be set
up in a mapping, though, and if that mapping is incomplete, an error will
now be logged.
Send an event to the event bus whenever the farm status changes. The event
contains a farm status report (like `{status: "active"}`), and is sent to
the `/status` topic.
Note that at this moment the status is only polled every X seconds, and
thus may lag behind other events.
Add a new API operation to get the overall farm status. This is based on
the jobs and workers, and their status.
The statuses are:
- `active`: Actively working on jobs.
- `idle`: Farm could be active, but has no work to do.
- `waiting`: Work has been queued, but all workers are asleep.
- `asleep`: Farm is idle, and all workers are asleep.
- `inoperative`: Cannot work: no workers, or all are offline/error.
- `starting`: Farm is starting up.
- `unknown`: Unexpected configuration of worker and job statuses.
Only include the `blendfile` parameter if it is not empty. This makes it
possible to pass a Python script that loads/constructs the blend file,
instead of loading one directly.
When a more complex list of frames is to be rendered (like `1, 4, 5, 10,
15`), simplify the video filename to `{first}-{last}`.
Before: `somename-1, 4, 5, 10, 15.mp4`
Now: `somename-1-15.mp4`
Add a little intro that explains that there are multiple config files. And
add a section that explains that the worker-specific files are not to be
edited, just deleted.
The worker-written config files would all refer to
`flamenco-worker-example.yaml`, even though this file doesn't even
exist. Instead, the configuration file will refer to the appropriate
documentation on the website, and the credentials file will explain what
happens when you delete it.
The credentials are otherwise intentionally left undocumented, as their
contents are not to be manually edited. The only thing to do with that
file is delete it so that the Worker re-registers itself at startup.
Instead of passing an arbitrary string ("Configuration" or "Credentials"),
use an explicit type for this. This will make it possible to have the
config-writing functions behave slightly differently depending on which
configuration type is being written.
No functional changes.
Send events on Manager startup & shutdown. To make this possible, events
sent to MQTT are now queued up until an MQTT server can be reached.
Otherwise the startup event would be sent before the MQTT connection was
established.
Events were previously only sent via SocketIO, but now they can also be
sent via MQTT. These are now renamed from `SocketIO…` to `Event…`.
There is still the SocketIO subscription mechanism, for which the types are
still prefixed with `SocketIO`. MQTT manages subscription on the broker,
not on Flamenco Manager itself, for now this will remain SocketIO-only
functionality.
Fix SocketIO subscriptions so that the client also subscribes to
job-specific last-rendered images whenever subscribing to job-specific
events. These are sent to another event topic, and thus need some extra
care. Before the introduction of the generic event bus, both message types
were sent to the same topic, but that's not supported by MQTT, and so things
had to change.
