1847 Commits

Author SHA1 Message Date
Sybren A. Stüvel
99062d42ac Website: add draft documentation for failure handling
The documentation isn't complete yet, I want to have a nice flow diagram
in there as well. But it's a start.
2024-02-22 13:00:24 +01:00
Sybren A. Stüvel
dfdb8e82a1 Worker: Refer to website instead of non-existent example file
The worker-written config files would all refer to
`flamenco-worker-example.yaml`, even though this file doesn't even
exist. Instead, the configuration file will refer to the appropriate
documentation on the website, and the credentials file will explain what
happens when you delete it.

The credentials are otherwise intentionally left undocumented, as their
contents are not to be manually edited. The only thing to do with that
file is delete it so that the Worker re-registers itself at startup.
2024-02-22 12:46:13 +01:00
Sybren A. Stüvel
7033028a0b Worker: use explicit type when writing config file
Instead of passing an arbitrary string ("Configuration" or "Credentials"),
use an explicit type for this. This will make it possible to have the
config-writing functions behave slightly differently depending on which
configuration type is being written.

No functional changes.
2024-02-22 12:46:13 +01:00
Sybren A. Stüvel
12bfa82854 Manager: add lifecycle events to the event bus
Send events on Manager startup & shutdown. To make this possible, events
sent to MQTT are now queued up until an MQTT server can be reached.
Otherwise the startup event would be sent before the MQTT connection was
established.
2024-02-21 22:20:56 +01:00
Sybren A. Stüvel
fd9605583f OAPI: regenerate code 2024-02-21 22:16:23 +01:00
Sybren A. Stüvel
a137ecd3cd OAPI: add lifecycle events
Add events for manager-startup and manager-shutdown.
2024-02-21 22:12:43 +01:00
Sybren A. Stüvel
e7bd8dff8a Bump Go version to 1.22 2024-02-21 21:23:08 +01:00
Sybren A. Stüvel
17c92e18c7 Website: adjust documentation for the new event bus
Needs more work though. At least now it doesn't point to non-existing
files any more.
2024-02-05 09:27:06 +01:00
Sybren A. Stüvel
e7c4285ac6 Manager: Adjust code for renaming SocketIO... types to Event...
No functional changes, just adjusting to the OpenAPI renames.
2024-02-05 09:25:43 +01:00
Sybren A. Stüvel
1ed893fa84 OAPI: regenerate code 2024-02-05 09:14:31 +01:00
Sybren A. Stüvel
441079aa06 OAPI: rename SocketIOxxx types to Eventxxx types
Events were previously only sent via SocketIO, but now they can also be
sent via MQTT. These are now renamed from `SocketIO…` to `Event…`.

There is still the SocketIO subscription mechanism, for which the types are
still prefixed with `SocketIO`. MQTT manages subscription on the broker,
not on Flamenco Manager itself, for now this will remain SocketIO-only
functionality.
2024-02-05 09:11:13 +01:00
Sybren A. Stüvel
3326f683da Manager: consistent MQTT server/broker naming
Consistently log about the MQTT "broker", not the "server". The former
is common MQTT terminology.
2024-02-04 18:28:12 +01:00
Sybren A. Stüvel
91d15df765 Manager Cleanup: consistent variable name for MQTTClient receiver pointer
No functional changes.
2024-02-04 17:03:38 +01:00
Sybren A. Stüvel
44bfe58891 Manager: friendlier log message when MQTT server connection cannot be made 2024-02-04 16:32:36 +01:00
Sybren A. Stüvel
f0c7acd903 Manager: fix web interface not showing last-rendered images on job view
Fix SocketIO subscriptions so that the client also subscribes to
job-specific last-rendered images whenever subscribing to job-specific
events. These are sent to another event topic, and thus need some extra
care. Before the introduction of the generic event bus, both message types
were sent to the same topic, but that's not supported by MQTT, and so things
had to change.
2024-02-04 16:12:16 +01:00
Sybren A. Stüvel
4f804958e5 Manager: add unittest for eventbus topics
The code was doing its work just fine, but I wanted to be sure.
2024-02-04 16:12:16 +01:00
Sybren A. Stüvel
dd98c7471d Manager: don't log event payload in event logging
Don't log event payload in MQTT/SocketIO debug logs. It's getting too
noisy.
2024-02-04 16:11:58 +01:00
Sybren A. Stüvel
740ede80fa Manager: log eventbus events with 'eventbus' prefix instead of 'socketio'
These messages are now no longer SocketIO-specific, so should use the
'eventbus' prefix.
2024-02-04 16:08:45 +01:00
Sybren A. Stüvel
b375acb1a1 Cleanup: add SPDX license identifiers 2024-02-03 23:42:51 +01:00
Sybren A. Stüvel
4fe8605744 Manager: Add MQTT client for sending events
Add an MQTT client to send events from the event bus to an MQTT broker.
2024-02-03 23:20:15 +01:00
Sybren A. Stüvel
2af9a3e98e Webapp: update documentation URL to the SocketIO client docs
No functional changes.
2024-02-03 22:55:27 +01:00
Sybren A. Stüvel
76a24243f0 Manager: Introduce event bus system
Introduce an "event bus"-like system. It's more like a fan-out
broadcaster for certain events. Instead of directly sending events to
SocketIO, they are now sent to the broker, which in turn sends it to any
registered "forwarder". Currently there is ony one forwarder, for
SocketIO.

This opens the door for a proper MQTT client that sends the same events
to an MQTT server.
2024-02-03 22:55:23 +01:00
Sybren A. Stüvel
4f8d39f74a Fix MSYS2 breaking the base URL for webapp
Add `MSYS2_ARG_CONV_EXCL="*"` to avoid MSYS2 changing the yarn build
`--base=/app/` argument. That's a URL path, not a filesystem path, and it
should be used as-is on every platform.
2024-02-02 22:37:28 +01:00
Sybren A. Stüvel
f464aea137 Manager & website: provide more helpful info when Worker auth fails
Provide more useful info when a Worker tries to communicate but fails
the authentication check. The message about this is now more friendly
and links to a new FAQ entry at
https://flamenco.blender.org/faq/#what-does-unknown-worker-is-trying-to-communicate-mean
2024-01-25 14:19:24 +01:00
Sybren A. Stüvel
26c14bf095 Website: better styling for FAQ page
More vertical space between answers in the FAQ.
2024-01-25 13:04:45 +01:00
Sybren A. Stüvel
9afd79d8c0 Manager: prevent logging an error when fetching unknown worker
Prevent logging an error in the persistence layer when an unknown worker
is requested.

This reduces the noise & confusion when the web interface is showing the
details of a worker, but the worker gets removed by someone else. Or when
the Manager doesn't know about a Worker and it's trying to connect.

See #104282.
2024-01-25 12:38:13 +01:00
Sybren A. Stüvel
70faa4e225 Move URLs to the Flamenco website to constants in a dedicated package
Create a dedicated package `.../pkg/website` to contain constants for the
URLs of documentation, bug reporting, etc. That way it's easier to see
which parts of the website are being referred to from the Flamenco
binaries, and updates can happen in a central spot.

No functional changes.
2024-01-25 12:25:06 +01:00
Sybren A. Stüvel
13a793743f Website: add documentation for worker tags and sleep schedule
The worker tags and sleep schedule features are now actually documented.
2024-01-25 12:15:29 +01:00
Sybren A. Stüvel
c3a306a9c5 Website: slight layout tweaks
A few smaller tweaks for the navigation menu (on the left, not the top):

- Smaller font. The upgrade to the latest Geekdocs made this bigger, but I
  like it if it's a bit smaller than the main text.
- More visible indicator line for the active element. This was a dashed
  underline of just the text. It is now a dashed border-bottom that extends
  the entire width of the element. This visibly connects the main body of
  text to the active menu element.

And for the main text:

- Less padding for `h3` elements, as the whitespace became a bit too much.
2024-01-25 12:14:27 +01:00
Sybren A. Stüvel
3233d40b4c Website: upgrade Hugo to 0.121.2
Upgrade Hugo from 0.101.0 to 0.121.2. This seems to not introduce any
changes for us.
2024-01-25 12:11:04 +01:00
Sybren A. Stüvel
781aaa8282 Website: upgrade geekdocs to 0.44.1
Upgrade the Geekdocs theme from 0.32.4 to 0.44.1. This changes the layout
a little bit; most notably the 'documentation' menu is in a larger font.

I tried the upgrade to solve an issue of images not appearing (while
writing not-yet-committed changes). That wasn't solved by the upgrade, but
in the spirit of keeping up to date I'd thought I'd commit this upgrade
anyway.
2024-01-25 11:29:22 +01:00
Sybren A. Stüvel
fd29f58c2e Website: move info about cloud storage into the main documentation
Move the info about using cloud storage from the FAQ to the Shared
Storage chapter of the documentation. The FAQ entry of the topic is now
reduced to a short answer + a link to the moved documentation.
2024-01-23 10:27:20 +01:00
Sybren A. Stüvel
27eb7be547 Website: reorganise FAQ
Reorganise the FAQ, grouping the questions in "Features" and
"Troubleshooting". Also I removed the explanation of the difference
between Flamenco v2 and v3, as that's quite obsolete now.
2024-01-17 13:22:57 +01:00
Sybren A. Stüvel
aa2ad35155 Bumped version to 3.5-alpha0 2024-01-17 13:17:35 +01:00
Sybren A. Stüvel
19d4019142 Website: add FAQ entries on GPU use of Blender
This comes up a lot.
2024-01-17 13:16:31 +01:00
Sybren A. Stüvel
f9520ce147 Website: remove FAQ about a BAT packing error
This error hasn't been reported in 9 months, so I'll assume it's fixed.
2024-01-17 13:06:00 +01:00
Sybren A. Stüvel
b16b44b4d6 Run go mod tidy 2024-01-13 13:08:29 +01:00
Sybren A. Stüvel
8b9a2a15cf CHANGELOG: marked 3.4 as released today 2024-01-12 11:15:55 +01:00
Sybren A. Stüvel
e1306010d7 Bumped version to 3.4 2024-01-12 11:14:50 +01:00
Sybren A. Stüvel
b39f116b0e Manager: after deleting a job, perform a database consistency check
Deleting jobs from the database can still sometimes cause consistency
errors, as if foreign key constraints aren't enabled. This check is there
to try and get a grip on things.
2024-01-11 20:03:53 +01:00
Sybren A. Stüvel
7c08ec8654 Upgrade dependencies
Go dependency upgrades, most importantly:

- github.com/glebarez/go-sqlite v1.21.1 -> v1.22.0
- github.com/glebarez/sqlite v1.8.0 -> v1.10.0
- gorm.io/gorm v1.25.2 -> v1.25.5
- modernc.org/sqlite v1.26.0 -> v1.28.0
2024-01-11 19:53:07 +01:00
Sybren A. Stüvel
c80f898095 Makefile: Run make vet as part of the release-package rule 2024-01-11 19:06:49 +01:00
Sybren A. Stüvel
aac2ec7bf6 Manager: when requesting job deletion, also log its low-level database ID
When an API request comes in to delete a job, not only log the job's UUID,
but also include its database ID. This can help in figuring out database
issues, as when the job is deleted, it's unknown what UUID it had. Database
relations use the ID, and not the UUID.
2024-01-11 17:17:56 +01:00
Sybren A. Stüvel
6777e89589 Manager: refuse to delete job when foreign keys are disabled
Just as a safety measure, before deleting a job, check that foreign key
constraints are enabled. These are optional in SQLite, and the deletion
function assumes that they are on.
2024-01-11 17:17:56 +01:00
Sybren A. Stüvel
3e46322d14 Manager: reduce log level when last-rendered image was accepted
Reduce the log level when a last-rendered image was accepted from a Worker.
2024-01-11 17:17:56 +01:00
Sybren A. Stüvel
b110f5fb51 Bumped version down to 3.4-beta1
There are a few more things I want to do before releasing 3,4.
2024-01-11 17:17:56 +01:00
Sybren A. Stüvel
bfe2b9f53a Website: mark 3.4 as latest release 2024-01-11 16:53:38 +01:00
Sybren A. Stüvel
0c08a5ad5d Bumped version to 3.4 2024-01-11 16:51:21 +01:00
Sybren A. Stüvel
64332284d4 Mark 3.4 as released today 2024-01-11 16:50:56 +01:00
Sybren A. Stüvel
1095caeddd OAPI: regenerate code
This creates the Python and JavaScript files for the job mass-deletion
operation.
2024-01-11 16:47:25 +01:00