149 Commits

Author SHA1 Message Date
Sybren A. Stüvel
132ce8f2ec Merge 'shutdown' and 'offline' states
Move the 'shutdown' state code to the 'offline' state, to match the
removal of the 'shutdown' state from the OpenAPI definition.
2022-06-02 16:35:07 +02:00
Sybren A. Stüvel
fdb0b82664 OAPI: regenerate code 2022-05-31 17:19:06 +02:00
Sybren A. Stüvel
f2f6726ec0 OAPI: regenerate code 2022-05-31 11:21:41 +02:00
Sybren A. Stüvel
eab4fa2ca7 OAPI: regenerate code 2022-05-30 18:52:00 +02:00
Sybren A. Stüvel
bb7ac8319f OAPI: regenerate code 2022-05-20 16:30:42 +02:00
Sybren A. Stüvel
c4cda79ec0 Worker: chunk logs at 10kB instead of 1kB
Send logs in bigger chunks, otherwise a Blender render can cause too many
individual requests.
2022-05-20 13:36:16 +02:00
Sybren A. Stüvel
d9a955beee Worker: only call may-I-keep-running endpoint every 10 seconds
For debugging it was nice to have this called every second, but for
production use that's a bit too frequent.
2022-05-20 12:57:27 +02:00
Sybren A. Stüvel
dbd32e56cd Worker: fix FFmpeg issues on Windows
Fix the FFmpeg unit test on Windows, by:
- Having actual input files (otherwise the input-glob-creation-function
  errors out), and
- ensuring the cleanup function is always run, and
- testing for the right CLI arguments.
2022-05-19 16:42:40 +02:00
Sybren A. Stüvel
2c79a10650 Worker: don't log error if may-i-keep-running is shut down
Don't log an error if a worker shutdown (indicated by the context closing)
interrupts a may-i-keep-running call. Instead, log at debug level and just
return "yes, keep running"; we want the Worker to stop the task because it
is shutting down, and not because the Manager told us so.
2022-05-19 15:00:03 +02:00
Sybren A. Stüvel
0b39f229a1 Implement may-I-keep-running protocol
Worker and Manager implementation of the "may-I-kee-running" protocol.

While running tasks, the Worker will ask the Manager periodically
whether it's still allowed to keep running that task. This allows the
Manager to abort commands on Workers when:

- the Worker should go to another state (typically 'asleep' or
  'shutdown'),
- the task changed status from 'active' to something non-runnable
  (typically 'canceled' when the job as a whole is canceled).
- the task has been assigned to a different Worker. This can happen when
  a Worker loses its connection to its Manager, resulting in a task
  timeout (not yet implemented) after which the task can be assigned to
  another Worker. If then the connectivity is restored, the first Worker
  should abort (last-assigned Worker wins).
2022-05-12 15:06:05 +02:00
Sybren A. Stüvel
fd16f7939e OAPI: regenerate code 2022-05-12 15:06:05 +02:00
Sybren A. Stüvel
bedf10e435 Worker: clarify message when sleep command is aborted
Instead of logging "sleep aborted", the message is now "sleep command
aborted", to make it clear that it's about the sleep command, and not the
"asleep" worker state.
2022-05-12 14:59:10 +02:00
Sybren A. Stüvel
9dbc952c09 Worker: move wait time into variable
No functional changes.
2022-05-12 12:44:50 +02:00
Sybren A. Stüvel
7b1b6030d3 OAPI: regenerate code 2022-05-05 16:04:45 +02:00
Sybren A. Stüvel
4da7f67105 OAPI: generate code 2022-05-03 13:03:59 +02:00
Sybren A. Stüvel
992fc38604 OAPI: add endpoint for fetching the tasks of a job
Add `fetchJobTasks` operation to the Jobs API. This returns a summary of
each of the job's tasks, suitable for display in a task list view.

The actually used fields may need tweaking once we actually have a task
list view, but at least the functionality is there.
2022-04-22 12:52:57 +02:00
Sybren A. Stüvel
e34a0ba6ea Worker: more granular locking when flushing upstream buffer
Only lock the database mutex when actual queries are performed, but not
during the entire flush loop.
2022-04-21 19:19:01 +02:00
Sybren A. Stüvel
8937a6f06f Cleanup: worker, remove debug timers
Remove accidentally committed debug timing code.
2022-04-21 19:14:09 +02:00
Sybren A. Stüvel
bcbacf6c42 Worker: fix race condition getting logger with worker status 2022-04-21 19:12:53 +02:00
Sybren A. Stüvel
1176d85496 Worker: add -flush CLI option to just flush the buffer and quit
Add `-flush` CLI option to just flush the upstream queue and then quit.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
5466f65225 OAPI: add setJobStatus operation
Add API endpoint for updating the job status.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
d099a31531 OAPI: add endpoint for getting a single job type
This will be used by the web frontend to determine which job settings
to show and which to hide.
2022-04-15 16:21:48 +02:00
Sybren A. Stüvel
c79983bafb Worker: include -r {fps} in FFmpeg command
This should force the output created by FFmpeg to match the desired FPS.
2022-04-11 12:09:28 +02:00
Sybren A. Stüvel
e9212de196 Worker: simplify "context done" checks 2022-04-09 16:57:39 +02:00
Sybren A. Stüvel
d98dbaa333 Worker: implement ffmpeg for frame-to-video conversion on Windows 2022-04-09 16:20:29 +02:00
Sybren A. Stüvel
38aedfe8cc Worker: make blender-finder find blender.exe and not blender-launcher.exe
`blender-launcher.exe` doesn't pipe Blender's stdout/stderr output, so it's
not useful for Flamenco.
2022-04-09 16:16:19 +02:00
Sybren A. Stüvel
0959179225 Worker: redirect stderr to stdout at the right moment
It should be done after stdout has been assigned a pipe, and not before.
2022-04-09 16:14:44 +02:00
Sybren A. Stüvel
4181708709 Worker: automatically find Blender on Windows via file association
On Windows, when the Manager doesn't explicitly point at a Blender to use,
use the the application associated with `.blend` files instead.
2022-04-09 15:05:34 +02:00
Sybren A. Stüvel
bf47afc32b Worker: reduce log level of "fetching tasks" message 2022-04-09 12:52:28 +02:00
Sybren A. Stüvel
cc20b850ec Worker: separate contexts of upstream buffer
The upstream buffer takes care of two things: communication with Flamenco
Manager (first context) and buffering things in a SQLite database (second
context). This commit separates those two contexts, so that shutting down
the application isn't going to prevent buffering things in the database.
2022-04-08 16:49:53 +02:00
Sybren A. Stüvel
e7fc2c6f6e Worker: add various extra error checks 2022-04-08 14:47:20 +02:00
Sybren A. Stüvel
1960b668aa Cleanup: remove unused code 2022-04-08 14:47:07 +02:00
Sybren A. Stüvel
62ea7dcf1d Worker: avoid task errors on shutdown
When shutting down, the main worker context closes. This causes the
subprocess to be killed, which in turn caused a task execution error. This
now no longer happens, as such errors are expected on shutdown and do not
indicate task failure.
2022-04-08 14:32:25 +02:00
Sybren A. Stüvel
fdab07d987 Worker: slight clarification of error message 2022-04-05 16:28:22 +02:00
Sybren A. Stüvel
781f1d936a OAPI: add jobs query endpoint 2022-04-04 18:53:19 +02:00
Sybren A. Stüvel
bff5e30421 Worker: use defaults for partial configuration
Instead of having a full "defaults OR the loaded config" (where a partial
config file would thus have the nil value for missing properties) the
missing properties now retain their default value.
2022-04-04 16:56:52 +02:00
Sybren A. Stüvel
21227c8046 More Shaman integration work
- Addon switches between filesystem-packing and Shaman-packing
  automatically, depending on whether the Manager has Shaman enabled.
- Actually using BAT for Shaman packing.

It doesn't work though, some error occurs when receiving Shaman response
from the Manager in the Addon.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
b2288e7f28 Manager: More work on Shaman support
This introduces some more conceptual changes to Shaman. The most important
one is that there is no longer a "checkout ID", but a "checkout path".
The Shaman client can request any subpath of the checkout directory,
so that it can handle things like project- or scene-specific prefixes.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
4df0543661 Manager: Implementation of Shaman OAPI interface
This hooks up the OpenAPI Shaman endpoints to a Shaman interface. This
interface doesn't have any implementation yet.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
b6a7a15582 Implement frames-to-video command
This finishes the "Simple Blender Render" job type, as it can now be run
fully by a Worker!
2022-03-17 16:18:10 +01:00
Sybren A. Stüvel
5ea3bfe301 Worker: reduce logging 2022-03-17 15:39:59 +01:00
Sybren A. Stüvel
c989bce51e Implement move-directory command, and use it in Simple Blender Render 2022-03-17 15:39:52 +01:00
Sybren A. Stüvel
1bcf4139bf Worker: be a bit faster in requesting a new task 2022-03-17 13:07:48 +01:00
Sybren A. Stüvel
afc6b8e1c5 Worker: add -find-manager CLI arg
With `-find-manager` the Worker will use UPnP/SSDP autodiscovery, report
which Manager it found, and quit.
2022-03-11 10:52:03 +01:00
Sybren A. Stüvel
842255a065 UPnP/SSDP: actually serve a description.xml
The `description.xml` file is part of the UPnP Service Discovery protocol,
see https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-rxad/9225d145-6b2c-40d5-8ee8-7d837379fc25
2022-03-08 17:25:49 +01:00
Sybren A. Stüvel
353da58ee9 UPnP/SSDP: reduce log level for unreachable URLs 2022-03-08 16:46:14 +01:00
Sybren A. Stüvel
a61e3ca6f7 Worker tests: use in-memory SQLite database 2022-03-08 15:03:03 +01:00
Sybren A. Stüvel
fabb79e583 Worker: move autodiscovery code into the 'worker' package
Move the UPnP/SSDP Manager autodiscovery code into from `main.go` into the
`worker` package. This also means changing the error handling a bit, as
only the `main.go` file is allowed to do `log.Fatal()`.
2022-03-08 14:41:30 +01:00
Sybren A. Stüvel
d0b677ffe3 Add API endpoint /api/version to get the Flamenco Manager version
This will be used in the auto-discovery, to check that a discovered URL can
actually be reached & points to a Flamenco Manager.
2022-03-08 13:56:43 +01:00
Sybren A. Stüvel
42407865eb Worker: change how configuration & credentials are loaded/saved
The Worker config/credential management was a bit of a mess. It's now
better structured, and also allows runtime overrides of the Manager URL,
without writing that override to the config file.
2022-03-08 13:51:03 +01:00