419 Commits

Author SHA1 Message Date
Sybren A. Stüvel
eb89984db8 Simple Blender Render: remove blender_cmd setting
Remove the `blender_cmd` setting, and just hard-code it to `{blender}`.
The Blender add-on was already passing this string, and it's very unlikely
that people are already writing custom add-ons to pass something different.
It provided flexibility that was untested, so it's better to simplify
things.
2022-08-31 16:24:34 +02:00
Sybren A. Stüvel
25dd7b214b Manager: remove superfluous "error compiling job: " prefix from message
The wrapped error already mentioned it was about job compilation.
2022-08-31 16:23:10 +02:00
Sybren A. Stüvel
31cf0a4ecc Implement getSharedStorage operation & use it in the add-on
Implement the `getSharedStorage` operation in the Manager, and use it in
the add-on to get the shared storage location in a way that makes sense
for the platform of the user.

Manifest task: T100196
2022-08-31 11:44:37 +02:00
Sybren A. Stüvel
31769bcdf2 Manager: always set config.currentGOOS
This variable is used in tests to mock the current OS, but wasn't set
during normal operation of the Manager. This caused issues with the
two-way variable system.
2022-08-31 11:43:28 +02:00
Sybren A. Stüvel
2eae682b9a Manager: actually return the short version in the GetVersion operation 2022-08-31 08:58:59 +02:00
Sybren A. Stüvel
ab14c97b2e Manager: fix tests on Windows
Fix some tests that were failing because some parts of Flamenco now use
native path separators instead of always-forward ones.
2022-08-30 15:44:14 +02:00
Sybren A. Stüvel
e5a20425c4 Separate variables for Blender executable and its arguments.
Split "executable" from "its arguments" in blender & ffmpeg commands.

Use `{blenderArgs}` variable to hold the default Blender arguments,
instead of having both the executable and its arguments in `{blender}`.

The reason for this is to support backslashes in the Blender executable
path. These were interpreted as escape characters by the shell lexer.
The shell lexer based splitting is now only performed on the default
arguments, with the result that `C:\Program Files\Blender
Foundation\3.3\blender.exe` is now a valid value for `{blender}`.

This does mean that this is backward incompatible change, and that it
requires setting up Flamenco Manager again, and that older jobs will not
be able to be rerun.

It is recommended to remove `flamenco-manager.yaml`, restart Flamenco
Manager, and reconfigure via the setup assistant.
2022-08-30 14:58:16 +02:00
Sybren A. Stüvel
c504e68d8e Manager: store the jobs implicit variable in platform-native notation
Don't change backslashes to forward slashes on Windows. Trying to use
forward slashes everywhere was a mistake, and this is one of the steps to
make it right.
2022-08-29 17:51:20 +02:00
Sybren A. Stüvel
20395e0e26 Manager: always start the variable lookup table with a fresh map
If the loaded config doesn't define the default variables, the latter
should not be found in the lookup table any more; this is now fixed.
2022-08-29 17:44:47 +02:00
Sybren A. Stüvel
4a201d47b4 Cleanup: add unit test for parsing backslashes in variable values
Backslashes can be included in two ways, as-is (which works fine) and
between double quotes (in which case they need escaping). This test checks
for both.
2022-08-29 17:28:40 +02:00
Sybren A. Stüvel
0c91fe93d0 Manager: only do pathsep localisation on two-way variables
By accident the Manager was performing slash localisation on all
command parameters, causing some math expressions for FFmpeg to fail.
2022-08-25 15:02:56 +02:00
Sybren A. Stüvel
6b4b205c1c Manager: allow backslashes in variables
Windows machines should be able to simply use backslashes.
2022-08-25 13:59:02 +02:00
Sybren A. Stüvel
22aa041ec1 Allow relative render output root paths
Add a new `abspath(path)` function to the add-on, for use in job type
settings. With this, the "simple blender render" job can support relative
paths for the "render output root" setting, and still have an absolute
final "render output path".
2022-08-25 13:14:48 +02:00
Sybren A. Stüvel
63c60a5b15 Two-way variable replacement: change path separators to target platform
Two-way variable replacement now also changes the path separators. Since
the two-way replacement is made for paths, it makes sense to also clean up
the path for the target platform.
2022-08-25 12:19:30 +02:00
Sybren A. Stüvel
2a345a3d2c API for deleting workers
Workers can now be soft-deleted. Tasks assigned to the worker will remain
associated with that Worker. Active tasks will be re-queued so other
workers can pick them up.
2022-08-11 16:59:53 -07:00
Sybren A. Stüvel
cbafe0ff34 Manager: when finding Blender, be less dramatic when it can't be found
It's fine when Blender is not available on `$PATH`, so only log that at
debug level.
2022-08-02 13:36:25 +02:00
Sybren A. Stüvel
cbc6bfaf02 Manager: also recognise exec.ErrNotFound as a "blender not found" error 2022-08-02 13:36:25 +02:00
Sybren A. Stüvel
11e5363d24 Manager: reject removal of empty list of blocklist entries
A request to remove an empty list of blocklist entries now results in a
400 Bad Request.
2022-08-01 18:55:33 +02:00
Sybren A. Stüvel
3b978ceda0 Cleanup: manager, name variable correctly
It was an old name from copy-pasted code, now it reflects the actual code.

No functional changes.
2022-08-01 18:55:08 +02:00
Sybren A. Stüvel
1469345f3a Manager: sort blocklist by worker name 2022-08-01 18:54:28 +02:00
Sybren A. Stüvel
f3aab8611c Manager: include worker name when returning blocklist 2022-08-01 18:03:17 +02:00
Sybren A. Stüvel
fef3de28e1 Fix unit test
Fix unit test broken in rF449c83b9.

No functional changes.
2022-08-01 16:02:08 +02:00
Sybren A. Stüvel
449c83b94a Manager: broadcast worker update after assigning task
The Manager now broadcasts a worker update to SocketIO clients when a
worker gets a new task assigned. This ensures the "current task" shown in
the worker details view is up to date.
2022-08-01 14:29:08 +02:00
Sybren A. Stüvel
a6c935a634 Fix T99421: Introducing an etag for job types
The etag prevents job submissions with old settings, when the job
compiler script has been edited. The etag is the SHA1 hash of the
`JOB_TYPE` dictionary (as defined by the JavaScript file). The hash is
computed in a way that's independent of the exact formatting in the
JavaScript file. Also the actual JS code itself is irrelevant, just the
`JOB_TYPE` dictionary is used.
2022-07-29 21:13:37 +02:00
Sybren A. Stüvel
48ca73f550 Refactor, manager: rename compilerForJobType to compilerVMForJobType
The function returns a `*VM`, which contains a compiler, and allows you
to run a compiler, but is not a compiler itself.
2022-07-29 14:26:54 +02:00
Sybren A. Stüvel
370f935f65 Simple-blender-render job: use absolute path for render_output_path
Blender cannot be told to only allow absolute path for an RNA property
(of type `string`, subtype `dir_path`), so as a workaround the final
`render_output_path` is now using `bpy.path.abspath()` to make the path
absolute.

This has as advantage that the render output path can be defined by artists
as a blendfile-relative path, and that it'll be resolved when submitting
the blend file.
2022-07-29 11:03:14 +02:00
Sybren A. Stüvel
be1ddaa4eb Manager test: reduce timeout to practical value
The timeout was increased to aid debugging, but shouldn't have been
committed.
2022-07-29 09:59:54 +02:00
Sybren A. Stüvel
8c8855554e Manager: remove --factory-startup from default Blender arguments
Remove `--factory-startup` from the default Blender arguments. This makes
it simpler to configure each Worker to use its own GPU, without having to
inject Python code into the arguments.

Users can always add this when they need, but I think it's friendlier to
have Blender behave the same when they manually run it and when used by
Flamenco Worker.
2022-07-29 09:54:29 +02:00
Sybren A. Stüvel
d4dfa2d071 Add release cycle to versioning of Flamenco
Include `RELEASE_CYCLE` in the Makefile. This is mentioned at startup of
Manager and Worker, and reflects in the software version they report.

If `RELEASE_CYCLE == "release"`, Manager and Worker report their version
as `ApplicationVersion`. If it's any other string, the Git hash will get
appended.
2022-07-28 15:10:27 +02:00
Sybren A. Stüvel
c42665322b Cleanup: add a comment
Just a comment that explains why an error is ignored.

No functional changes.
2022-07-28 14:28:02 +02:00
Sybren A. Stüvel
b26374d480 Manager: when worker goes to sleep, log in task log which worker
When a worker's tasks get requeued because it goes to sleep, the task log
will now mention the worker identification (name + UUID). This aids in
figuring out what happened to tasks.
2022-07-28 14:27:44 +02:00
Sybren A. Stüvel
859a2e6eda Manager: better logging when trying to find Blender 2022-07-26 17:25:50 +02:00
Sybren A. Stüvel
f71bfdfafe Manager: fix unit test
Fix the unit test I broke in rF736ca103c3d7f37557ed541ca70117bc95bef932
2022-07-26 17:25:50 +02:00
Sybren A. Stüvel
736ca103c3 Manager: show current/last task in worker details
The Task details component already linked to the Worker it was assigned
to last, and now the Worker links back to the task.

There's only one task shown in the Worker details. If the Worker is
actively working on a task, that one's shown. Otherwise it's the
last-updated task that was assigned to the worker.
2022-07-26 10:36:02 +02:00
Francesco Siddi
9948fdab71 Rename First Time Wizard to Setup Assistant
This commit does not introduce functional changes, besides renaming
every mention of 'wizard' with 'setup assistant'. In order to run the
manager setup assistant use:

./flamenco-manager -setup-assistant

The change was introduced to favor more neutral and descriptive working
for this functionality. Thanks to Sybren for helping to get this done!
2022-07-25 17:17:04 +02:00
Sybren A. Stüvel
ab8ecc24cc Cleanup: Add missing license specifiers
Add license specifiers to Go files that were missing them:

```
// SPDX-License-Identifier: GPL-3.0-or-later
```

No functional changes.
2022-07-25 16:08:07 +02:00
Sybren A. Stüvel
0e6d61dd84 Remove the {ffmpeg} variable
Remove the `{ffmpeg}` variable from the default configuration, and its use
from the job compiler scripts. Now that the Worker can find its bundled
FFmpeg, it's no longer needed to configure its location on the Manager.
2022-07-22 16:37:14 +02:00
Pablo Vazquez
53598c3ee0 Manager: Rephrase wording on report for successfully writing to Shared Storage
* Replace "OK!" with "successfully"
  Remove exclamation mark since there is no need to call for attention.
  Use "successfully" as it is more descriptive in this case than OK,
  which can have other meanings.
2022-07-22 14:57:12 +02:00
Francesco Siddi
08f52993ad Setup Screen: Overall UI/UX tweaks
- Added initial description and illustration
- Swap "Check" button for fields with a debounced @input event
- Turn Blender's list into a radio selector
- Tweak wording when paths are not found
- Add microtip library for tooltips
- Make navigation steps clickable, according to the state
2022-07-22 14:57:11 +02:00
Sybren A. Stüvel
11a352968a Fix T99434: Two-way Variables
Two-way variable implementation in the job submission end-point. Where
Flamenco v2 did the variable replacement in the add-on, this has now
been moved to the Manager itself. The only thing the add-on needs to
pass is its platform, so that the right values can be recognised.

This also implements two-way replacement when tasks are handed out, such
that the `{jobs}` value gets replaced to a value suitable for the
Worker's platform as well.
2022-07-22 11:58:35 +02:00
Sybren A. Stüvel
af0389efc6 Cleanup: correct function name in docstring 2022-07-21 16:29:23 +02:00
Sybren A. Stüvel
894058bc69 Cleanup: variable replacement, avoid hard-coded "workers" string
Use `config.VariableAudienceWorkers` instead.

No functional changes.
2022-07-21 16:29:05 +02:00
Sybren A. Stüvel
27602174ae Variable replacement: fix issue replacing vars in nested lists
An array-of-strings in Go can become an array-of-`interface{}` when
converted to JSON and back again. Such cases are now handled properly.
2022-07-21 16:28:38 +02:00
Sybren A. Stüvel
48f081e03e Sleep Scheduler: don't overwrite error status from Worker
The Sleep Scheduler shouldn't push a Worker out of `error` status, as that
could hide problematic situations.
2022-07-21 12:49:32 +02:00
Sybren A. Stüvel
de80a09223 Manager: include job UUID in "last-rendered image received" log entries
This makes it possible to collect all "last-rendered image received"
entries for a single job.
2022-07-19 18:40:22 +02:00
Sybren A. Stüvel
d929885b06 Manager: only log task status change if there is an actual change
Don't log "changes" from, say, `active` -> `active`.
2022-07-19 17:47:43 +02:00
Sybren A. Stüvel
ac3236786b Manager: add entry to task log whenever task changes status
Add a line to the task log whenever task changes status. This only applies
to directly-changed tasks, and not to mass-updates (like all tasks going
from 'completed' to 'queued' on a job requeue).
2022-07-19 17:23:13 +02:00
Sybren A. Stüvel
696b97c553 Re-queue tasks of worker after changing to non-'awake' state
When a Worker changes state from `awake` to something else, it cannot
run tasks any more. This now triggers a requeue of its active task
(should be one at most, if things are sane) so that another worker can pick
it up.
2022-07-19 15:38:36 +02:00
Sybren A. Stüvel
0a5f87bc5a Sleep Scheduler: perform first check at startup
Instead of waiting for a minute, run the first sleep scheduler iteration
at startup.
2022-07-18 19:30:38 +02:00
Sybren A. Stüvel
83467e4c60 Sleep schedule: store 'next check' timestamp in UTC
SQLite doesn't parse the timezone info, so timestamps should always be in
UTC.
2022-07-18 19:30:17 +02:00