50 Commits

Author SHA1 Message Date
Sybren A. Stüvel
2a6cbcf030 Addon: Add worker cluster support
Worker clusters can be fetched from the Manager, and submitted jobs can
be assigned to those clusters.
2023-04-04 12:19:44 +02:00
Sybren A. Stüvel
f413a40f4e Store Shaman checkout ID when submitting a job
If Shaman is used to submit the job files, store the job's checkout ID
(i.e. the path relative to the checkout root) in the database. This will
make it possible in the future to remove the Shaman checkout along with
the job itself.
2023-01-04 01:18:21 +01:00
Sybren A. Stüvel
7f268f2177 Add-on: abort submission when job preparation fails
Job preparation (that is, constructing the data to be sent to the Flamenco
API) shouldn't fail, but in case it does, it should stop the rest of the
submission process.
2023-01-04 01:18:21 +01:00
Sybren A. Stüvel
4ef5373756 Add-on: add support for the use_all_linked_data_direct option
Add support for Blender's `use_all_linked_data_direct` experimental
option. This is a workaround for an issue with BAT, for which a quick
solution is required by the Blender Studio (production crunch).

Flamenco writes the `.flamenco.blend` file with
`preferences.experimental.use_all_linked_data_direct` set to `True`, so
that BAT has an easier time finding linked assets.
2022-12-02 14:31:23 +01:00
Yann Doersam
1cef0d48df Addon: Show error if the file path is not known before submitting
If file has never been saved, Flamenco cannot create a temp copy to
submit. This caused an exception, which is now avoided by showing a
clear error message instead.

Maniphest Tasks: T102192

Differential Revision: https://developer.blender.org/D16358
2022-11-08 11:48:10 +01:00
Sybren A. Stüvel
074c60df9d Cleanup: add-on, reformat with black
No functional changes.
2022-10-20 13:14:51 +02:00
Sybren A. Stüvel
32a3e48e24 Add-on: use job check endpoint before sending files to the farm 2022-10-20 13:14:01 +02:00
Sybren A. Stüvel
6185a428a8 Add-on: much faster processing of BAT progress messages
BAT-packing is a multi-threaded operation, so messages about its progress
are queued up in a thread-safe manner. The modal operator was only handling
4 messages per second, causing major slowdowns when there are lots of
small files to be processed. This is now done much faster, and thus the
UI shouldn't lag behind of the actual submissions progress.
2022-09-12 15:33:37 +02:00
Sybren A. Stüvel
fa69cc102b Add-on: clearer error message when job compiler refuses the job
Blender now shows the actual returned error from the Manager when job
submission fails, rather than a generic "failed" message.
2022-08-31 16:22:34 +02:00
Sybren A. Stüvel
59f41d0546 Add-on: show warning when versions are not matching
Before submitting a job, the add-on now checks the version of the Manager.
If this is not the same version of the add-on, a warning is shown and a
"Force Submit" button appears. This makes it both explicit that something
is iffy and still allows for pushing forward.

This is important when upgrading Flamenco, because I'm sure many people
will forget to actually redownload and reinstall the add-on.
2022-08-31 09:27:50 +02:00
Sybren A. Stüvel
0b176eda96 Add-on: include error cause in "Error performing BAT pack" message
The error can get cryptic, but without opening the terminal people
otherwise would have no idea what's going on.
2022-08-31 08:09:11 +02:00
Sybren A. Stüvel
a7510f4042 Cleanup: addon, split ping_manager() into two functions
Split the `comms.ping_manager()` function into two: one that returns the
version & config of the Manager, and the other that reports on it.

This will make it possible to do the former without the latter in certain
other situations where we want to refresh the manager info in the
background.
2022-08-31 08:08:06 +02:00
Sybren A. Stüvel
f82ebea11d Add-on: handle submission errors more gracefully 2022-08-25 13:28:45 +02:00
Sybren A. Stüvel
a6c935a634 Fix T99421: Introducing an etag for job types
The etag prevents job submissions with old settings, when the job
compiler script has been edited. The etag is the SHA1 hash of the
`JOB_TYPE` dictionary (as defined by the JavaScript file). The hash is
computed in a way that's independent of the exact formatting in the
JavaScript file. Also the actual JS code itself is irrelevant, just the
`JOB_TYPE` dictionary is used.
2022-07-29 21:13:37 +02:00
Sybren A. Stüvel
866513e06a Add-on: check connection with Flamenco on URL change
Whenever the URL to Flamenco Manager is updated, check it and report.

This required moving the 'ping manager' code into a separate function, as
calling an operator from an RNA update callback is not a good idea.
2022-07-29 14:25:47 +02:00
Sybren A. Stüvel
764973f116 Cleanup: add-on, remove unused parameter
No functional changes.
2022-07-29 10:25:23 +02:00
Sybren A. Stüvel
ca8a909e41 Add-on: refactor FLAMENCO_OT_submit_job.poll()
Refactor the `FLAMENCO_OT_submit_job.poll()` function to actually use the
`job_types.active_job_type()` function instead of reimplementing similar
logic.

No functional changes.
2022-07-29 10:10:44 +02:00
Sybren A. Stüvel
fb5501028d Add-on: use bat.bpathlib.make_absolute() for making paths absolute
This should resolve the issue where on Windows file paths to a network
share would be converted to UNC notation (and thus loose their drive
letter).
2022-07-28 17:49:01 +02:00
Sybren A. Stüvel
344a62f37b Add-on: Skip BAT-packing when file already in shared storage
When the to-be-submitted blend file is already stored in the shared
storage, and Shaman is disabled, the add-on now skips BAT-packing it.
Instead, the file is copied to `filename.flamenco.blend` and the path is
submitted as-is.
2022-07-21 14:18:48 +02:00
Sybren A. Stüvel
19d3ac1a0a Fix T99739: Add-on: Prevent job submission when no job type has been chosen
Prevent the `FLAMENCO_OT_submit_job` operator from running when there has
been no job type selection.
2022-07-21 13:03:15 +02:00
Sybren A. Stüvel
5b1b0b30a9 Add-on: remove .flamenco.blend after job submission 2022-07-12 17:46:08 +02:00
Sybren A. Stüvel
ac5bb5e378 Remove assumption {jobs} only exists when Shaman is enabled
Manager always creates an implicit variable `{jobs}`. This used to be
Shaman-dependent, but now it's always there (has been for a while). This
is now reflected in an add-on comment, and in an extra unit test.
2022-07-05 18:19:49 +02:00
Sybren A. Stüvel
e6bf60cb77 Addon: fix issue where Shaman checkout directory wasn't used properly
Shaman prevents duplicate checkout directories, and communicates the actual
location back to the client. This wasn't used by the add-on though, so
the Flamenco job definition would point to the wrong files.
2022-06-28 16:28:28 +02:00
Sybren A. Stüvel
dcc0acd8b0 Cleanup: remove unused import 2022-04-01 14:28:51 +02:00
Sybren A. Stüvel
12e6211fc9 Addon: get storage directory from Manager
Get the job storage location from the Manager, don't allow editing it, and
don't allow per-scene overrides.
2022-04-01 14:28:43 +02:00
Sybren A. Stüvel
1c752fd8e3 Add implicit "{jobs}" variable when Shaman is enabled
This allows the Blender add-on to submit jobs at path
`{jobs}/path/file.blend`. Due to the nature of the system, the add-on
doesn't know (and shouldn't know) where exactly the Manager has its
Shaman storage.
2022-03-25 16:20:42 +01:00
Sybren A. Stüvel
724938c7ae Shaman: allow Manager to determine the final checkout path
The checkout request now responds with the final checkout path. This makes
it possible for the Manager to ensure the checkout is unique.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
0e682282f0 Addon: fix mypy errors 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
c69e0909ae Shaman submission is working! 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
21227c8046 More Shaman integration work
- Addon switches between filesystem-packing and Shaman-packing
  automatically, depending on whether the Manager has Shaman enabled.
- Actually using BAT for Shaman packing.

It doesn't work though, some error occurs when receiving Shaman response
from the Manager in the Addon.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
f9b7510c42 Streamlining Shaman protocol further, plus rough implementation for addon 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
59f58d92b8 Addon: add file browser buttons to file/directory path settings/properties 2022-03-15 18:45:28 +01:00
Sybren A. Stüvel
6764ee8259 Addon: actually allow setting the job storage path 2022-03-15 18:37:48 +01:00
Sybren A. Stüvel
f593ef4a3e Addon: nicer error message when the Manager cannot be reached 2022-03-15 17:49:06 +01:00
Sybren A. Stüvel
09a476e11a Compute render output path when evaluating job settings
Compute render output path when evaluating job settings, which is done
within the Flamenco add-on, instead of in the job compiler script. This
allows the UI to show the render path, rather than it only being known
after the job has been submitted.
2022-03-15 16:56:44 +01:00
Sybren A. Stüvel
e15f066dde Automatically evaluate hidden job settings
Job settings that are not visible and have an `eval` key will be
automatically evaluated when the job is submitted.
2022-03-15 10:56:58 +01:00
Sybren A. Stüvel
f1a13ad521 Addon: allow undoing setting evaluation 2022-03-14 18:45:29 +01:00
Sybren A. Stüvel
a6ac331612 Allow Python evaluation for computing job settings in the addon 2022-03-14 18:44:39 +01:00
Sybren A. Stüvel
4196460c29 Automatically include input blend file path in job settings 2022-03-14 18:18:10 +01:00
Sybren A. Stüvel
9fec9c479f Cleanup: Addon, remove unused imports 2022-03-14 18:17:32 +01:00
Sybren A. Stüvel
b5053734dd Addon: submit job to Manager after packing the files 2022-03-14 17:00:12 +01:00
Sybren A. Stüvel
33b5faff2b Addon: fix MyPy errors 2022-03-14 17:00:12 +01:00
Sybren A. Stüvel
f6e636f8de Addon: cleanup, prefix TYPE_CHECKING imports with underscores
Prefix anything that's imported in an `if TYPE_CHECKING` block with
underscores, to indicate they're not the real types.
2022-03-14 16:35:21 +01:00
Sybren A. Stüvel
b993f38889 Addon: store available job types on scene
Storing the available job types on the Scene makes it possible to restore
the Flamenco job submission UI without communication with the Manager.
2022-03-14 15:36:44 +01:00
Sybren A. Stüvel
a803edcce4 Addon: bundle BAT and start of interfacing with it
The add-on can now create BAT packs, but still only at a hard-coded path.
2022-03-11 17:30:27 +01:00
Sybren A. Stüvel
d18f5d25c5 Addon: store current job type on the Scene, not the WindowManager
The job type (+ its configuration) is something scene/file dependent, and
should thus be stored there.
2022-03-11 17:30:27 +01:00
Sybren A. Stüvel
850678f495 Addon: allow configuring the Manager URL
Previously it was hard-coded to `http://localhost:8080/`, now that's just
the default value.
2022-03-10 17:57:26 +01:00
Sybren A. Stüvel
9f5e4cc0cc License: license all code under "GPL-3.0-or-later"
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
2022-03-07 15:26:46 +01:00
Sybren A. Stüvel
60d1710020 Addon: job type form seems to be working 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
616784df0a Addon: start of framework for API communication 2022-03-01 17:28:24 +01:00