16 Commits

Author SHA1 Message Date
Sybren A. Stüvel
63c60a5b15 Two-way variable replacement: change path separators to target platform
Two-way variable replacement now also changes the path separators. Since
the two-way replacement is made for paths, it makes sense to also clean up
the path for the target platform.
2022-08-25 12:19:30 +02:00
Sybren A. Stüvel
0e6d61dd84 Remove the {ffmpeg} variable
Remove the `{ffmpeg}` variable from the default configuration, and its use
from the job compiler scripts. Now that the Worker can find its bundled
FFmpeg, it's no longer needed to configure its location on the Manager.
2022-07-22 16:37:14 +02:00
Sybren A. Stüvel
11a352968a Fix T99434: Two-way Variables
Two-way variable implementation in the job submission end-point. Where
Flamenco v2 did the variable replacement in the add-on, this has now
been moved to the Manager itself. The only thing the add-on needs to
pass is its platform, so that the right values can be recognised.

This also implements two-way replacement when tasks are handed out, such
that the `{jobs}` value gets replaced to a value suitable for the
Worker's platform as well.
2022-07-22 11:58:35 +02:00
Sybren A. Stüvel
3c290b1f6d Manager: ensure the {jobs} implicit variable uses forward slashes
Since the variable expansion is unaware of path semantics, using forward
slashes is the safest way to go about things in a platform-indepdent way.
2022-07-13 12:45:55 +02:00
Sybren A. Stüvel
d4429d593c Unify task log storage & manager-local storage
The task logs storage system is refactored to use the `local_storage`
package. Configuration options have also changed:

- `task_logs_path` is renamed to `local_manager_storage_path`, to
  emphasise that only the Manager deals with those files, with default
  value `./flamenco-manager-storage`.
- `storage_path` is renamed to `shared_storage_path`, to emphasise this
  is the storage shared between Manager and Workers, with default value
  `./flamenco-shared-storage`.

Task logs are still stored in
`${local_manager_storage_path}/job-{jobUUID[0:4]}/{jobUUID}/task-{taskUUID}.txt`

Manifest task: T99409
2022-07-05 17:58:58 +02:00
Sybren A. Stüvel
fd31a85bcd Manager: add blocking of workers when they fail certain tasks too much
When a worker fails too many tasks, of the same task type, on the same job,
it'll get blocked from doing those.
2022-06-17 15:49:16 +02:00
Sybren A. Stüvel
e70dad2177 Manager: unify Flamenco and Shaman storage paths
Flamenco Manager now has a "storage path" config option, which will be
used by Shaman if enabled. Now the `{jobs}` implicit variable will always
exist, its value depending on whether Shaman is enabled or not.
2022-04-01 13:43:51 +02:00
Sybren A. Stüvel
292d07d13c Manager: erase configured variables when overlapping with implicit vars
When a variable is found in the config file, with the same name as an
implicit variable, it will be removed from the configuration (i.e. implicit
ones always win). A warning is logged when this happens.
2022-03-25 16:40:58 +01:00
Sybren A. Stüvel
1c752fd8e3 Add implicit "{jobs}" variable when Shaman is enabled
This allows the Blender add-on to submit jobs at path
`{jobs}/path/file.blend`. Due to the nature of the system, the add-on
doesn't know (and shouldn't know) where exactly the Manager has its
Shaman storage.
2022-03-25 16:20:42 +01:00
Sybren A. Stüvel
98a5d48611 Manager: make the 'platform' of a variable its own type
This prevents too many `string` types; those are now just used for the
variable name & value, whereas the platform is a `VariablePlatform` type.
2022-03-25 16:19:59 +01:00
Sybren A. Stüvel
bedf7869f8 Manager: replace "direction=twoway" with "is_twoway=true" in config
A boolean provides less context to the setting, so it's not as easy to
understand. However, in this case the simple case will have `is_twoway=false` and be ommitted from the configuration file. This makes the simple case even simpler.
2022-03-17 11:33:41 +01:00
Sybren A. Stüvel
9f5e4cc0cc License: license all code under "GPL-3.0-or-later"
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
2022-03-07 15:26:46 +01:00
Sybren A. Stüvel
7d67a1dc7a Manager: default paths for executables are now relative
This assumes that `blender` and `ffmpeg` are available on the commandline.
2022-02-22 13:55:03 +01:00
Sybren A. Stüvel
6b5b3eacdd Fix some warnings 2022-02-22 10:36:17 +01:00
Sybren A. Stüvel
ad6cbcbae2 Remove SSDP GUID from settings 2022-02-22 10:33:28 +01:00
Sybren A. Stüvel
12481a47e7 Start of configuration/settings framework, including variable replacement 2022-02-21 18:09:45 +01:00