Two-way variable replacement now also changes the path separators. Since
the two-way replacement is made for paths, it makes sense to also clean up
the path for the target platform.
Remove `--factory-startup` from the default Blender arguments. This makes
it simpler to configure each Worker to use its own GPU, without having to
inject Python code into the arguments.
Users can always add this when they need, but I think it's friendlier to
have Blender behave the same when they manually run it and when used by
Flamenco Worker.
This commit does not introduce functional changes, besides renaming
every mention of 'wizard' with 'setup assistant'. In order to run the
manager setup assistant use:
./flamenco-manager -setup-assistant
The change was introduced to favor more neutral and descriptive working
for this functionality. Thanks to Sybren for helping to get this done!
Remove the `{ffmpeg}` variable from the default configuration, and its use
from the job compiler scripts. Now that the Worker can find its bundled
FFmpeg, it's no longer needed to configure its location on the Manager.
Two-way variable implementation in the job submission end-point. Where
Flamenco v2 did the variable replacement in the add-on, this has now
been moved to the Manager itself. The only thing the add-on needs to
pass is its platform, so that the right values can be recognised.
This also implements two-way replacement when tasks are handed out, such
that the `{jobs}` value gets replaced to a value suitable for the
Worker's platform as well.
Every time the web interface starts, it queries the config to see whether
it should be in first-time-wizard mode or not. This caused unnecessary
info-level logging.
In the future it would be better to load the config file just once,
instead.
Trigger the first-time wizard on first-time runs of Flamenco, by defaulting
the storage path to the empty string.
The wizard can always be triggered with the `-wizard` CLI argument. This is
just for detection of first-time / unconfigured runs.
This just updates the config and saves it to `flamenco-manager.yaml`.
Saving the configuration doesn't restart the Manager yet, that's for
another commit.
This adds a `-wizard` CLI option to the Manager, which opens a webbrowser
and shows the First-Time Wizard to aid in configuration of Flamenco.
This is work in progress. The wizard is just one page, and doesn't save
anything yet to the configuration.
The task logs storage system is refactored to use the `local_storage`
package. Configuration options have also changed:
- `task_logs_path` is renamed to `local_manager_storage_path`, to
emphasise that only the Manager deals with those files, with default
value `./flamenco-manager-storage`.
- `storage_path` is renamed to `shared_storage_path`, to emphasise this
is the storage shared between Manager and Workers, with default value
`./flamenco-shared-storage`.
Task logs are still stored in
`${local_manager_storage_path}/job-{jobUUID[0:4]}/{jobUUID}/task-{taskUUID}.txt`
Manifest task: T99409
Various config sections were commented out, because they were brought in
from Flamenco 2 but weren't implemented yet. These have now been removed,
as the basic functionality is there, and new functionality will likely
be different from Flamenco 2 anyway.
`os.IsNotExist()` is from before `errors.Is()` existed. The latter is the
recommended approach, as it also recognised wrapped errors.
No functional changes, except for recognising more cases of "does not
exist" errors as such.
When a Worker indicates a task failed, mark it as `soft-failed` until
enough workers have tried & failed at the same task.
This is the first step in a blocklisting system, where tasks of an
often-failing worker will be requeued to be retried by others.
NOTE: currently the failure list of a task is NOT reset whenever it is
requeued! This will be implemented in a future commit, and is tracked in
`FEATURES.md`.
Tasks that are in state `active` but haven't been 'touched' by a Worker
for 10 minutes or longer will transition to state `failed`.
In the future, it might be better to move the decision about which state
is suitable to the Task State Machine service, so that it can be smarter
and take the history of the task into account. Going to `soft-failed`
first might be a nice touch.
This helps to get things consistent on Windows and Linux. Otherwise a path
like `/some/path` is absolute on one platform but not on the other. This is
mostly for getting the unit tests in this package to work on Windows, but
using absolute paths also helps in clarity of error logging.
Flamenco Manager now has a "storage path" config option, which will be
used by Shaman if enabled. Now the `{jobs}` implicit variable will always
exist, its value depending on whether Shaman is enabled or not.
When a variable is found in the config file, with the same name as an
implicit variable, it will be removed from the configuration (i.e. implicit
ones always win). A warning is logged when this happens.
This allows the Blender add-on to submit jobs at path
`{jobs}/path/file.blend`. Due to the nature of the system, the add-on
doesn't know (and shouldn't know) where exactly the Manager has its
Shaman storage.
Add some callback functionality, so that test code can inject/change
settings before they are processed.
Will be used in an actual test in the following commit.
Flamenco v2 allowed separate configuration of the Shaman file store and
checkout paths. This is now just one setting for "the storage". The file
store will be in `{storage}/file-store` and the checkout will happen in
`{storage}/jobs`.
This introduces some more conceptual changes to Shaman. The most important
one is that there is no longer a "checkout ID", but a "checkout path".
The Shaman client can request any subpath of the checkout directory,
so that it can handle things like project- or scene-specific prefixes.
A boolean provides less context to the setting, so it's not as easy to
understand. However, in this case the simple case will have `is_twoway=false` and be ommitted from the configuration file. This makes the simple case even simpler.
Comment out all unused-but-desired-in-the-future settings, remove some
settings that will never be used, and rename `ssdp_discovery` to
`autodiscoverable`.
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).
This is now all streamlined, and all code is licensed as "GPL v3 or later".
Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
Instead of returning an error "error doing X", just return "doing X". The
fact that it's returned as an error object says enough about that it's
an error.
This also makes it easier to chain error messages, without seeing the
word "error" in every part of the chain.
