Show jobs that have been marked for deletion with a red strike-through
line in the jobs table, and show the deletion-request timestamp in the
job details.
Implement the `deleteJob` API endpoint. Calling this endpoint will mark
the job as "deletion requested", after which it's queued for actual
deletion. This makes the API response fast, even when there is a lot of
work to do in the background.
A new background service "job deleter" keeps track of the queue of such
jobs, and performs the actual deletion. It removes:
- Shaman checkout for the job (but see below)
- Manager-local files of the job (task logs, last-rendered images)
- The job itself
The removal is done in the above order, so the job is only removed from the
database if the rest of the removal was succesful.
Shaman checkouts are only removed if the job was submitted with Flamenco
version 3.2. Earlier versions did not record enough information to reliably
do this.
If Shaman is used to submit the job files, store the job's checkout ID
(i.e. the path relative to the checkout root) in the database. This will
make it possible in the future to remove the Shaman checkout along with
the job itself.
Add fields to the job schemas (`SubmittedJob` and `Job`) to allow
storing the shaman checkout ID (so the Shaman checkout can be deleted
along with the job later).
Job preparation (that is, constructing the data to be sent to the Flamenco
API) shouldn't fail, but in case it does, it should stop the rest of the
submission process.
Manager had a limit of 10 MB, but the Worker can produce images that are
larger than that (even after down-scaling the image). I've bumped the
limit to 25 MB, which should be enough (it's 2x the bug reporter's file
size).
Add a timeout when fetching a job from the persistence layers.
It's my intention to add more timeouts, so this also introduces some code
to make it easier to test that a context has a deadline set.
Deduplicate API implementation code to fetch a job from the persistence
service.
Almost no functional changes. Checking that the requested job UUID is
actually a valid UUID is now consistently done on all fetches. This is
not a functional change in normal Flamenco operations, where only valid
UUIDs are used anyway.
Expand the 'Variables' page so that it links to the two-way variables
page from within the body of text. Having the link in only the side-bar
was too easy to miss.
Move the "Custom Job Types" expandable section to its own, normal
section.
The expandable sections in the docs aren't that clear, visually. It's
easy to miss that they can be expanded. This was the only use of this
feature.
Add support for Blender's `use_all_linked_data_direct` experimental
option. This is a workaround for an issue with BAT, for which a quick
solution is required by the Blender Studio (production crunch).
Flamenco writes the `.flamenco.blend` file with
`preferences.experimental.use_all_linked_data_direct` set to `True`, so
that BAT has an easier time finding linked assets.
Run `PRAGMA journal_mode = WAL` and `PRAGMA synchronous = normal` when
connecting to the SQLite database. This enables the write-ahead-log journal
mode, which makes it safe to enable "normal" synchronisation (instead of
the default "full" synchronisation).
When a command (like `blender` or `ffmpeg`) outputs lines that are longer
than our buffer, they are broken into buffer-sized chunks. Extra code has
been added to ensure those chunks consist of valid UTF-8 characters.
The preview video task would attempt to use JPEG preview files when the
"Preview" checkbox is checked, even when this checkbox is not shown in
Blender's UI (when the output format is not EXR). Blender still stores
this property, even when it's unused, and that's what tripped up the job
compiler.
The "Simple Blender Render" job type now first checks whether the previews
are necessary at all, and only then uses them.
If file has never been saved, Flamenco cannot create a temp copy to
submit. This caused an exception, which is now avoided by showing a
clear error message instead.
Maniphest Tasks: T102192
Differential Revision: https://developer.blender.org/D16358
Remove the "what is new" link, because it's not that relevant for a quick-
start page. Also it got lost a bit between the videos and felt out of
place.
Add an endpoint that mimicks the job submission endpoint, to see whether
the job survives the job compiler script. This can be used to fail early,
before actually sending files to the farm.
The two-way variable replacement function changes the submitted job. To
clarify that this happens, pass the pointer `&submittedJob`.
Both pass-by-pointer and pass-by-value work, because the variable
replacement typically works on maps/slices, which are passed by reference
anyway. Better to be explicit about this, though.
No functional changes.
Fix an issue where workers would switch immediately on a state change
request, even if it was of the "after task is finished" kind.
The "may I keep running" endpoint wasn't checking the lazyness flag, and
thus any state change, lazy or otherwise, would interrupt the worker's
current task.
