Change the package base name of the Go code, from
`git.blender.org/flamenco` to `projects.blender.org/studio/flamenco`.
The old location, `git.blender.org`, has no longer been use since the
[migration to Gitea][1]. The new package names now reflect the actual
location where Flamenco is hosted.
[1]: https://code.blender.org/2023/02/new-blender-development-infrastructure/
Fix an issue where a shared storage path on Linux, that maps via two-way
variables to a drive root on Windows, caused problems with the path
translation system.
Windows paths that consist only of a drive letter (`F:`) cannot just be
concatenated to a relative path, as that will result in `F:path\to\file`,
which is still a relative path of sorts. This is now handled correctly,
and should result in `F:\path\to\file`.
This fixes#104237.
The data is still the same, but the names of the properties have changed
a bit so that they're more generic, declarative, instead of specific to
one bit of functionality.
The goal is to have the `evalInfo.description` field usable for the
'evaluate now' button in the add-on as well. That way it should be
clearer what that does.
This commit just updates the OpenAPI definition.
Refactor the job settings. The `autoevalLockable` boolean is now
replaced with an `evalOnSubmit` nested object:
```
evalOnSubmit: {
showButton: true,
placeholder: "Scene frame range",
},
```
This makes it possible to add a placeholder text, and later maybe some
other parameters. The `showButton: true` part always has to be there, as
the entire feature is disabled with `showButton: false`, in which case
it's better to just remove the entire `evalOnSubmit` sub-object
altogether. Still, I think it's preferred to have that `showButton:
true` in there, as it makes it more explicit what this section of the
settings is for.
This commit just contains the OpenAPI definition.
Add a new job setting option `autoevalLockable`. Setting this to `true` in
the job compiler's `JOB_TYPE` settings has the following effect:
- By default, the setting will not be editable in Blender's job submission
interface. Instead, a toggle button with a 'car' icon will be shown.
- When the 'car' button is toggled off, the setting becomes editable again.
In its default, uneditable state, the setting will be auto-evaluated before
submission.
This makes it possible to 'lock in' auto-evaluation. The main use case is
for the frame range of the render job. By default this will be locked to
the scene frame range, but it can still be overridden if a different
range is wanted.
This commit just contains the necessary OpenAPI change.
Make it explicit that the `version` property is for human consumption.
Also add a new `git` property so that all info from `version` is also
included in separate fields for machine consumption.
Add more logging of timestamps in the actual code, and a few sanity
checks in unit tests.
These were useful while trying to find the root cause of #104218 and might
be useful in the future too. The solution to that issue will be committed
later.
Remove the following statuses from `flamenco-openapi.yaml`:
- 'construction-failed'
- 'archiving'
- 'archived'
These were a leftover from Flamenco v2 and have never been used in
Flamenco v3.
Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104215
Reimplement the `touch()` function on Linux to avoid depending on the
`syscall` package, and use the `sys/unix` package instead. This is
slightly higher level, and seems to build on AMD64 and ARM64.
Clusters can be created without UUID now. In that case, a random one will
be generated. The cluster will be returned by the creation call, so that
the caller can know that generated UUID.
Worker Clusters can be managed via the API, workers can be assigned to
any number of clusters (if not assigned to any, they'll pick up any task).
Jobs can be submitted with a cluster ID, in which case only workers that
are in that cluster or are clusterless will pick up its tasks.
Add a "what-would-delete-do" operation, to query the Manager about what
the deletion of a specific job would entail. For some jobs the job files
will also be deleted (if they were created with a new enough Flamenco),
otherwise they will remain untouched.
Also expand the `SocketIOJobUpdate` schema to include info about job
deletion.
Implement the `deleteJob` API endpoint. Calling this endpoint will mark
the job as "deletion requested", after which it's queued for actual
deletion. This makes the API response fast, even when there is a lot of
work to do in the background.
A new background service "job deleter" keeps track of the queue of such
jobs, and performs the actual deletion. It removes:
- Shaman checkout for the job (but see below)
- Manager-local files of the job (task logs, last-rendered images)
- The job itself
The removal is done in the above order, so the job is only removed from the
database if the rest of the removal was succesful.
Shaman checkouts are only removed if the job was submitted with Flamenco
version 3.2. Earlier versions did not record enough information to reliably
do this.
If Shaman is used to submit the job files, store the job's checkout ID
(i.e. the path relative to the checkout root) in the database. This will
make it possible in the future to remove the Shaman checkout along with
the job itself.
Add fields to the job schemas (`SubmittedJob` and `Job`) to allow
storing the shaman checkout ID (so the Shaman checkout can be deleted
along with the job later).
Add a timeout when fetching a job from the persistence layers.
It's my intention to add more timeouts, so this also introduces some code
to make it easier to test that a context has a deadline set.
Add an endpoint that mimicks the job submission endpoint, to see whether
the job survives the job compiler script. This can be used to fail early,
before actually sending files to the farm.
Add an operation `getSharedStorage` that can return the shared storage
location, adjusted for the given audience & platform. This uses the
two-way variables system to adjust the Manager's configuration.
Include a `shortversion` property in the `FlamencoVersion` schema, which
will just be the version number with the release phase (and not the git
hash, the number of commits since the last tag, and the `-dirty` suffix).
Two-way variable replacement now also changes the path separators. Since
the two-way replacement is made for paths, it makes sense to also clean up
the path for the target platform.
This will remove a worker by soft-deletion. Any task still assigned to
the worker will be requeued.
Note that this removal should only happen when the worker is offline, or
it will cause errors on the worker as its credentials will not be
accepted any more.
Instead of erroring out when a symlink already exists, investigate it. If
the linked file is the one that's intended, just use it.
For some reason, BAT and/or the Flamenco add-on include some files twice
in the checkout request to Shaman. This is now handled gracefully.
