Add a `Worker` column to the Job Tasks Table. This lets artists quickly
visualize on which machine a task is currently running, giving better
insights on worker utilization, as well as better expectations on how
long a task might take to finish when running Flamenco on a Renderfarm
made of different slow / fast workers.
Similarly to the Task Details panel for the "Assigned To" field
`LinkWorker` Vue element, the cell element contains an hyperlink to the
corresponding worker in the Workers page. Since the Worker page also
contains a backlink to the currently running task, this lets user
quickly navigate between the two pages, as seen in the screen recording
demo below.
Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104402
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
Remove the "extra checkout paths" feature in order to simplify the
configuration file, and thus also the upcoming web interface to edit it.
The "extra checkout paths" feature was added to aid in transition from
the Flamenco v2 shaman system to the v3 system. It is very unlikely that
there is still use of Flamenco v2 by people who will want to migrate to
v3 in the future. I expect that if they wanted to, they'd have done so
by now.
Ref: #104403
When a Worker is soft-deleted, references to it are not cleaned up by
the database (that only happens when it is really deleted). As a result,
the "last-assigned worker of a task" field is still set, but the worker
itself cannot be fetched any more.
This is just a quick fix to avoid an error. It's probably better to
remove the soft-deletion of Workers, as the feature is not really used
anyway. Or to implement it properly, so that the info is used.
The `else` block does contain a comment to explain what situation we're
in when the `if` condition is false. I like that construct, but
golangci-lint did not.
No functional changes.
Explicitly ignore an error when requesting the Manager to shut down.
This was already ignored implicitly, but now it's explicit to make
linters happy.
No functional changes.
Log an error when a database transaction rollback fails. I wouldn't know
how to test, because usually stuff Just Works, but it's good to have
this logged anyway.
When the `blender` variable has no value, Flamenco will not be able to
run Blender, which is worth emitting a warning about early on.
This required a bit of a reshuffle of the configuration loading logic, to
avoid loading things twice (and thus warning twice).
Fixes part of #104366
When a Worker would sign off while working on a task, and that task's
job would be in `pause-requested` state, it would always re-queue the
task. This in turn would not get detected, which caused the job to get
stuck.
Now tasks correctly go to `paused` when a Worker signs off, and
subsequently the job will be re-checked and go to `paused` when possible
as well.
Note that this does not handle already-stuck jobs. That'll be for the
following commit.
The conversion from a known path prefix to a variable is now done in a
case-insensitive way. This means that if the variable `{storage}` has
value `S:\Flamenco`, a path `s:\flamenco\project\file.blend` will be
recognised and replaced with `{storage}\project\file.blend`.
This happens uniformly, regardless of the platform. So also on Linux,
which has a case-sensitive filesystem, this matching is done in a
case-insensitive way. It is very unlikely that a Flamenco configuration
has two separate variables, for paths that only differ in their case.
Fixes: #104336 Drive letter case mismatch causes two way variables not
to work correctly
The Manager now loads the JavaScript files for job types on demand,
instead of caching them in memory at startup.
This will make certain calls a bit less performant, but in practice this
is around the order of a millisecond so it shouldn't matter much.
Fixes: #104349
Return a custom error type from `VM.getCompileJob()`, so that underlying
errors can be wrapped and more info can be given to the caller. This
replaces a local logger call, preventing both the function and its
caller logging the same error.
Protect the public functions of the task state machine with a mutex, so
that only one task/job state change is handled at a time.
This should avoid race conditions.
Prevent an error when fetching a task that was never assigned to a
worker.
The error:
```
WRN error fetching task worker
error="fetching worker : worker not found: sql: no rows in result set"
```
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit deals with the remaining areas, like the job deleter, task
timeout checker, and task state machine. And anything else to get things
running again.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit deals with the worker sleep schedule.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit deals with worker tags (on both workers and jobs).
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit mostly deals with workers, including the sleep schedule and
task scheduler.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit covers job blocklists and last-rendered images.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Fix most linter warnings reported by 'staticcheck'. This doesn't fix all
of them, some unused functions are still there, and some generated code
also still triggers some warnings. Most issues are fixed, though.
No functional changes, except for the captialisation of some error
messages.
Move the 'Single Image Render' job type to the 'third party job types'
section on the website. It needs more testing & finessing before it can be
bundled with Flamenco.
Use RFC 2047 (aka MIME encoding) to send the original filename when
uploading a file to the Shaman server.
HTTP headers should be ASCII-only, and some systems use Latin-1 as
fallback. That's not suitable in general, though, because almost all
characters fall outside the Latin-1 range.
GORM implicitly sets 'created at', 'updated at' and 'deleted at' timestamps
to 'now' by calling a 'now function'. This is now implemented by Flamenco
directly, instead of relying on GORM.
Ref: #104305
This also corrects the sleep schedule schema to actually store the
`is_active` field as `boolean` (it was `numeric`, which is the same
underlying field type in SQLite, but produces a different struct field
in the sqlc-generated Go code).
Ref: #104305
Instead of returning an error when getting the sqlc queries object, just
panic. This'll make the calling code quite a bit simpler. The situation
in which it might error out is so rare that I've never seen it, and I
don't even know if it will ever be possible to happen with the SQLite
implementation we use now. Furthermore, once we get rid of GORM, it
should just always work anyway.
Ref: #104305
