The Manager now loads the JavaScript files for job types on demand,
instead of caching them in memory at startup.
This will make certain calls a bit less performant, but in practice this
is around the order of a millisecond so it shouldn't matter much.
Fixes: #104349
Return a custom error type from `VM.getCompileJob()`, so that underlying
errors can be wrapped and more info can be given to the caller. This
replaces a local logger call, preventing both the function and its
caller logging the same error.
Protect the public functions of the task state machine with a mutex, so
that only one task/job state change is handled at a time.
This should avoid race conditions.
Prevent an error when fetching a task that was never assigned to a
worker.
The error:
```
WRN error fetching task worker
error="fetching worker : worker not found: sql: no rows in result set"
```
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit deals with the remaining areas, like the job deleter, task
timeout checker, and task state machine. And anything else to get things
running again.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit deals with the worker sleep schedule.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit deals with worker tags (on both workers and jobs).
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit mostly deals with workers, including the sleep schedule and
task scheduler.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Replace old used-to-be-GORM datastructures (#104305) with sqlc-generated
structs. This also makes it possible to use more specific structs that
are more taylored to the specific queries, increasing efficiency.
This commit covers job blocklists and last-rendered images.
Functional changes are kept to a minimum, as the API still serves the
same data.
Because this work covers so much of Flamenco's code, it's been split up
into different commits. Each commit brings Flamenco to a state where it
compiles and unit tests pass. Only the result of the final commit has
actually been tested properly.
Ref: #104343
Fix most linter warnings reported by 'staticcheck'. This doesn't fix all
of them, some unused functions are still there, and some generated code
also still triggers some warnings. Most issues are fixed, though.
No functional changes, except for the captialisation of some error
messages.
Move the 'Single Image Render' job type to the 'third party job types'
section on the website. It needs more testing & finessing before it can be
bundled with Flamenco.
Fix a build issue when `go mod vendor` was used to vendor all the sources.
This now includes Mage (`github.com/magefile/mage/mage`), so that the
build tool itself can be built too.
Use RFC 2047 (aka MIME encoding) to send the original filename when
uploading a file to the Shaman server.
HTTP headers should be ASCII-only, and some systems use Latin-1 as
fallback. That's not suitable in general, though, because almost all
characters fall outside the Latin-1 range.
GORM implicitly sets 'created at', 'updated at' and 'deleted at' timestamps
to 'now' by calling a 'now function'. This is now implemented by Flamenco
directly, instead of relying on GORM.
Ref: #104305
This also corrects the sleep schedule schema to actually store the
`is_active` field as `boolean` (it was `numeric`, which is the same
underlying field type in SQLite, but produces a different struct field
in the sqlc-generated Go code).
Ref: #104305
Instead of returning an error when getting the sqlc queries object, just
panic. This'll make the calling code quite a bit simpler. The situation
in which it might error out is so rare that I've never seen it, and I
don't even know if it will ever be possible to happen with the SQLite
implementation we use now. Furthermore, once we get rid of GORM, it
should just always work anyway.
Ref: #104305
Add an option to the setup assistant to skip configuring the path to
Blender. It will just use the `default` option, which causes the Workers
to try and find Blender on their own.
Fixes#104306
Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104306
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
Add a new job type that can render a single image. It is broken up
into separate tiles, each of which can be rendered independently by a
worker. Only tested with Cycles.
Adaptive sampling is supported. For this, each tile is expanded by 16
pixels in each direction, which is later cropped off before merging
the tiles.
Denoising is not (yet) supported, as Blender/Cycles does not output
all the necessary data into EXR layers.
Tile sizes should (for now) be a power of 2, to avoid alignment
issues.
Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104327
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
The command used to be `create-video`, but got renamed to
`frames-to-video`. This cleanup renames the `CreateVideoParams` struct
to `FramesToVideoParams`.
No functional changes.
Convert most of the job blocklist queries from GORM to sqlc. The management
functions (add worker, remove worker, clear list, fetch list) have been
converted.
There was a missing condition, which meant that configuring any two-way
variable caused path separator normalisation on all job/task/command
parameters.
