2205 Commits

Author SHA1 Message Date
Mateus Abelli
eeb79f4feb OpenAPI: Add a default option for Blender's location in the setup assist
The setup assistant configuration can now contain `source: default` to
indicate that the default `blender` command should be used (which in
turn tells the Worker to find whatever Blender is available on the
system).

Choosing this option will make the setup assistant skip the path check,
and just trust that the Workers will find Blender.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104306
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2024-09-09 11:22:16 +02:00
Sybren A. Stüvel
3fa15eaa0c Fix test that ensures all expected job types are embedded
It was missing the 'single image render' job type.
2024-09-09 11:21:54 +02:00
Mateus Abelli
6eb4b12840 Addon: add 'disabled' message on submit button when no job type is selected
Add a 'disabled' message on the Submit Job button when no job type has
been selected yet.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104326
2024-09-05 14:46:10 +02:00
Sybren A. Stüvel
423c5c29d7 Shaman: use relative paths for symlinks (#104332)
Use relative paths for symlinks, so that the Shaman storage can be mounted
on different locations on the Worker without the symlinks breaking.

Fixes studio/flamenco#104328

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104332
2024-09-05 12:12:44 +02:00
David Zhang
60872c89f9 Manager: distributed rendering of single images
Add a new job type that can render a single image. It is broken up
into separate tiles, each of which can be rendered independently by a
worker. Only tested with Cycles.

Adaptive sampling is supported. For this, each tile is expanded by 16
pixels in each direction, which is later cropped off before merging
the tiles.

Denoising is not (yet) supported, as Blender/Cycles does not output
all the necessary data into EXR layers.

Tile sizes should (for now) be a power of 2, to avoid alignment
issues.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104327
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2024-09-03 06:47:45 +02:00
Sybren A. Stüvel
acf721c6d8 Add-on & README: Change "Flamenco 3" to just "Flamenco"
When Flamenco 3 was just released, it was important to indicate the
difference between the Flamenco 2 panel (which was just labeled
"Flamenco"), and the new Flamenco 3 panel. The release of v3.0 was
almost two years ago, so it's time to remove the "3" from the titles and
just name things "Flamenco".
2024-08-26 18:24:24 +02:00
Sybren A. Stüvel
329640aa62 Cleanup: rename CreateVideoParams to FramesToVideoParams
The command used to be `create-video`, but got renamed to
`frames-to-video`. This cleanup renames the `CreateVideoParams` struct
to `FramesToVideoParams`.

No functional changes.
2024-08-26 18:16:04 +02:00
waterwheels
8d57a60114 update ffmpeg command in docs (#104333)
The docs listed the ffmpeg command as `create-video` but it seems that the actual command is `frames-to-video`.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104333
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2024-08-26 18:15:14 +02:00
Sybren A. Stüvel
4073c34b0d Website: update another chat link
I missed a link in my previous commit.
2024-08-26 18:05:12 +02:00
Sybren A. Stüvel
2df4f91b68 Website: update link to Blender Chat
Blender Chat moved to Matrix, and with that to a new URL.

https://chat.blender.org/#/room/#flamenco:blender.org
2024-08-25 14:53:04 +02:00
TrevorSmale
819ded5406 Fixed some grammar.
Check out the diff, just made a few minor grammatical adjustments.
2024-08-07 20:49:02 +02:00
Sybren A. Stüvel
1f7cab0ef2 Manager: convert job blocklist management queries to sqlc
Convert most of the job blocklist queries from GORM to sqlc. The management
functions (add worker, remove worker, clear list, fetch list) have been
converted.
2024-08-02 12:50:17 +02:00
Sybren A. Stüvel
e758f8c79d Cleanup: fix typo in comment
No functional changes.
2024-08-02 11:39:59 +02:00
Sybren A. Stüvel
280e6643a5 Fix #104331: Path separator normalisation replaces too much
There was a missing condition, which meant that configuring any two-way
variable caused path separator normalisation on all job/task/command
parameters.
2024-08-02 11:39:59 +02:00
Sybren A. Stüvel
6841a99a90 Simple Blender Render: include scene name in job settings
Include the current scene name in a hidden setting of the Simple Blender
Render, and pass that as CLI argument to Blender.

This ensures that the correct scene is rendered when working directly on
shared storage (as that does not have a copy of the blend file per job).

The job setting `"scene"` is still optional. If it's missing or empty,
the `--scene <scene name>` CLI arg will simply not be passed to Blender.
2024-08-01 14:55:17 +02:00
Sybren A. Stüvel
6704f6619d Add-on: don't write a .flamenco.blend when already on shared storage
Skip writing the `.flamenco.blend` file when working directly on shared
storage. This was already the intended behaviour, but I think it got lost
in a recent refactor.
2024-08-01 14:41:54 +02:00
Sybren A. Stüvel
b68e51976d Manager: actually respect autodiscoverable: false in configuration
The Manager configuration has an option to disable the UPnP/SSDP server,
and now it actually listens to it.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
3f86364fa4 Shaman tests: colorise log output on some tests
Easier to read logging when tests fail. I've just done this where I need
it, as there is no central place where Flamenco logs are initialised for
unit tests.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
466abf1387 Shaman test: replace assert.True(...) with assert.ErrorIs()
Replace `assert.True(t, errors.Is(err, errtype), ...)` with
`assert.ErrorIs(t, err, errtype, ...)`.

No functional changes to the test, except better failure reporting.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
3b6102fe3a Shaman: replace deprecated function with its modern replacement
Replace `ioutil.TempDir()` with `os.MkdirTemp()`.

No functional changes.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
5da5edd333 Manager: modernise Shaman unit test code
Replace calls to `assert.NoError()` with `require.NoError()`
2024-08-01 13:51:43 +02:00
Sybren A. Stüvel
8f9ed0cc28 Manager & Webapp: code changes to handle the new worker tag info in Job
Manager: Instead of embedding the worker tag info in a fetched `Job`,
just include its UUID.

Webapp: fetch the worker tag by UUID, instead of using the embedded
info.
2024-07-30 11:03:03 +02:00
Sybren A. Stüvel
2f4124ebc2 OAPI: regenerate code 2024-07-30 11:01:51 +02:00
Sybren A. Stüvel
7e88ea345d Fix error submitting job to Flamenco
The OpenAPI changes in e561c8080e8f47b6fe792591033be4fab6faef42 did not
sit well with the generated Python code. The worker tag is now
communicated just as UUID (so that it's the same for the `SubmittedJob`
and `Job` types).
2024-07-30 11:01:25 +02:00
Sybren A. Stüvel
786422c8af Update CHANGELOG.md 2024-07-29 17:53:40 +02:00
Sybren A. Stüvel
8f7dd2bc07 Webapp: remove unused code
Remove some code that was meant to show the worker tag, but that never
worked properly.
2024-07-29 17:53:25 +02:00
Sybren A. Stüvel
031667924a Add-on: reformat with black
Use 'black' to reformat the add-on code.

No functional changes.
2024-07-29 17:50:49 +02:00
Sybren A. Stüvel
df4f94c642 Manager: show worker tag in job details
Show the worker tag name (and its description in a tooltip) in the job
details. When no worker tag is assigned, "All Workers" is shown in a more
dimmed colour.

This also renames the "Type" field to "Job Type". "Tag" and "Type" could
be confused, and now they're displayed as "Worker Tag" and "Job Type".

The UI in the add-on's submission interface is also updated for this, so
that that also shows "Worker Tag" (instead of just "Tag").
2024-07-29 17:50:11 +02:00
Sybren A. Stüvel
e8438bb645 OAPI: regenerate code 2024-07-29 17:45:14 +02:00
Sybren A. Stüvel
e561c8080e OAPI: include worker tag info when fetching job
When fetching the details of a job, include the worker tag.
2024-07-29 17:44:29 +02:00
David Zhang
754ee9278e Add-on: Add checkbox to submit jobs in paused status (#104323)
Add a checkbox to the submission interface to submit the job in
`paused` status.

Internally this translates the boolean value of the checkbox to the
desired state (`queued` or `paused`). A checkbox was chosen to
simplify the user interface; technically a drop-down could have been
used to select the initial state, but this would require more clicks
to get the same result.

Pull Request: https://projects.blender.org/studio/flamenco/pulls/104323
2024-07-25 04:26:03 +02:00
Sybren A. Stüvel
eb763a605b Website: document absolute vs. relative asset paths
Flamenco treats assets differently, depending on whether they're referenced
by an absolute or a blendfile-relative path. This is now actually
documented.
2024-07-23 09:33:52 +02:00
David Zhang
00848a3755 Manager: backend for submitting jobs in paused status
With a new `initial_status` field, the user could choose whether to submit a job in `paused` status.

Pull request: https://projects.blender.org/studio/flamenco/pulls/104322
2024-07-09 22:57:01 -04:00
David Zhang
376112b392 OAPI: regenerate code 2024-07-09 22:57:00 -04:00
David Zhang
6f3ba146de OAPI: add initial_status field in SubmittedJob
Pull request: https://projects.blender.org/studio/flamenco/pulls/104322
2024-07-09 22:56:59 -04:00
Sybren A. Stüvel
3b2b5c47f3 Manager: convert FetchWorkerTask() from gorm to sqlc
Ref: #104305

No functional changes
2024-07-02 10:16:41 +02:00
Sybren A. Stüvel
7db91294c2 Manager: fix unit test TestFetchWorkerTask reliability
The unit test was using the real clock, and depends on ordering by
last-updated timestamps. This means that sometimes two updates both fell
within the granularity of the timestamp in sqlite, which made the
ordering unreliable.

Switching to a mocked clock fixes this, as it moves forward with a hard-
coded pace, regardless of the execution speed of the test.

No functional changes to Flamenco itself.
2024-07-02 10:00:08 +02:00
David Zhang
aac55e7e3c Manager: Support pausing jobs
A job first goes to `pause-requested` status, during which any `active` task
gets a chance to be completed. Once there are no more active tasks, the job
goes to `paused` state (or `failed`, if that is applicable).

Pull request: https://projects.blender.org/studio/flamenco/pulls/104313
2024-07-01 10:59:37 -04:00
David Zhang
1330487078 OAPI: regenerate code 2024-07-01 10:46:57 -04:00
David Zhang
8a82b1ee0a OAPI: add pause-requested job status
Pull request: https://projects.blender.org/studio/flamenco/pulls/104313
2024-07-01 10:44:46 -04:00
Sybren A. Stüvel
46d56a7b49 Mark go.sum as 'generated'
The `go.sum` file is not directly managed, but just the result of running
`go get`, `go mod tidy`, etc.
2024-07-01 11:59:52 +02:00
Sybren A. Stüvel
7d8afb5be7 Worker: reduce log level of last-rendered-image processing
Reduce the log level from 'info' to 'debug' for the "output uploader:
processing file before uploading to Manager" message.
2024-07-01 11:34:22 +02:00
Sybren A. Stüvel
842d1ab9a4 Manager: convert storing authored jobs from gorm to sqlc
Convert the creation of new jobs & tasks in the database (which happens
by storing the 'authored job' + its tasks) from gorm to sqlc.
2024-06-30 23:31:28 +02:00
Sybren A. Stüvel
bfe47ea394 Manager: convert task scheduler from gorm to sqlc
Convert the task scheduler from gorm to sqlc. This makes the query
considerably easier to read.

No functional changes intended.
2024-06-30 21:18:08 +02:00
Sybren A. Stüvel
d86c97d06e Manager: Use require.XXX() functions in task scheduler test
Instead of explicitly checking errors / nil values and calling
`t.Fatal()`, just use `require.NoError()` or `require.NotNil()`.

No functional changes. The wording of test failures will be slightly
different though.
2024-06-30 21:18:00 +02:00
Sybren A. Stüvel
6e52d41968 Manager: more sqlc to gorm conversionfunctions for jobs & tasks
Some reorganisation to make it easier to convert a job & its tasks from
sqlc to gorm data structures.

The persistence layer is being converted to sqlc. Once that is done, the
remainder of the code can switch over from using gorm structs to sqlc
structs. Then this code will no longer be necessary.
2024-06-30 21:17:53 +02:00
Sybren A. Stüvel
6fb8ab92e2 Bumped version to 3.6-alpha3 2024-06-27 09:58:55 +02:00
Sybren A. Stüvel
a00c1daac6 Website: bump experimental version to 3.6-alpha2 2024-06-27 09:58:31 +02:00
Sybren A. Stüvel
38f41c096c Make clean after building release packages
To prevent non-native binaries from polluting the working directory, also
run `make clean` after building the release packages.
2024-06-27 09:57:09 +02:00
Sybren A. Stüvel
8b08b91840 Bump golang.org/x/image to fix vulnerability
Vulnerability #1: GO-2024-2937
    Panic when parsing invalid palette-color images in golang.org/x/image
  More info: https://pkg.go.dev/vuln/GO-2024-2937
  Module: golang.org/x/image
    Found in: golang.org/x/image@v0.10.0
    Fixed in: golang.org/x/image@v0.18.0
    Example traces found:
      #1: internal/worker/output_uploader.go:144:36: worker.loadAsJPEG calls image.Decode, which calls tiff.Decode
2024-06-27 09:54:33 +02:00