2219 Commits

Author SHA1 Message Date
Sybren A. Stüvel
4b0cdfb735 Take release cycle from makefile instead of parsing the version
Take the release cycle from the `RELEASE_CYCLE` variable in the Makefile,
instead of parsing `VERSION`.

For alpha/beta/rc/release this doesn't matter, it was parsed correctly.
But other suffixes were all marked as "release", which wasn't correct.
2025-07-29 17:11:56 +02:00
Sybren A. Stüvel
cb8354f784 Update CHANGELOG 2025-07-29 17:06:14 +02:00
Sybren A. Stüvel
37b073b8d5 Worker: add JSON tags to config structs
Add JSON tags to the worker configuration structs. This way, the config
is converted to JSON the same as it is to YAML, which is reflected in how
the loaded configuration is logged at startup.
2025-07-29 17:04:55 +02:00
Sybren A. Stüvel
9cc7556720 Website: better redirect URL for SIGGRAPH 2025 2025-07-24 15:15:14 +02:00
Sybren A. Stüvel
5c9a9eacbc Website: replace SIGGRAPH 2023 page with one for 2025
I'm doing another SIGGRAPH talk about Flamenco, and it helps to have a
page on flamenco.blender.org to send people to.

It directs to my own website, as then I can use syncthing to control its
contents while I'm at the conference. Not needing access to the Blender
infrastructure seems safer to me.
2025-07-24 15:13:40 +02:00
Sybren A. Stüvel
fda68ddd33 Cleanup: remove commented-out code
Remove some commented-out old Python code.

No functional changes.
2025-07-22 17:55:04 +02:00
Vivian Leung
78dcd2bdc1 Website: add explanation of flamenco-manager.yaml settings (#104400)
Most of the definitions are based on the documentation in
`internal/manager/config/config.go` or inferred by labels and
descriptions from Flamenco v2 config pages.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104400
2025-07-22 15:14:24 +02:00
Vivian Leung
6284734521 Fix Workers and Tags view when window width is 960px or less (#104413)
Issue likely began after 7b31eba8d7614e82cfcc4583e1dd3c2f40f87944 as
it was an issue with header not being properly positioned.

Just needed to add `header-L` and `header-R` to `grid-template-areas`
and adjust the rest accordingly, and fix a class name typo.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104413
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-07-18 08:15:29 +02:00
Vivian Leung
dcb184e28e Prevent text in nav list items from wrapping onto multiple lines (#104411)
Force menu items to stay on one line.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104411
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-07-17 10:54:51 +02:00
Sybren A. Stüvel
4551be29f6 Fix #104412: Typo in tooltip ".blend_project" → ".blender_project" 2025-07-17 10:34:25 +02:00
Sybren A. Stüvel
0000619f67 Manager: improve "loading configuration" log message
Raise the log level from DEBUG to INFO, as it's quite important to be
explicit about which configuration file is loaded. Also ensure that the
file path is made absolute, so that it's again more explicit.
2025-07-15 10:54:48 +02:00
Vivian Leung
7e40e1bbb0 Also use our Duration in pkg/shaman (#104408)
Move the `Duration` struct from `internal/manager/config` to
`pkg/duration` so that it can be used by `pkg/shaman` as well.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104408
2025-07-11 16:41:41 +02:00
Sybren A. Stüvel
4a0c7b21db Upgrade BAT to 1.20
To get this:
16c208bc8e
2025-07-11 16:32:18 +02:00
Sybren A. Stüvel
f6338064ab Git-ignore temporary files from mockgen
Ignore files that can linger when [mockgen][1] fails to generate code.
They are indeed not meant to be committed.

[1]: https://github.com/golang/mock)
2025-07-07 11:03:14 +02:00
Sybren A. Stüvel
9e7e711ebd Update changelog 2025-07-04 11:23:36 +02:00
Vivian Leung
11a052d854 Web app: Add multi-select of jobs (#104391)
Add the following features:

- `Ctrl + Click` or `Cmd + Click` to toggle selection of additional jobs
- `Shift + Click` to select a range of additional jobs
- Ability to perform `Pause`, `Cancel`, `Requeue`, and `Delete` actions on multiple jobs concurrently
- Notifications on how many jobs successfully/failed to have an action performed

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104391
2025-07-04 11:18:48 +02:00
Vivian Leung
1044350b4b Remove unused code: ShamanGarbageCollect struct in config.go (#104407)
No functional changes.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104407
2025-07-03 09:24:28 +02:00
Sybren A. Stüvel
4492d824cb Manager: Add custom Duration type for the configuration file
Add a custom `time.Duration` wrapper that can be marshalled to JSON as
well. Go's built-in marshaller just treats it as an `int64` and
therefore the values are nanoseconds. This new wrapper keeps the JSON
representation the same as the YAML marshaller (which uses the
`time.Duration.String()` function).

In preparation for !104406.
2025-07-01 19:43:09 +02:00
Jonas Holzman
f0fca60427 Tasks Table: Add Worker column (#104402)
Add a `Worker` column to the Job Tasks Table. This lets artists quickly
visualize on which machine a task is currently running, giving better
insights on worker utilization, as well as better expectations on how
long a task might take to finish when running Flamenco on a Renderfarm
made of different slow / fast workers.

Similarly to the Task Details panel for the "Assigned To"  field
`LinkWorker` Vue element, the cell element contains an hyperlink to the
corresponding worker in the Workers page. Since the Worker page also
contains a backlink to the currently running task, this lets user
quickly navigate between the two pages, as seen in the screen recording
demo below.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104402
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-06-30 14:37:00 +02:00
Sybren A. Stüvel
8f8cd564f6 Mark sqlc-generated files as 'generated' in .gitattributes
No functional changes.
2025-06-30 14:30:35 +02:00
Sybren A. Stüvel
1b9b40728d Website: improve docs about installation of code generators
Link to the official Java installer, as that doesn't require XCode to be
installed (on macOS). And that works on Windows too.
2025-06-30 11:23:43 +02:00
Sybren A. Stüvel
65f391183e Add install-generators make target and improve documentation
Add Make target `install-generators` as alias for `mage installGenerators`
and update the documentation for it.
2025-06-30 11:01:15 +02:00
Sybren A. Stüvel
807f665587 Remove the Shaman's "extra checkout paths" feature
Remove the "extra checkout paths" feature in order to simplify the
configuration file, and thus also the upcoming web interface to edit it.

The "extra checkout paths" feature was added to aid in transition from
the Flamenco v2 shaman system to the v3 system. It is very unlikely that
there is still use of Flamenco v2 by people who will want to migrate to
v3 in the future. I expect that if they wanted to, they'd have done so
by now.

Ref: #104403
2025-06-25 10:14:50 +02:00
Sybren A. Stüvel
716ee49d7e Fix #104401: Worker Sign-On Failure after vulnerability library upgrade
Upgrade `github.com/getkin/kin-openapi` from v0.131.0 to v0.132.0, as that
fixes exactly this bug.
2025-06-19 12:41:24 +02:00
Sybren A. Stüvel
6f54cfc1b8 Update CHANGELOG 2025-06-19 12:11:07 +02:00
Vivian Leung
6c23d5966d Tasks Table: Freeze sorting on Shift key down (#104388)
Listen to the key press of the Shift key on the Tasks list and disable
any table sorting until it is released.

Related to #104386. When a user is performing multi-select, they hold
the Shift key down and intend to click the next row which determines
which rows are selected. To avoid unintended selections, the UI should
not update or shift around in the middle of the user interaction.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104388
2025-06-19 12:09:37 +02:00
Sybren A. Stüvel
0b50f154cc Website: remove the instructions to install make
`make` is no longer needed to build Flamenco, and so the installation
instructions (especially for Windows and macOS) can be made considerably
simpler.
2025-06-17 10:41:31 +02:00
Sybren A. Stüvel
40d66d4a8e Update CHANGELOG 2025-06-17 10:38:35 +02:00
Sybren A. Stüvel
d7d4c62804 Fix vulnerability GO-2025-3503
Vulnerability: GO-2025-3503
    HTTP Proxy bypass using IPv6 Zone IDs in golang.org/x/net
  More info: https://pkg.go.dev/vuln/GO-2025-3503
  Module: golang.org/x/net
    Found in: golang.org/x/net@v0.27.0
    Fixed in: golang.org/x/net@v0.36.0
    Example traces found:
      #1: internal/manager/eventbus/mqtt_client.go:103:37: eventbus.MQTTForwarder.Connect calls autopaho.NewConnection, which eventually calls proxy.FromEnvironment
      #2: internal/manager/eventbus/mqtt_client.go:103:37: eventbus.MQTTForwarder.Connect calls autopaho.NewConnection, which eventually calls proxy.PerHost.Dial
2025-06-17 10:36:45 +02:00
Sybren A. Stüvel
8e18a5aa8e Fix vulnerability GO-2025-3533
Vulnerability: GO-2025-3533
    Improper Handling of Highly Compressed Data (Data Amplification) in
    github.com/getkin/kin-openapi/openapi3filter
  More info: https://pkg.go.dev/vuln/GO-2025-3533
  Module: github.com/getkin/kin-openapi
    Found in: github.com/getkin/kin-openapi@v0.88.0
    Fixed in: github.com/getkin/kin-openapi@v0.131.0
    Example traces found:
      #1: cmd/flamenco-manager/webservice.go:193:17: flamenco.runWebService calls echo.Echo.Start, which eventually calls openapi3filter.ValidateRequest
2025-06-17 10:36:45 +02:00
Sybren A. Stüvel
8813538c31 Bump Go from 1.23.1 to 1.24.4
This fixes four security issues:

Vulnerability #1: GO-2025-3751
    Sensitive headers not cleared on cross-origin redirect in net/http
  More info: https://pkg.go.dev/vuln/GO-2025-3751
  Standard library
    Found in: net/http@go1.23.5
    Fixed in: net/http@go1.23.10
    Example traces found:
      #1: pkg/api/openapi_client.gen.go:1212:20: api.Client.TaskOutputProducedWithBody calls http.Client.Do
      #2: pkg/api/openapi_spec.gen.go:318:36: api.GetSwagger calls openapi3.Loader.LoadFromData, which eventually calls http.Get

Vulnerability #2: GO-2025-3750
    Inconsistent handling of O_CREATE|O_EXCL on Unix and Windows in os in
    syscall
  More info: https://pkg.go.dev/vuln/GO-2025-3750
  Standard library
    Found in: os@go1.23.5
    Fixed in: os@go1.23.10
    Platforms: windows
    Example traces found:
      #1: internal/manager/last_rendered/image_processing.go:54:24: last_rendered.saveJPEG calls os.Create
      #2: internal/manager/api_impl/meta.go:144:28: api_impl.Flamenco.CheckSharedStoragePath calls os.CreateTemp
      #3: internal/manager/job_compilers/scripts.go:54:31: job_compilers.loadScriptsFrom calls fs.ReadDir, which calls os.File.ReadDir
      #4: pkg/shaman/filestore/testing.go:107:25: filestore.LinkTestFileStore calls filepath.Walk, which eventually calls os.File.Readdirnames
      #5: internal/manager/local_storage/local_storage.go:116:22: local_storage.getSuitableStorageRoot calls os.Getwd
      #6: pkg/shaman/filestore/testing.go:107:25: filestore.LinkTestFileStore calls filepath.Walk, which calls os.Lstat
      #7: pkg/shaman/checkout/manager.go:274:24: checkout.Manager.SymlinkToCheckout calls os.MkdirAll
      #8: pkg/shaman/filestore/testing.go:38:30: filestore.CreateTestStore calls os.MkdirTemp
      #9: pkg/shaman/touch/touch.go:26:2: touch.init calls os.init, which calls os.NewFile
      #10: internal/manager/task_logs/task_logs.go:183:22: task_logs.Storage.Tail calls os.Open
      #11: internal/manager/config/config.go:662:23: config.Conf.Write calls os.OpenFile
      #12: cmd/blender-runner/blender-runner.go:43:35: blender.main calls exec.Cmd.StderrPipe, which calls os.Pipe
      #13: internal/manager/job_compilers/scripts.go:54:31: job_compilers.loadScriptsFrom calls fs.ReadDir, which eventually calls os.ReadDir
      #14: internal/manager/config/config.go:199:30: config.loadConf calls os.ReadFile
      #15: pkg/shaman/checkout/manager.go:196:11: checkout.Manager.EraseCheckout calls os.Remove
      #16: pkg/shaman/checkout/manager.go:189:24: checkout.Manager.EraseCheckout calls os.RemoveAll
      #17: internal/manager/config/config.go:647:21: config.Conf.Overwrite calls os.Rename
      #18: cmd/blender-runner/blender-runner.go:52:21: blender.main calls exec.Cmd.Start, which calls os.StartProcess
      #19: pkg/shaman/checkout/manager.go:175:18: checkout.Manager.EraseCheckout calls os.Stat
      #20: pkg/shaman/checkout/manager.go:244:18: checkout.Manager.SymlinkToCheckout calls os.Symlink
      #21: cmd/sqlc-export-schema/main.go:87:24: sqlc.saveSchema calls os.WriteFile
      #22: web/web_app.go:55:24: web.FSWrapper.Open calls echo.defaultFS.Open, which calls os.dirFS.Open
      #23: internal/manager/job_compilers/scripts.go:54:31: job_compilers.loadScriptsFrom calls fs.ReadDir, which calls os.dirFS.ReadDir
      #24: internal/manager/config/config.go:662:23: config.Conf.Write calls os.OpenFile, which eventually calls syscall.Open

Vulnerability #3: GO-2025-3563
    Request smuggling due to acceptance of invalid chunked data in net/http
  More info: https://pkg.go.dev/vuln/GO-2025-3563
  Standard library
    Found in: net/http/internal@go1.23.5
    Fixed in: net/http/internal@go1.23.8
    Example traces found:
      #1: pkg/shaman/hasher/copier.go:51:21: hasher.Copy calls http.body.Read, which eventually calls internal.chunkedReader.Read

Vulnerability #4: GO-2025-3447
    Timing sidechannel for P-256 on ppc64le in crypto/internal/nistec
  More info: https://pkg.go.dev/vuln/GO-2025-3447
  Standard library
    Found in: crypto/internal/nistec@go1.23.5
    Fixed in: crypto/internal/nistec@go1.23.6
    Platforms: ppc64le
    Example traces found:
      #1: internal/manager/config/config.go:666:13: config.Conf.Write calls fmt.Fprintf, which eventually calls nistec.P256Point.ScalarBaseMult
      #2: internal/manager/config/config.go:666:13: config.Conf.Write calls fmt.Fprintf, which eventually calls nistec.P256Point.ScalarMult
      #3: internal/manager/config/config.go:666:13: config.Conf.Write calls fmt.Fprintf, which eventually calls nistec.P256Point.SetBytes
2025-06-17 10:35:34 +02:00
Sybren A. Stüvel
0315f15277 Add make check target
Add a `make check` target, which just forwards to `mage check`.
2025-06-17 10:15:58 +02:00
Sybren A. Stüvel
67d66422fd Website: update & expand doc of mage and make targets
Update & expand the available argets for `mage` and `make`.
2025-06-17 10:14:46 +02:00
Sybren A. Stüvel
564375d40f Addon: bump BAT v1.18 → 1.19
This is necessary to support Blender 5.0 files.

See blender/blender-asset-tracer!92893 for more info.
2025-06-16 12:46:38 +02:00
Vivian Leung
dbb393f62f Add a test script to add a mock Worker (#104394)
Add a script `debug-worker-add.sh` which allows additions of mock
workers to Flamenco. It is useful for testing the UI for larger lists of
worker items.

It assists the development of multi-selection for workers #104395

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104394
2025-06-10 11:41:19 +02:00
Jonas Holzman
542fb5403b Docs: Fix wrong exe parameter name for the exec command (#104396)
Small documentation issue I ran into while writing a custom job type

The exec command takes an `exe` parameter, similar to other commands,
but the documentation wrongly indicates the parameter should be `exec`.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104396
2025-06-07 11:42:43 +02:00
Sybren A. Stüvel
8b4eeb7c69 Update CHANGELOG.md 2025-06-02 12:19:52 +02:00
Vivian Leung
984b132d81 Webapp: Add multi-select of tasks and support actions for multiple tasks (#104386)
Add the following features:
- `Ctrl + Click` / `Cmd + Click` to toggle selection of additional
  tasks.
- `Shift + Click` to select a range of additional tasks.
- Ability to perform `Cancel` and `Requeue` actions on multiple tasks
  concurrently.
- Notifications on how many tasks successfully/failed to have an
  action performed.

Tabulator has selectable rows built-in and provides a function that
can return an array of all selected rows. However, tabulator's default
behavior for multi-selection does not reset to a single task after
each regular click. Therefore, I built a custom multi-select using the
tabulator API, introducing `Shift + click` and `Ctrl + click` and
matching their behaviors as they work in most file explorers,
including Blender's.

In addition to manipulating the Tabulator's row selection, the state
of selected Tasks is also needs to be copied to Pinia stores. This
stores will allow us to access selected Tasks from any component and
make API calls on them.

Ref: #99396

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104386
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-06-02 12:17:43 +02:00
Jonas Holzman
ccc002036b Fix v3.7 release year in CHANGELOG.md (#104393)
It's 2025 already!

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104393
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-05-30 09:12:49 +02:00
Vivian Leung
c5785a7c97 Update Tabulator (#104390)
Updates Tabulator package from 5.4 to 6.3

The relevant breaking change is the change of the `selectable` variable to `selectableRows`

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104390
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-05-27 18:07:42 +02:00
Sybren A. Stüvel
3541575a75 Avoid error when fetching task that was assigned to soft-deleted worker
When a Worker is soft-deleted, references to it are not cleaned up by
the database (that only happens when it is really deleted). As a result,
the "last-assigned worker of a task" field is still set, but the worker
itself cannot be fetched any more.

This is just a quick fix to avoid an error. It's probably better to
remove the soft-deletion of Workers, as the feature is not really used
anyway. Or to implement it properly, so that the info is used.
2025-05-26 17:57:51 +02:00
tariqsulley
55491675e2 Typo Fix (#104389)
This pr fixes a typo on the flamenco shared storage page.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104389
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-05-25 15:52:19 +02:00
Sybren A. Stüvel
33fbddc32b Add 'task poker' command
Add a utility command `cmd/task-poker` that takes a job ID and changes the
status of a random task every second.

This is a development utility, to help work on the Manager's web interface
and deal with constantly-updating tasks.
2025-05-20 18:52:00 +02:00
Vivian Leung
22d43ee068 Blocklist: Wrap table rows in table body, add key directive (#104383)
Fix a potential hydration issue warning from vite, by adding the `key`
attribute to each table row.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104383
2025-05-09 11:31:29 +02:00
Sybren A. Stüvel
dc152d6c9a Website: add alias /devstart to the developer "getting started" page
Add `/devstart` as an alias to `/development/getting-started/` page. This
is to be able to use a shorter link in the documentation I'm writing for
my hands-on class at SIGGRAPH 2025.
2025-05-06 15:27:43 +02:00
Vivian Leung
190e26adc0 Update the Javascript instructions for initializing Flamenco's API (#104387)
Updated to match the current way of initializing API's with `getAPIClient()`

It seems possible the Python instructions might need an update as well.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104387
2025-05-06 09:19:43 +02:00
Vivian Leung
74d96bd02e Fix #104289: Task Name Column partially hidden (#104379)
Adjust the padding for the task name column, by setting a minimum
width.

This makes the task name column wide enough to go into the 10k-99k
frame numbers.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104379
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-04-25 11:02:58 +02:00
Srivathsav-Kyatham
84e4a28d6c Allow CORS requests from localhost for development (#104385)
Allow CORS requests from `http://localhost:<port>`. Some webbrowsers
(like Safari) are picky about hostname vs. IP addresses, and this
makes them work well with the webapp development server.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104385
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2025-04-24 15:23:09 +02:00
Srivathsav-Kyatham
d2bdc9690d Website: update file path of SocketIO handling
Change `webupdates/webupdates.go` to `eventbus/socketio.go`

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104381
2025-04-10 12:02:35 +02:00
Avi Gupta
0c811721fe Website: Add docs for MacOS under "utilities" in "getting started"
Add instructions for installing `make`.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104378
2025-04-07 11:28:20 +02:00