289 Commits

Author SHA1 Message Date
Sybren A. Stüvel
c71ae574f1 OAPI: replace queryJobs operation with fetchJobs
Replace the `queryJobs` operation (which takes a query object to do
filtering & pagination) with the new `fetchJobs` operation (which just
returns all jobs without filtering).

The reason for this is that the querying that this operation supported
was never used by the front-end. And now it's getting in the way of other
development work, so it has to go.
2024-09-18 14:27:47 +02:00
Mateus Abelli
0a98fd2b96 OAPI: regenerate code 2024-09-09 11:22:16 +02:00
Mateus Abelli
eeb79f4feb OpenAPI: Add a default option for Blender's location in the setup assist
The setup assistant configuration can now contain `source: default` to
indicate that the default `blender` command should be used (which in
turn tells the Worker to find whatever Blender is available on the
system).

Choosing this option will make the setup assistant skip the path check,
and just trust that the Workers will find Blender.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104306
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2024-09-09 11:22:16 +02:00
Sybren A. Stüvel
423c5c29d7 Shaman: use relative paths for symlinks (#104332)
Use relative paths for symlinks, so that the Shaman storage can be mounted
on different locations on the Worker without the symlinks breaking.

Fixes studio/flamenco#104328

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104332
2024-09-05 12:12:44 +02:00
Sybren A. Stüvel
3f86364fa4 Shaman tests: colorise log output on some tests
Easier to read logging when tests fail. I've just done this where I need
it, as there is no central place where Flamenco logs are initialised for
unit tests.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
466abf1387 Shaman test: replace assert.True(...) with assert.ErrorIs()
Replace `assert.True(t, errors.Is(err, errtype), ...)` with
`assert.ErrorIs(t, err, errtype, ...)`.

No functional changes to the test, except better failure reporting.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
3b6102fe3a Shaman: replace deprecated function with its modern replacement
Replace `ioutil.TempDir()` with `os.MkdirTemp()`.

No functional changes.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
5da5edd333 Manager: modernise Shaman unit test code
Replace calls to `assert.NoError()` with `require.NoError()`
2024-08-01 13:51:43 +02:00
Sybren A. Stüvel
2f4124ebc2 OAPI: regenerate code 2024-07-30 11:01:51 +02:00
Sybren A. Stüvel
7e88ea345d Fix error submitting job to Flamenco
The OpenAPI changes in e561c8080e8f47b6fe792591033be4fab6faef42 did not
sit well with the generated Python code. The worker tag is now
communicated just as UUID (so that it's the same for the `SubmittedJob`
and `Job` types).
2024-07-30 11:01:25 +02:00
Sybren A. Stüvel
e8438bb645 OAPI: regenerate code 2024-07-29 17:45:14 +02:00
Sybren A. Stüvel
e561c8080e OAPI: include worker tag info when fetching job
When fetching the details of a job, include the worker tag.
2024-07-29 17:44:29 +02:00
David Zhang
376112b392 OAPI: regenerate code 2024-07-09 22:57:00 -04:00
David Zhang
6f3ba146de OAPI: add initial_status field in SubmittedJob
Pull request: https://projects.blender.org/studio/flamenco/pulls/104322
2024-07-09 22:56:59 -04:00
David Zhang
1330487078 OAPI: regenerate code 2024-07-01 10:46:57 -04:00
David Zhang
8a82b1ee0a OAPI: add pause-requested job status
Pull request: https://projects.blender.org/studio/flamenco/pulls/104313
2024-07-01 10:44:46 -04:00
Sybren A. Stüvel
c307342f9d OAPI: regenerate code 2024-05-20 10:09:22 +02:00
Sybren A. Stüvel
1dc86119f0 OAPI: change URL to https://flamenco.blender.org/
The flamenco.io website has been off the air for quite a while already.
2024-05-20 10:05:45 +02:00
Sybren A. Stüvel
5ee4871e39 OAPI: regenerate code 2024-05-07 11:55:56 +02:00
Sybren A. Stüvel
e712e9450c OAPI: add label to AvailableJobSetting 2024-05-07 11:54:49 +02:00
Sybren A. Stüvel
e2bca9ad61 Worker: add configuration for Linux out-of-memory killer
Add a Worker configuration option to configure the Linux out-of-memory
behaviour. Add `oom_score_adjust=500` to `flamenco-worker.yaml` to increase
the chance that Blender gets killed when the machine runs out of memory,
instead of Flamenco Worker itself.
2024-04-15 17:21:11 +02:00
Taylor Wiebe
2bbb7b48d2 OAPI: regenerate code 2024-04-04 11:12:39 +02:00
Taylor Wiebe
03a889345d OAPI: add optional description to job types
This description will be shown as a tooltip in the job submission UI.
2024-04-04 11:08:55 +02:00
Sybren A. Stüvel
00dfbc10b6 Worker: link to FAQ entry when the worker cannot be found 2024-03-18 22:38:20 +01:00
Sybren A. Stüvel
8cf19876c0 OAPI: regenerate code 2024-03-01 08:36:40 +01:00
Sybren A. Stüvel
3b1421c227 OAPI: add farm status events to the event bus 2024-03-01 08:36:19 +01:00
Sybren A. Stüvel
d9ffe8a1b6 OAPI: regenerate code 2024-02-29 20:38:38 +01:00
Sybren A. Stüvel
0c7bf3bf3b OAPI: add farm status service
Add an OpenAPI operation to fetch the overall farm status from the Manager.
2024-02-29 20:27:44 +01:00
Sybren A. Stüvel
dfdb8e82a1 Worker: Refer to website instead of non-existent example file
The worker-written config files would all refer to
`flamenco-worker-example.yaml`, even though this file doesn't even
exist. Instead, the configuration file will refer to the appropriate
documentation on the website, and the credentials file will explain what
happens when you delete it.

The credentials are otherwise intentionally left undocumented, as their
contents are not to be manually edited. The only thing to do with that
file is delete it so that the Worker re-registers itself at startup.
2024-02-22 12:46:13 +01:00
Sybren A. Stüvel
fd9605583f OAPI: regenerate code 2024-02-21 22:16:23 +01:00
Sybren A. Stüvel
a137ecd3cd OAPI: add lifecycle events
Add events for manager-startup and manager-shutdown.
2024-02-21 22:12:43 +01:00
Sybren A. Stüvel
1ed893fa84 OAPI: regenerate code 2024-02-05 09:14:31 +01:00
Sybren A. Stüvel
441079aa06 OAPI: rename SocketIOxxx types to Eventxxx types
Events were previously only sent via SocketIO, but now they can also be
sent via MQTT. These are now renamed from `SocketIO…` to `Event…`.

There is still the SocketIO subscription mechanism, for which the types are
still prefixed with `SocketIO`. MQTT manages subscription on the broker,
not on Flamenco Manager itself, for now this will remain SocketIO-only
functionality.
2024-02-05 09:11:13 +01:00
Sybren A. Stüvel
f464aea137 Manager & website: provide more helpful info when Worker auth fails
Provide more useful info when a Worker tries to communicate but fails
the authentication check. The message about this is now more friendly
and links to a new FAQ entry at
https://flamenco.blender.org/faq/#what-does-unknown-worker-is-trying-to-communicate-mean
2024-01-25 14:19:24 +01:00
Sybren A. Stüvel
70faa4e225 Move URLs to the Flamenco website to constants in a dedicated package
Create a dedicated package `.../pkg/website` to contain constants for the
URLs of documentation, bug reporting, etc. That way it's easier to see
which parts of the website are being referred to from the Flamenco
binaries, and updates can happen in a central spot.

No functional changes.
2024-01-25 12:25:06 +01:00
Sybren A. Stüvel
b9e41065c1 OAPI: regenerate code 2023-12-16 23:03:53 +01:00
Sybren A. Stüvel
aff6d41a91 OAPI: Add endpoint for mass job deletion
This endpoint will not actually delete the jobs, but just mark them for
deletion. Flamenco Manager will process the deletions in the background.
2023-12-16 22:59:28 +01:00
Sybren A. Stüvel
ff2d8f649f OpenAPI: update job submission example
Update the job submission example YAML to reflect the current state of the
`simple-blender-render` job compiler script.

No functional changes.
2023-12-16 13:29:12 +01:00
Sybren A. Stüvel
aa4d3cff75 Windows: check user's system privileges when Shaman is enabled
Symbolic links on Windows require some special user privilege, and
Shaman can now check for this at startup. Hopefully this helps in guiding
people towards a working Shaman system.
2023-10-16 21:36:16 +02:00
Sybren A. Stüvel
40f76efec9 Shaman: log a warning on platforms that don't reliably support symlinks
Windows 10 Home does not support symlinks, and Shaman will cause errors
unless running as admin (which is not recommended for security reasons).
Now Flamenco Manager will log a warning when Shaman is enabled on this
platform.
2023-10-15 14:37:15 +02:00
Sybren A. Stüvel
d60af809be Shaman: skip certain tests if the platform cannot do symlinking reliably
Windows 10 Home ("Core") only allows symlinking when running as admin,
which is not recommended for Flamenco Manager. Instead of failing unit
tests on this system, simply skip them. This reduces noise when developing
on this platform (i.e. my personal laptop) a lot.
2023-10-15 14:26:32 +02:00
Sybren A. Stüvel
12bc182857 Log more OS info at startup
In addition to logging `GOOS` and `GOARCH`, also log more info about the
system:

- Windows: the Windows version and edition.
- Linux: distribution, distribution version, and kernel version.
- macOS: just "macOS", until we know more about getting info there too.
2023-10-15 14:16:56 +02:00
Sybren A. Stüvel
621f050a68 OAPI: regenerate code 2023-08-23 13:54:02 +00:00
Sybren A. Stüvel
56081e5572 OAPI: SocketIO broadcasting for worker tags CUD operations 2023-08-23 13:54:02 +00:00
Sybren A. Stüvel
1eb7764d00 OAPI: regenerate code 2023-08-14 16:00:06 +02:00
Sybren A. Stüvel
7cfc2aa9e7 OAPI: Restartable workers
Extend the data model so that Workers can mark themselves as 'restartable'
at sign-on.
2023-08-14 14:57:00 +02:00
Sybren A. Stüvel
02fac6a4df Change Go package name from git.blender.org to projects.blender.org
Change the package base name of the Go code, from
`git.blender.org/flamenco` to `projects.blender.org/studio/flamenco`.

The old location, `git.blender.org`, has no longer been use since the
[migration to Gitea][1]. The new package names now reflect the actual
location where Flamenco is hosted.

[1]: https://code.blender.org/2023/02/new-blender-development-infrastructure/
2023-08-01 12:42:31 +02:00
Sybren A. Stüvel
dae5b1a571 Fix #104237: fix issue with drive-only paths on Windows
Fix an issue where a shared storage path on Linux, that maps via two-way
variables to a drive root on Windows, caused problems with the path
translation system.

Windows paths that consist only of a drive letter (`F:`) cannot just be
concatenated to a relative path, as that will result in `F:path\to\file`,
which is still a relative path of sorts. This is now handled correctly,
and should result in `F:\path\to\file`.

This fixes #104237.
2023-07-31 15:28:07 +02:00
Sybren A. Stüvel
4b53c06467 OAPI: regenerate code 2023-07-13 15:00:34 +02:00
Sybren A. Stüvel
6686ba3610 OAPI: yet another refactor of 'eval-on-submit' behavior for job settings
The data is still the same, but the names of the properties have changed
a bit so that they're more generic, declarative, instead of specific to
one bit of functionality.

The goal is to have the `evalInfo.description` field usable for the
'evaluate now' button in the add-on as well. That way it should be
clearer what that does.

This commit just updates the OpenAPI definition.
2023-07-13 14:59:22 +02:00