282 Commits

Author SHA1 Message Date
Sybren A. Stüvel
992fc38604 OAPI: add endpoint for fetching the tasks of a job
Add `fetchJobTasks` operation to the Jobs API. This returns a summary of
each of the job's tasks, suitable for display in a task list view.

The actually used fields may need tweaking once we actually have a task
list view, but at least the functionality is there.
2022-04-22 12:52:57 +02:00
Sybren A. Stüvel
634d59e622 Use VSCode + Prettier to reformat YAML
This commit includes settings for Visual Studio Code, so that at least
different people with the same editor (or me on multiple machines) get the
same formatting.

No functional changes.
2022-04-22 12:01:09 +02:00
Sybren A. Stüvel
5466f65225 OAPI: add setJobStatus operation
Add API endpoint for updating the job status.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
b699647ed4 OpenAPI: add activity field to Job schema 2022-04-21 12:40:25 +02:00
Sybren A. Stüvel
d099a31531 OAPI: add endpoint for getting a single job type
This will be used by the web frontend to determine which job settings
to show and which to hide.
2022-04-15 16:21:48 +02:00
Sybren A. Stüvel
70584ce951 OAPI: add sections to OpenAPI paths
This just makes the grouping a bit more obvious. It follows the already
available tags.
2022-04-15 15:11:29 +02:00
Sybren A. Stüvel
e672289a11 OAPI: include all info for the jobs table in the JobUpdate schema
By having all info for the jobs table in the `JobUpdate` schema, it won't
have to query for the full job when a new job is added. This fetching of
the full job is now delayed until someone clicks on the table row.
2022-04-14 09:41:04 +02:00
Sybren A. Stüvel
863ecee954 OAPI: regenerate OpenAPI code 2022-04-08 12:04:34 +02:00
Sybren A. Stüvel
930d7497d7 OAPI: Better 'SQLITE_BUSY' error handling
SQLite can return `SQLITE_BUSY` errors when it's doing too many things at
the same time. This is now improved a bit by setting a 5-second timeout,
during which the SQLite driver will wait for the database to become
available. If that doesn't happen, Flamenco Manager will return a
`503 Service Unavailable` response so that the client knows to back off
a little.
2022-04-08 12:02:30 +02:00
Sybren A. Stüvel
df3f7b44b9 Hook up web interface to job updates 2022-04-07 18:46:09 +02:00
Sybren A. Stüvel
0c0df41f5d Job status change system for SocketIO broadcasts
Not fully tested yet.
2022-04-05 15:52:55 +02:00
Sybren A. Stüvel
781f1d936a OAPI: add jobs query endpoint 2022-04-04 18:53:19 +02:00
Sybren A. Stüvel
99852838d2 OAPI: fix job submission example
It now works with the current simple-blend-render compiler script again.
2022-04-04 18:24:50 +02:00
Sybren A. Stüvel
80ffc7da5d OAPI: use GET instead of OPTIONS to get file info from Shaman
OPTIONS is used by browser for CORS pre-flight requests.
2022-04-01 15:52:17 +02:00
Sybren A. Stüvel
5f16201832 OAPI: expose effective job storage path in configuration API 2022-04-01 14:01:22 +02:00
Sybren A. Stüvel
e70dad2177 Manager: unify Flamenco and Shaman storage paths
Flamenco Manager now has a "storage path" config option, which will be
used by Shaman if enabled. Now the `{jobs}` implicit variable will always
exist, its value depending on whether Shaman is enabled or not.
2022-04-01 13:43:51 +02:00
Sybren A. Stüvel
88fe62ddef Manager: skip shaman permission unit test on Windows
The test was made with umask on UNIX-like systems in mind. It doesn't
make much sense on Windows, at least not in its current form, so it's
better to just explicitly skip it.
2022-03-28 11:33:09 +02:00
Sybren A. Stüvel
1b11233c92 Shaman: more path -> filepath
Just more Windows fixes.
2022-03-28 11:11:50 +02:00
Sybren A. Stüvel
fe9849b134 Shaman: fix bunch of path issues on Windows
Shaman was made on Linux, using the `path` module, which only supports
forward slashes. This mostly replaces `path` with `path/filepath` to use
native paths and increase Windows compatibility.

The work isn't done yet, though.
2022-03-28 11:02:18 +02:00
Sybren A. Stüvel
a6834137b7 Tests: test some more returned errors 2022-03-26 14:07:49 +01:00
Sybren A. Stüvel
3905135465 Tests: replace assert.Nil(t, err) with assert.NoError(t, err)
This is a better way to check errors, both semantically (no usually nil is
bad, but nil error is good, so this avoids a mental negation) and for
the reporting by the assert package.
2022-03-26 14:05:53 +01:00
Sybren A. Stüvel
6c827ffc52 Shaman: only configure the root directory of the Shaman files
Flamenco v2 allowed separate configuration of the Shaman file store and
checkout paths. This is now just one setting for "the storage". The file
store will be in `{storage}/file-store` and the checkout will happen in
`{storage}/jobs`.
2022-03-25 14:32:42 +01:00
Sybren A. Stüvel
4b9501590e Shaman: only add random suffix to checkout dir if already existing
Only add a random suffix to the checkout dir if it is necessary to ensure
uniqueness. If the client-supplied checkout directory doesn't exist yet,
it will be used as-is.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
724938c7ae Shaman: allow Manager to determine the final checkout path
The checkout request now responds with the final checkout path. This makes
it possible for the Manager to ensure the checkout is unique.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
2e4ada2ade Manager: add 'shaman' prefix to logging 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
7e035ec3bf Manager: check returned error of touchfile call 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
eeeec23f43 Cleanup, fix typo in OAPI description 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
ccf45f2f6c Manager: add Shaman checkout test 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
c69e0909ae Shaman submission is working! 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
21227c8046 More Shaman integration work
- Addon switches between filesystem-packing and Shaman-packing
  automatically, depending on whether the Manager has Shaman enabled.
- Actually using BAT for Shaman packing.

It doesn't work though, some error occurs when receiving Shaman response
from the Manager in the Addon.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
f9b7510c42 Streamlining Shaman protocol further, plus rough implementation for addon 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
b2288e7f28 Manager: More work on Shaman support
This introduces some more conceptual changes to Shaman. The most important
one is that there is no longer a "checkout ID", but a "checkout path".
The Shaman client can request any subpath of the checkout directory,
so that it can handle things like project- or scene-specific prefixes.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
2b0d154a07 Manager: more work on Shaman integration 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
6f35b3303d Manager: more work on porting Shaman code to Flamenco 3 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
4df0543661 Manager: Implementation of Shaman OAPI interface
This hooks up the OpenAPI Shaman endpoints to a Shaman interface. This
interface doesn't have any implementation yet.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
4e8e71e4e2 Initial checkin of Shaman of Flamenco 2
This is not yet working, it's just a direct copy of the Manager of Flamenco
2, with Logrus replaced by Zerolog. The API has been documented in
flamenco-manager.yaml as a starting point for the integration.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
09a476e11a Compute render output path when evaluating job settings
Compute render output path when evaluating job settings, which is done
within the Flamenco add-on, instead of in the job compiler script. This
allows the UI to show the render path, rather than it only being known
after the job has been submitted.
2022-03-15 16:56:44 +01:00
Sybren A. Stüvel
a6ac331612 Allow Python evaluation for computing job settings in the addon 2022-03-14 18:44:39 +01:00
Sybren A. Stüvel
7b090bdbd6 OAPI: job submission endpoint now returns Job instead of SubmittedJob 2022-03-14 16:59:59 +01:00
Sybren A. Stüvel
76ba0a201b OAPI: remove Configuration component schema
Nothing was using the `Configuration` schema, so I removed it from the
flamenco-manager.yaml OpenAPI definition file.
2022-03-14 14:40:16 +01:00
Sybren A. Stüvel
7181c2c4cf OAPI: allow job settings to have a tooltip/description 2022-03-11 17:30:27 +01:00
Sybren A. Stüvel
282f91243b OAPI: replace placeholder description of 'jobs' tag 2022-03-10 15:45:56 +01:00
Sybren A. Stüvel
416fa9bd28 OAPI: add 'meta' tag for /api/version endpoint 2022-03-10 15:45:46 +01:00
Sybren A. Stüvel
78927c2c31 OAPI: put quotes around description with comma
The comma was interpreted as property separator, and not as part of the
'description' property value.
2022-03-10 15:41:48 +01:00
Sybren A. Stüvel
d0b677ffe3 Add API endpoint /api/version to get the Flamenco Manager version
This will be used in the auto-discovery, to check that a discovered URL can
actually be reached & points to a Flamenco Manager.
2022-03-08 13:56:43 +01:00
Sybren A. Stüvel
9f5e4cc0cc License: license all code under "GPL-3.0-or-later"
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
2022-03-07 15:26:46 +01:00
Sybren A. Stüvel
cd2fe8170e Errors: remove "error" prefix from message
Instead of returning an error "error doing X", just return "doing X". The
fact that it's returned as an error object says enough about that it's
an error.

This also makes it easier to chain error messages, without seeing the
word "error" in every part of the chain.
2022-03-04 11:30:31 +01:00
Sybren A. Stüvel
3c6d7773ef Crosspath: add ToNative() function
Add `crosspath.ToNative(path)`, which returns the path with platform-
native path separators. This is meant for use in the Worker, to convert
paths before attempting to use them.
2022-03-04 11:10:10 +01:00
Sybren A. Stüvel
70f8140df5 Crosspath: extend tests with "long UNC" paths
See https://en.wikipedia.org/wiki/Path_(computing)#Universal_Naming_Convention
2022-03-04 11:09:14 +01:00
Sybren A. Stüvel
0a97d04741 Cleanup: rewrite crosspath.Stem unit test
Use the same style of testing as the other test functions in the file.

No functional changes.
2022-03-04 10:27:31 +01:00