260 Commits

Author SHA1 Message Date
Sybren A. Stüvel
b535dfed3c OAPI: expand the description of the TaskUpdate schema
No functional changes.
2022-05-20 10:44:27 +02:00
Sybren A. Stüvel
ce43eead9a OAPI: regenerate code 2022-05-19 17:24:59 +02:00
Sybren A. Stüvel
507985e258 OAPI: rename job status requeued to requeueing
Rename job status `requeued` to `requeueing`, because it indicates a
process that's (if things are working well) happening currently.
2022-05-19 17:22:35 +02:00
Sybren A. Stüvel
1086fcbc28 OAPI: Regenerate code 2022-05-19 16:04:10 +02:00
Sybren A. Stüvel
476f584192 OAPI: job settings, allow more granular control over setting visibility
Instead of having a per-setting boolean to control whether it's visible
or hidden, there are now four values:

- `visible`: always show.
- `submission`: only show in the UI of a job submitter (like a Blender
  add-on).
- `web`: only show in the web interface for management, but not when
  submitting the job.
- `hidden`: never show; only available to the job compiler script as
  internal setting.
2022-05-19 16:01:53 +02:00
Sybren A. Stüvel
744fabea78 OAPI: rename pkg/api/flamenco-manager.yaml to flamenco-openapi.yaml
Rename `pkg/api/flamenco-manager.yaml` to `flamenco-openapi.yaml`, to
distinguish the OpenAPI definition file from the Flamenco Manager
configuration file of the same name (but in a different directory).

No functional changes.
2022-05-19 15:22:37 +02:00
Sybren A. Stüvel
b928896066 OAPI: regenerate code 2022-05-19 15:17:19 +02:00
Sybren A. Stüvel
7753e65b03 OAPI: rename the JobUpdate schema to SocketIOJobUpdate
Rename `JobUpdate` to `SocketIOJobUpdate` to make it consistent with
`SocketIOTaskUpdate`.

Note that `TaskUpdate` exists as another schema, used by Workers to
update tasks they're performing.
2022-05-19 15:16:27 +02:00
Sybren A. Stüvel
12d5b2a1fc OAPI: regenerate code 2022-05-19 14:11:33 +02:00
Sybren A. Stüvel
6afb1151cc OAPI: add task activity to SocketIO task updates 2022-05-19 14:07:42 +02:00
Sybren A. Stüvel
cd8080fb44 OAPI: regenerate code 2022-05-17 14:48:25 +02:00
Sybren A. Stüvel
9d9ff77ee0 OAPI: add mass-task-refresh option to SocketIO job updates
Add `JobUpdate.refresh_tasks` property, to signal to SocketIO clients that
the job update is so big, all the job's tasks should be refreshed.

This is much faster than sending individual task updates one by one.
2022-05-17 14:47:33 +02:00
Sybren A. Stüvel
fd16f7939e OAPI: regenerate code 2022-05-12 15:06:05 +02:00
Sybren A. Stüvel
52e9db6ef4 OAPI: add may-worker-run endpoint
Add an endpoint for Workers to poll while they're running a task, to see
if they're allowed to keep running it.
2022-05-12 15:05:59 +02:00
Sybren A. Stüvel
24755e32b6 OAPI: regenerate code 2022-05-06 15:10:31 +02:00
Sybren A. Stüvel
c847929469 OAPI: remove job statuses fail-requested and waiting-for-files
These statuses are no longer needed, as they are a relic from Flamenco 2.
2022-05-06 15:09:01 +02:00
Sybren A. Stüvel
1fc71ccf92 Manager: reduce log level 2022-05-06 14:35:27 +02:00
Sybren A. Stüvel
23680c27bf OAPI: regenerate code 2022-05-05 16:36:38 +02:00
Sybren A. Stüvel
b838b10ead OAPI: remove task status "cancel-requested"
This task status was a remainder of the old Flamenco 2 structure, and is
no longer necessary.
2022-05-05 16:35:45 +02:00
Sybren A. Stüvel
7b1b6030d3 OAPI: regenerate code 2022-05-05 16:04:45 +02:00
Sybren A. Stüvel
9985860c66 OAPI: endpoint /api/tasks/{task_id}/setstatus for changing task status
This endpoint is for humans to manage individual tasks. Workers post task
updates to another endpoint.
2022-05-05 16:02:47 +02:00
Sybren A. Stüvel
ed2e709618 Cleanup: reformat long lines in pkg/api/flamenco-manager.yaml
I didn't find the way VSCode was rewrapping those lines the prettiest,
so I manually changed the formatting. It's now still compatible with the
auto-formatting, but nicer.

No functional changes.
2022-05-05 16:00:54 +02:00
Sybren A. Stüvel
4da7f67105 OAPI: generate code 2022-05-03 13:03:59 +02:00
Sybren A. Stüvel
adba72176d OAPI: add /api/tasks/{task_id} endpoint
Add an OpenAPI endpoint definition for fetching a single task.
2022-05-03 12:45:08 +02:00
Sybren A. Stüvel
222d618ef6 OAPI: generate code 2022-05-03 11:10:57 +02:00
Sybren A. Stüvel
a3752f3113 OAPI: Add SocketIOTaskUpdate schema
Add the schema for task updates sent via SocketIO.
2022-05-03 11:09:47 +02:00
Sybren A. Stüvel
824425f466 OAPI: regenerate code 2022-05-02 18:31:29 +02:00
Sybren A. Stüvel
24941142b0 OAPI: add SocketIOSubscription type for managing SocketIO rooms
A client can send a `SocketIOSubscription` object with the
`/subscription` event type, in order to manage the subscriptions to
job-related updates.

Subscribing to a job means receiving updates to its tasks. General job
updates are sent to all clients anyway.

This commit only contains the OpenAPI YAML change, to see if it works
well to separate such commits into three parts:

    - OpenAPI definition change.
    - Regenerated code changes.
    - Hand-written code changes.
2022-05-02 18:29:35 +02:00
Sybren A. Stüvel
992fc38604 OAPI: add endpoint for fetching the tasks of a job
Add `fetchJobTasks` operation to the Jobs API. This returns a summary of
each of the job's tasks, suitable for display in a task list view.

The actually used fields may need tweaking once we actually have a task
list view, but at least the functionality is there.
2022-04-22 12:52:57 +02:00
Sybren A. Stüvel
634d59e622 Use VSCode + Prettier to reformat YAML
This commit includes settings for Visual Studio Code, so that at least
different people with the same editor (or me on multiple machines) get the
same formatting.

No functional changes.
2022-04-22 12:01:09 +02:00
Sybren A. Stüvel
5466f65225 OAPI: add setJobStatus operation
Add API endpoint for updating the job status.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
b699647ed4 OpenAPI: add activity field to Job schema 2022-04-21 12:40:25 +02:00
Sybren A. Stüvel
d099a31531 OAPI: add endpoint for getting a single job type
This will be used by the web frontend to determine which job settings
to show and which to hide.
2022-04-15 16:21:48 +02:00
Sybren A. Stüvel
70584ce951 OAPI: add sections to OpenAPI paths
This just makes the grouping a bit more obvious. It follows the already
available tags.
2022-04-15 15:11:29 +02:00
Sybren A. Stüvel
e672289a11 OAPI: include all info for the jobs table in the JobUpdate schema
By having all info for the jobs table in the `JobUpdate` schema, it won't
have to query for the full job when a new job is added. This fetching of
the full job is now delayed until someone clicks on the table row.
2022-04-14 09:41:04 +02:00
Sybren A. Stüvel
863ecee954 OAPI: regenerate OpenAPI code 2022-04-08 12:04:34 +02:00
Sybren A. Stüvel
930d7497d7 OAPI: Better 'SQLITE_BUSY' error handling
SQLite can return `SQLITE_BUSY` errors when it's doing too many things at
the same time. This is now improved a bit by setting a 5-second timeout,
during which the SQLite driver will wait for the database to become
available. If that doesn't happen, Flamenco Manager will return a
`503 Service Unavailable` response so that the client knows to back off
a little.
2022-04-08 12:02:30 +02:00
Sybren A. Stüvel
df3f7b44b9 Hook up web interface to job updates 2022-04-07 18:46:09 +02:00
Sybren A. Stüvel
0c0df41f5d Job status change system for SocketIO broadcasts
Not fully tested yet.
2022-04-05 15:52:55 +02:00
Sybren A. Stüvel
781f1d936a OAPI: add jobs query endpoint 2022-04-04 18:53:19 +02:00
Sybren A. Stüvel
99852838d2 OAPI: fix job submission example
It now works with the current simple-blend-render compiler script again.
2022-04-04 18:24:50 +02:00
Sybren A. Stüvel
80ffc7da5d OAPI: use GET instead of OPTIONS to get file info from Shaman
OPTIONS is used by browser for CORS pre-flight requests.
2022-04-01 15:52:17 +02:00
Sybren A. Stüvel
5f16201832 OAPI: expose effective job storage path in configuration API 2022-04-01 14:01:22 +02:00
Sybren A. Stüvel
e70dad2177 Manager: unify Flamenco and Shaman storage paths
Flamenco Manager now has a "storage path" config option, which will be
used by Shaman if enabled. Now the `{jobs}` implicit variable will always
exist, its value depending on whether Shaman is enabled or not.
2022-04-01 13:43:51 +02:00
Sybren A. Stüvel
88fe62ddef Manager: skip shaman permission unit test on Windows
The test was made with umask on UNIX-like systems in mind. It doesn't
make much sense on Windows, at least not in its current form, so it's
better to just explicitly skip it.
2022-03-28 11:33:09 +02:00
Sybren A. Stüvel
1b11233c92 Shaman: more path -> filepath
Just more Windows fixes.
2022-03-28 11:11:50 +02:00
Sybren A. Stüvel
fe9849b134 Shaman: fix bunch of path issues on Windows
Shaman was made on Linux, using the `path` module, which only supports
forward slashes. This mostly replaces `path` with `path/filepath` to use
native paths and increase Windows compatibility.

The work isn't done yet, though.
2022-03-28 11:02:18 +02:00
Sybren A. Stüvel
a6834137b7 Tests: test some more returned errors 2022-03-26 14:07:49 +01:00
Sybren A. Stüvel
3905135465 Tests: replace assert.Nil(t, err) with assert.NoError(t, err)
This is a better way to check errors, both semantically (no usually nil is
bad, but nil error is good, so this avoids a mental negation) and for
the reporting by the assert package.
2022-03-26 14:05:53 +01:00
Sybren A. Stüvel
6c827ffc52 Shaman: only configure the root directory of the Shaman files
Flamenco v2 allowed separate configuration of the Shaman file store and
checkout paths. This is now just one setting for "the storage". The file
store will be in `{storage}/file-store` and the checkout will happen in
`{storage}/jobs`.
2022-03-25 14:32:42 +01:00