66 Commits

Author SHA1 Message Date
Sybren A. Stüvel
ff2d8f649f OpenAPI: update job submission example
Update the job submission example YAML to reflect the current state of the
`simple-blender-render` job compiler script.

No functional changes.
2023-12-16 13:29:12 +01:00
Sybren A. Stüvel
56081e5572 OAPI: SocketIO broadcasting for worker tags CUD operations 2023-08-23 13:54:02 +00:00
Sybren A. Stüvel
7cfc2aa9e7 OAPI: Restartable workers
Extend the data model so that Workers can mark themselves as 'restartable'
at sign-on.
2023-08-14 14:57:00 +02:00
Sybren A. Stüvel
6686ba3610 OAPI: yet another refactor of 'eval-on-submit' behavior for job settings
The data is still the same, but the names of the properties have changed
a bit so that they're more generic, declarative, instead of specific to
one bit of functionality.

The goal is to have the `evalInfo.description` field usable for the
'evaluate now' button in the add-on as well. That way it should be
clearer what that does.

This commit just updates the OpenAPI definition.
2023-07-13 14:59:22 +02:00
Sybren A. Stüvel
886f21e958 OAPI: refactor how 'eval-on-submit' settings are configured
Refactor the job settings. The `autoevalLockable` boolean is now
replaced with an `evalOnSubmit` nested object:

```
  evalOnSubmit: {
    showButton: true,
    placeholder: "Scene frame range",
  },
```

This makes it possible to add a placeholder text, and later maybe some
other parameters. The `showButton: true` part always has to be there, as
the entire feature is disabled with `showButton: false`, in which case
it's better to just remove the entire `evalOnSubmit` sub-object
altogether. Still, I think it's preferred to have that `showButton:
true` in there, as it makes it more explicit what this section of the
settings is for.

This commit just contains the OpenAPI definition.
2023-07-13 14:39:29 +02:00
Sybren A. Stüvel
553fffa21e OAPI: add option to make a job setting auto-evaluatable by the user
Add a new job setting option `autoevalLockable`. Setting this to `true` in
the job compiler's `JOB_TYPE` settings has the following effect:

- By default, the setting will not be editable in Blender's job submission
  interface. Instead, a toggle button with a 'car' icon will be shown.
- When the 'car' button is toggled off, the setting becomes editable again.

In its default, uneditable state, the setting will be auto-evaluated before
submission.

This makes it possible to 'lock in' auto-evaluation. The main use case is
for the frame range of the render job. By default this will be locked to
the scene frame range, but it can still be overridden if a different
range is wanted.

This commit just contains the necessary OpenAPI change.
2023-07-13 12:00:29 +02:00
Eveline Anderson
df51561640 OAPI: rename worker 'cluster' to 'tag'
This implements the OpenAPI changes necessary for the rename.

See #104204

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104223
2023-07-10 11:08:39 +02:00
Sybren A. Stüvel
c6dcf6d7bd OAPI: better reporting of version info
Make it explicit that the `version` property is for human consumption.
Also add a new `git` property so that all info from `version` is also
included in separate fields for machine consumption.
2023-07-06 12:18:26 +02:00
Adi Sage
b56a610f8d OAPI: Remove obsolete job statuses
Remove the following statuses from `flamenco-openapi.yaml`:

- 'construction-failed'
- 'archiving'
- 'archived'

These were a leftover from Flamenco v2 and have never been used in
Flamenco v3.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104215
2023-05-26 11:25:35 +02:00
Sybren A. Stüvel
0e6b6a6013 API: allow worker cluster creation without explicit UUID
Clusters can be created without UUID now. In that case, a random one will
be generated. The cluster will be returned by the creation call, so that
the caller can know that generated UUID.
2023-04-04 13:18:24 +02:00
Sybren A. Stüvel
cbbf7b01d6 API: add worker clusters to the API
Worker Clusters can be managed via the API, workers can be assigned to
any number of clusters (if not assigned to any, they'll pick up any task).

Jobs can be submitted with a cluster ID, in which case only workers that
are in that cluster or are clusterless will pick up its tasks.
2023-04-04 12:16:54 +02:00
Sybren A. Stüvel
92ac1d99de OAPI: add job-deletion-related operation & schema extensions
Add a "what-would-delete-do" operation, to query the Manager about what
the deletion of a specific job would entail. For some jobs the job files
will also be deleted (if they were created with a new enough Flamenco),
otherwise they will remain untouched.

Also expand the `SocketIOJobUpdate` schema to include info about job
deletion.
2023-02-03 16:41:50 +01:00
Sybren A. Stüvel
4b489dbde7 OAPI: add delete_requested_at in Job schema
If job deletion was requested, this field will contain the timestamp that
indicates when the request came in.
2023-01-08 13:45:57 +01:00
Sybren A. Stüvel
23afbfbea2 OAPI: add operation deleteJob 2023-01-04 01:18:21 +01:00
Sybren A. Stüvel
014c0d3b3d OAPI: add field to Job schemas to record storage info
Add fields to the job schemas (`SubmittedJob` and `Job`) to allow
storing the shaman checkout ID (so the Shaman checkout can be deleted
along with the job later).
2023-01-04 01:18:21 +01:00
Sybren A. Stüvel
cdd304ffdb OAPI: add endpoint for a pre-flight job check
Add an endpoint that mimicks the job submission endpoint, to see whether
the job survives the job compiler script. This can be used to fail early,
before actually sending files to the farm.
2022-10-20 13:13:03 +02:00
Sybren A. Stüvel
07f0b38e8a OAPI: add operation to set job priority 2022-09-30 16:27:14 +02:00
Sybren A. Stüvel
48a1c06277 OAPI: add operation to get the shared storage, for current platform
Add an operation `getSharedStorage` that can return the shared storage
location, adjusted for the given audience & platform. This uses the
two-way variables system to adjust the Manager's configuration.
2022-08-31 11:38:14 +02:00
Sybren A. Stüvel
f7366c87ce OAPI: include "short version" in the FlamencoVersion schema
Include a `shortversion` property in the `FlamencoVersion` schema, which
will just be the version number with the release phase (and not the git
hash, the number of commits since the last tag, and the `-dirty` suffix).
2022-08-31 08:58:05 +02:00
Sybren A. Stüvel
99bf0a67a7 OAPI: add operation to delete workers
This will remove a worker by soft-deletion. Any task still assigned to
the worker will be requeued.

Note that this removal should only happen when the worker is offline, or
it will cause errors on the worker as its credentials will not be
accepted any more.
2022-08-11 16:50:05 -07:00
Sybren A. Stüvel
e6f76f520b OAPI: include worker name in blocklist entries
To show the blocklist on the web interface, it's nice to identify workers
by their name and not just their UUID.
2022-08-01 18:02:21 +02:00
Sybren A. Stüvel
a6c935a634 Fix T99421: Introducing an etag for job types
The etag prevents job submissions with old settings, when the job
compiler script has been edited. The etag is the SHA1 hash of the
`JOB_TYPE` dictionary (as defined by the JavaScript file). The hash is
computed in a way that's independent of the exact formatting in the
JavaScript file. Also the actual JS code itself is irrelevant, just the
`JOB_TYPE` dictionary is used.
2022-07-29 21:13:37 +02:00
Sybren A. Stüvel
ed1a968691 OAPI: include "task" in fetchWorker response
Include the worker's current/last task in the response of the
`fetchWorker` operation.
2022-07-26 10:27:57 +02:00
Francesco Siddi
3c22becc18 OAPI: rename wizard to setup-assistant
No functional changes, just a rename.
2022-07-25 17:14:21 +02:00
Sybren A. Stüvel
4c1a21f4a0 OAPI: when submitting a job, include the submitter's platform
The submitter's platform is used to perform two-way variable
replacement. The variables of that submitter's platform are looked up,
and their values are replaced with the variable names. This only applies
to the job's settings and metadata, and is only performed on prefixes.

For example, if the submitter's platform has a variable
`render = /shared/frames`, a job setting
`output = "/shared/frames/shot123"` will be stored as
`output = "{render}/shot123".

When a Worker gets a task of this job, `{render}` will be expanded to
the value appropriate for their platform, hence the "two-way" name.
2022-07-22 11:53:08 +02:00
Sybren A. Stüvel
b6c6eee251 OAPI: add examples to some schemas
Add examples to the `WorkerSignOn` and `WorkerStateChanged` schemas.
These will make it easier to test with SwaggerUI, as they reflect a worker
signing on with the default task types.
2022-07-19 16:52:38 +02:00
Sybren A. Stüvel
f60849189a OAPI: Clean up FlamencoVersion schema definition
Change the `FlamencoVersion` schema definition so that it follows the style
of the other schema definitions:

- List properties before mentioning which are required.
- Put quotes around the property names, so that they stand out from the
  other YAML keys.
2022-07-18 17:47:57 +02:00
Sybren A. Stüvel
f23ba6d53e OAPI: correction to the sleep schedule example
The `days_of_week` field must be space-separated, not comma-separated.
2022-07-17 17:25:05 +02:00
Sybren A. Stüvel
26f92503cf OAPI: operations for getting & setting worker sleep schedule
Manifest Task: T99397
2022-07-16 15:57:41 +02:00
Sybren A. Stüvel
d170cda3ae OAPI: change task log fetching from providing the log to providing metadata
The task log API endpoint was loading the entire log into RAM, then sending
it as response. This makes display in a browser also a bit harder.

The API endpoint now returns some JSON with info about the task log,
including its size and which URL can be used to download it.

Manifest task: T99730
2022-07-16 12:52:23 +02:00
Sybren A. Stüvel
7521bff1a7 OAPI: add operation to read the entire task log
Manifest task: T99730
2022-07-16 11:05:58 +02:00
Sybren A. Stüvel
f2438a89a7 OAPI: operation to save config from First-Time Wizard 2022-07-14 17:25:37 +02:00
Sybren A. Stüvel
186dd06401 OAPI: endpoints for finding & checking Blender executables
These will be used in the first-time wizard.
2022-07-14 12:19:59 +02:00
Sybren A. Stüvel
597e9fd5d4 OAPI: add operations for the First Time Wizard
Manifest task: T99437
2022-07-14 11:16:30 +02:00
Sybren A. Stüvel
f9acb3f413 OAPI: add operation for getting variables from the Manager
This operation takes the platform & audience, and returns the variables
that are applicable for that combination. This can then be used by the
web frontend or by the Blender add-on to do variable replacement.
2022-07-08 11:33:41 +02:00
Sybren A. Stüvel
b34fee514a OAPI: include worker's "last seen" timestamp
Include the worker's "last seen" timestamp in the worker summary, details,
and SocketIO updates.

This is different from the "updated" timestamp, which indicates the last
time the worker entry in the database was updated (which could also be
in response to a status change request from the web interface).
2022-07-04 12:47:25 +02:00
Sybren A. Stüvel
9daecf2b45 OAPI: version all API URLs with /v3/
Change all API URL paths from `/api/…` to `/api/v3/…`. This ensures that
old Flamenco v2 clients/workers stop working when the new Manager is in
place. This also allows redirecting API calls based on the URL, also for
future upgrades.
2022-07-04 12:11:58 +02:00
Sybren A. Stüvel
30f7df5259 OAPI: change Shaman URL paths from /shaman/… to /api/shaman/…
This makes all OpenAPI URLs start with `/api/`.
2022-07-04 12:09:19 +02:00
Sybren A. Stüvel
60f3f5482d OAPI: add operation & SocketIO subscription for global last-rendered img
Add an operation to get the last-rendered image from whatever job received
such output from a Worker, and add a SocketIO subscription type to receive
updates about all jobs' last-rendered images.

These are necessary for a global last-rendered image display.
2022-07-01 12:31:18 +02:00
Sybren A. Stüvel
cca8155894 OAPI: add 204 as "nothing rendered" response to fetchJobLastRenderedInfo
Add a "204 No Content" response to the `fetchJobLastRenderedInfo` operation,
indicating that there is no rendered output yet.
2022-06-30 19:18:52 +02:00
Sybren A. Stüvel
c9500b85c2 OAPI: add SocketIOLastRenderedUpdate for broadcasting last-rendered imgs
`SocketIOLastRenderedUpdate` objects are meant to be broadcast to SocketIO
clients (i.e. the web interface). They are sent to the job-specific room,
just like task updates.
2022-06-30 18:32:42 +02:00
Sybren A. Stüvel
5d30459359 OAPI: add endpoint to get URLs of last-rendered images of a job 2022-06-28 17:05:52 +02:00
Sybren A. Stüvel
a084f6e2a4 OAPI: add operations for fetching blocklist & removing individual items 2022-06-27 11:30:37 +02:00
Sybren A. Stüvel
4e9950a143 OAPI: add endpoint for sending Last Rendered Output to Manager 2022-06-24 16:38:49 +02:00
Sybren A. Stüvel
aaed1e0589 OAPI: include task failure list in Task schema
Include the list of workers who failed this task in the `Task` schema.
2022-06-17 11:37:28 +02:00
Sybren A. Stüvel
55676b000e OAPI: change worker 'nickname' to just 'name'
There was no need to have the extra four letters 'nick', and some parts
of the code were already using just 'name' for the workers. This simplifies
and unifies things.
2022-06-16 11:01:27 +02:00
Sybren A. Stüvel
f020582bf7 OpenAPI: include last_touched in Task schema
Include the timestamp of when a Worker last touched the task in the OpenAPI
`Task` schema.
2022-06-09 11:59:01 +02:00
Sybren A. Stüvel
31e128d7f5 OAPI: remove shutdown worker state
Remove the `shutdown` worker state. It was only used in Flamenco 2 to
signal to the worker that it should go offline. In other words, it didn't
use the target state in the request.

As of this change, just send `offline` as the requested state for the
worker if you want it to go offline.
2022-06-02 16:19:26 +02:00
Sybren A. Stüvel
0d50a7eae5 OAPI: clean, remove unnecessary quotes 2022-06-02 12:16:14 +02:00
Sybren A. Stüvel
169aa3ddc3 OAPI: extract worker status request fields into their own struct
Instead of always passing `status_requested` and `lazy_status_request`
in tandem, and requiring one when the other is present, they are now
always passed as a struct `WorkerStatusChangeRequest`. This struct has
both fields required.

`WorkerStatusChangeRequest` already existed, and is now altered by changing
its field `status_requested` to simply `status` -- the fact that it's
requested is already clear from the context.
2022-06-02 12:16:03 +02:00