570 Commits

Author SHA1 Message Date
Sybren A. Stüvel
4da7f67105 OAPI: generate code 2022-05-03 13:03:59 +02:00
Sybren A. Stüvel
adba72176d OAPI: add /api/tasks/{task_id} endpoint
Add an OpenAPI endpoint definition for fetching a single task.
2022-05-03 12:45:08 +02:00
Sybren A. Stüvel
891e791853 Manager: reduce log level of socketIO subscription changes 2022-05-03 12:04:27 +02:00
Sybren A. Stüvel
50c8cd39f2 Task update notifications via SocketIO
Manager now sends out task updates via SocketIO, and the web interface
handles those.

Note that there is a `BroadcastTaskUpdate()` function, but not a
`BroadcastNewTask`. The 'new job' broadcast is sent after the job's
tasks have been created, and thus there is no need for a separate
broadcast per task.
2022-05-03 11:26:24 +02:00
Sybren A. Stüvel
222d618ef6 OAPI: generate code 2022-05-03 11:10:57 +02:00
Sybren A. Stüvel
a3752f3113 OAPI: Add SocketIOTaskUpdate schema
Add the schema for task updates sent via SocketIO.
2022-05-03 11:09:47 +02:00
Sybren A. Stüvel
782c60a6e7 OAPI: remove '-dirty' suffix from the version number in generated code
The version number used in the OpenAPI-generated code shouldn't contain
the `-dirty` suffix. In the common development workflow, those files
will always be dirty; because they're only committed after things are
locally working, the implementation has already been written.
2022-05-03 11:09:47 +02:00
Sybren A. Stüvel
bb68488c5e Cleanup: Manager, add bit of documentation 2022-05-03 10:39:44 +02:00
Sybren A. Stüvel
9b330280b7 Add SocketIO subscription system for job-related updates
SocketIO clients can now send a message with `/subscription` event type
in order to subscribe to or unsubscribe from job-related updates.

These job-related updates themselves aren't sent yet, so this is a change
that's impossible to really test. The socketIO code for joining/leaving
rooms is called, though.
2022-05-02 18:36:14 +02:00
Sybren A. Stüvel
824425f466 OAPI: regenerate code 2022-05-02 18:31:29 +02:00
Sybren A. Stüvel
24941142b0 OAPI: add SocketIOSubscription type for managing SocketIO rooms
A client can send a `SocketIOSubscription` object with the
`/subscription` event type, in order to manage the subscriptions to
job-related updates.

Subscribing to a job means receiving updates to its tasks. General job
updates are sent to all clients anyway.

This commit only contains the OpenAPI YAML change, to see if it works
well to separate such commits into three parts:

    - OpenAPI definition change.
    - Regenerated code changes.
    - Hand-written code changes.
2022-05-02 18:29:35 +02:00
Sybren A. Stüvel
d6c257d396 Include .editorconfig file
This is an attempt to get more cross-editor compatibility.
2022-05-02 18:25:40 +02:00
Sybren A. Stüvel
8d69bfe069 Cleanup: Manager, reorganise the socketio code a bit 2022-04-29 16:58:48 +02:00
Sybren A. Stüvel
c07d2e3249 Web: tweak webapp layout 2022-04-29 13:11:44 +02:00
Sybren A. Stüvel
4ebf4f31f9 Web: add task table 2022-04-29 13:11:19 +02:00
Sybren A. Stüvel
03d6f99c3d Makefile: include getting webapp dependencies with yarn install
Running `make` will now also run `yarn install` for the webapp.
2022-04-29 12:30:02 +02:00
Sybren A. Stüvel
629c073ed7 Manager: fix query for job tasks 2022-04-29 12:26:53 +02:00
Sybren A. Stüvel
cbc8600d90 FEATURES: reorder by priority and add some new things 2022-04-29 11:16:47 +02:00
Sybren A. Stüvel
cc58f0d81a Web: remove unused parts of the Vue template project
The project was created with things (components, router, views) we don't
use at the moment. To keep a clearer separation between "our code" and
"example code", I just removed the latter.
2022-04-25 17:12:52 +02:00
Sybren A. Stüvel
54587348e3 Web: remove unused import 2022-04-25 17:06:25 +02:00
Sybren A. Stüvel
992fc38604 OAPI: add endpoint for fetching the tasks of a job
Add `fetchJobTasks` operation to the Jobs API. This returns a summary of
each of the job's tasks, suitable for display in a task list view.

The actually used fields may need tweaking once we actually have a task
list view, but at least the functionality is there.
2022-04-22 12:52:57 +02:00
Sybren A. Stüvel
e399b14e66 Manager: cleanup, rename jobId to jobID
No functional changes.
2022-04-22 12:16:11 +02:00
Sybren A. Stüvel
634d59e622 Use VSCode + Prettier to reformat YAML
This commit includes settings for Visual Studio Code, so that at least
different people with the same editor (or me on multiple machines) get the
same formatting.

No functional changes.
2022-04-22 12:01:09 +02:00
Sybren A. Stüvel
0cd478a409 Manager: move FetchJob function into jobs_query.go
I want to put more of the "get stuff" code into `jobs_query.go`, keeping
`jobs.go` for creationg & manipulation.
2022-04-22 11:51:02 +02:00
Sybren A. Stüvel
fc4cc11483 Web: style buttons and footer in Blender Blue 2022-04-22 11:46:45 +02:00
Sybren A. Stüvel
4129ed11bb Web: use wrapper for the OpenAPI client to track requests
A wrapper for the generated `ApiClient` class tracks the number of running
queries. This makes it much simpler to show the "API calls pending" UI
element, regardless of which part of the webapp performs the queries.
2022-04-22 11:46:33 +02:00
Sybren A. Stüvel
fc68e8a413 Web: action button styling
Making the button respond to hovering makes it slightly easier to see
which button is enabled and which is disabled.
2022-04-22 10:33:19 +02:00
Sybren A. Stüvel
e34a0ba6ea Worker: more granular locking when flushing upstream buffer
Only lock the database mutex when actual queries are performed, but not
during the entire flush loop.
2022-04-21 19:19:01 +02:00
Sybren A. Stüvel
8937a6f06f Cleanup: worker, remove debug timers
Remove accidentally committed debug timing code.
2022-04-21 19:14:09 +02:00
Sybren A. Stüvel
bcbacf6c42 Worker: fix race condition getting logger with worker status 2022-04-21 19:12:53 +02:00
Sybren A. Stüvel
6bdc198301 Manager: more graceful errors when receiving task update of unknown task
Return a 404 Not Found when the task can't be found, and a 500 on other
errors.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
1176d85496 Worker: add -flush CLI option to just flush the buffer and quit
Add `-flush` CLI option to just flush the upstream queue and then quit.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
55d264632f Worker: completely flush upstream buffer at startup
Completely flush the upstream buffer at startup, before attempting to
fetch a new task. These updates could impact any task on the Manager side,
and first flushing the buffer before appending new updates also seems
like a good idea.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
90be370095 Manager: reduce password strength of Workers
The password check of worker API calls was 2 orders of magnitude slower
than actually handling the API call itself. Since the Worker authentication
is not that important (it's all on the same network anyway, and Worker
account registration is automatic too), lowering the BCrypt cost to the
minimum helps.

On my machine, this reduces the time for password checks from 50 to 2 ms.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
65427ee38e Manager: use e.NoContent(http.StatusNoContent) to return "no content"
No functional changes, just the right call for the job.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
d0e96ad7bd Web: hide the "delete job" action button
Hide the "Delete job" action button, as it has no backend implementation
yet.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
79bac3a5f3 Manager: fix race condition in logging of worker properties
Dereferencing the `w *persistence.Worker` pointer should happen directly
in the function call, not in the zerolog callback function.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
bb1acef054 Web: style action buttons and move the delete button to the last place 2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
eeb8305de0 Web: add button to requeue jobs 2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
698db0bf7c Web: Cleanup, abstract some code away into separate functions
No functional changes, just preparation for adding similar job actions.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
2e2184df62 Web: allow cancelling a job also after it failed
For task scheduling this doesn't matter, but for human workflow it can be
useful to differentiate between "failed" (and should be retried) and
"cancelled" (no longer relevant).
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
1ac37ab625 Web: clearer style for focused action button 2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
d84efb8c9a Web: implement 'Cancel Job' button
The button calls the 'setJobStatus' API endpoint to set the job status to
`cancel-requested`.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
1331d29b10 Makefile: mark new race-checker-enabled targets as 'phony' 2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
5a5a9aec8b Makefile: add rules for builds with race condition checker enabled
New rules `flamenco-manager_race` and `flamenco-worker_race` will build
with the race condition checker enabled.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
5466f65225 OAPI: add setJobStatus operation
Add API endpoint for updating the job status.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
b699647ed4 OpenAPI: add activity field to Job schema 2022-04-21 12:40:25 +02:00
Sybren A. Stüvel
d79fde17f3 Manager: keep track of the reason of job status changes
To prepare for job status changes being requestable from the API, store
the reason for any status change on the job itself.

Not yet part of the API, just on the persistence layer.
2022-04-21 12:32:07 +02:00
Sybren A. Stüvel
954af37fd5 Manager: rename assertXXXResponse to assertResponseXXX
Rename test functions like `assertJSONResponse` to `assertResponseJSON`,
so that they get ordered together by autocompletion.

No functional changes.
2022-04-21 12:01:46 +02:00
Sybren A. Stüvel
c3b694ab2a Manager: wrap job/task errors in persistence layer
Avoid users of the persistence layer to have to test against Gorm errors,
by wrapping job/task errors in a new `PersistenceError` struct.

Instead of testing for `gorm.ErrRecordNotFound`, code can now test for
`persistence.ErrJobNotFound` or `persistence.ErrTaskNotFound`.
2022-04-21 11:54:59 +02:00