615 Commits

Author SHA1 Message Date
Sybren A. Stüvel
9d9ff77ee0 OAPI: add mass-task-refresh option to SocketIO job updates
Add `JobUpdate.refresh_tasks` property, to signal to SocketIO clients that
the job update is so big, all the job's tasks should be refreshed.

This is much faster than sending individual task updates one by one.
2022-05-17 14:47:33 +02:00
Sybren A. Stüvel
163611ff19 Web: add filter bar for job status 2022-05-16 18:15:51 +02:00
Sybren A. Stüvel
9fd4d55fdb Web: move status indicator code into its own function
Status indicators are used in Tabulator cells, and it's unknown whether
we can use Vue components there. Moving the code to a central place makes
it a bit easier to reuse the function in various places.
2022-05-16 17:31:30 +02:00
Sybren A. Stüvel
32737ef17b Web: cleanup, add some trailing commas and remove unused code
No functional changes.
2022-05-16 15:36:39 +02:00
Sybren A. Stüvel
8f27ea9714 Web: move Tabulator options into mounted() function
Move the Tabulator options from the `data()` function to the `mounted()`
function. This allows upcoming changes to refer to the Vue component.
2022-05-16 15:36:20 +02:00
Sybren A. Stüvel
eb0bfe820b Web: allow filtering jobs by their status
This is a very simple approach, where clicking on a job's status dot
toggles filtering by that status. There is no visual indication that this
filtering is active.

The Tabulator support for filtering from the headers is a bit buggy, see
https://github.com/olifolkerd/tabulator/issues/3745 -- I'm getting this
error even without `multiselect: true`.
2022-05-16 15:02:38 +02:00
Sybren A. Stüvel
edce178c34 Web: upgrade Tabulator 5.1.8 → 5.2.4 2022-05-16 14:34:56 +02:00
Sybren A. Stüvel
e72ea6b310 make clean: no longer regenerate source
Regenerating code with `make clean` was somewhat useful when only Go code
was generated, because that didn't include the version number / git hash.
The Python and JS code do have that version/hash, so `make clean` became
a source-altering operation, which wasn't nice.
2022-05-16 10:10:48 +02:00
Sybren A. Stüvel
88929db582 make clean: also remove race files
Also remove `flamenco-manager_race` and `flamenco-worker_race` when running
`make clean`.
2022-05-16 10:08:37 +02:00
Pablo Vazquez
fd38ad2aea Web: Use row background color for Tabulator tables 2022-05-12 15:53:57 +02:00
Pablo Vazquez
ae13ff6410 Web: Remove custom styling for tasks-list-container
Style comes from Tabulator, no need to set properties again.
2022-05-12 15:53:57 +02:00
Pablo Vazquez
d0d40c8e46 Web: Set a fixed height for Tabulator tables.
It's ugly but couldn't find a way to make it responsive and not break
the Virtual DOM.
2022-05-12 15:53:57 +02:00
Sybren A. Stüvel
abed318500 Web: remove Vue logo 2022-05-12 15:11:20 +02:00
Sybren A. Stüvel
ecbd0acb30 FEATURES.md: update for current state of the project 2022-05-12 15:11:14 +02:00
Sybren A. Stüvel
0b39f229a1 Implement may-I-keep-running protocol
Worker and Manager implementation of the "may-I-kee-running" protocol.

While running tasks, the Worker will ask the Manager periodically
whether it's still allowed to keep running that task. This allows the
Manager to abort commands on Workers when:

- the Worker should go to another state (typically 'asleep' or
  'shutdown'),
- the task changed status from 'active' to something non-runnable
  (typically 'canceled' when the job as a whole is canceled).
- the task has been assigned to a different Worker. This can happen when
  a Worker loses its connection to its Manager, resulting in a task
  timeout (not yet implemented) after which the task can be assigned to
  another Worker. If then the connectivity is restored, the first Worker
  should abort (last-assigned Worker wins).
2022-05-12 15:06:05 +02:00
Sybren A. Stüvel
fd16f7939e OAPI: regenerate code 2022-05-12 15:06:05 +02:00
Sybren A. Stüvel
52e9db6ef4 OAPI: add may-worker-run endpoint
Add an endpoint for Workers to poll while they're running a task, to see
if they're allowed to keep running it.
2022-05-12 15:05:59 +02:00
Sybren A. Stüvel
bedf10e435 Worker: clarify message when sleep command is aborted
Instead of logging "sleep aborted", the message is now "sleep command
aborted", to make it clear that it's about the sleep command, and not the
"asleep" worker state.
2022-05-12 14:59:10 +02:00
Sybren A. Stüvel
d35ca9d98f Manager: limit database connections
Limit the database connection pool to only a single connection. I hope that
this will solve the intermittent `SQLITE_BUSY` errors I've been seeing.
2022-05-12 13:58:15 +02:00
Sybren A. Stüvel
4bdaeb73a6 Web: set fixed height for the task table
It looks rather ugly, and this should be addressed, but without the
`height` option of Tabulator it won't be using the Virtual DOM and result
in very slow browser performance.
2022-05-12 13:53:17 +02:00
Sybren A. Stüvel
3d606a3fa0 Manager: task scheduler, fix handling of worker assignment of tasks
Improve how the task scheduler deals with tasks that already have a
worker assigned to them:

- When a Worker asks for a task, and there is already an active task
  assigned to it, always return that task.
- Otherwise, never allow scheduling of active tasks, as those are
  already being run by another worker. If this is not the case, their
  status should change to queued/failed, instead of handling the
  situation in the task scheduler.
- Apart from the assigned-and-active case above, ignore task's worker ID
  when scheduling tasks. If the status is 'queued' or 'soft-failed', the
  task's worker ID just indicates who ran the task last.
2022-05-12 13:52:16 +02:00
Sybren A. Stüvel
9dbc952c09 Worker: move wait time into variable
No functional changes.
2022-05-12 12:44:50 +02:00
Sybren A. Stüvel
41d3c27c40 FEATURES: more ideas 2022-05-12 12:35:22 +02:00
Sybren A. Stüvel
d3e2638f84 Cleanup: rename uri to dsn
"DSN" (Data Source Name) is used to indicate which database to open, and
was intermixed with "URI". This is now consistent.

No functional changes.
2022-05-12 11:08:54 +02:00
Sybren A. Stüvel
07576f3225 Web: ensure switching active jobs doesn't retain previous job data
Pinia's `$patch()` function will merge the given state with the current
state, instead of doing a replacement. As a result, going from an active
job with metadata fields `A` and `B`, to a job with metadata fields `B`
and `C` would actually have fields `A`, `B`, and `C` in the Pinia store.

This is resolved by replacing the `$patch(object)` with `$patch(function)`
and having that function replace the entire job.
2022-05-12 10:55:42 +02:00
Sybren A. Stüvel
47361cdc69 Web: show task activity in details view 2022-05-11 18:11:02 +02:00
Sybren A. Stüvel
2559238701 Web: CSS, tweak status colours for Queued, Requeued, and Paused
I didn't like how "queued", "paused", and "cancelled" had the same colour,
as they represent rather different statuses (may run in future, vs. won't
run at all).
2022-05-11 18:10:46 +02:00
Sybren A. Stüvel
c992966723 Web: CSS, use variables for "active" row colours in tables 2022-05-11 18:09:42 +02:00
Sybren A. Stüvel
b908ac75f5 Features: Add removal of doneChan from worker 2022-05-11 18:09:22 +02:00
Sybren A. Stüvel
0c8fce7b42 Web: hide notifications after a few seconds
The notifications (like "job requeued" after pushing the "requeue" button)
are now hidden after 5 seconds.
2022-05-11 15:11:17 +02:00
Sybren A. Stüvel
1f76c3bc64 Web: remove job/task ID column from table 2022-05-11 15:03:40 +02:00
Sybren A. Stüvel
cc10d3e4bb Web: also let Vue Router track the active task
This basically does the same as 63ac7287321a101c3f601eeb151be73154ef7720
but then for tasks.
2022-05-11 15:02:02 +02:00
Sybren A. Stüvel
6b9d7dba6d Web: extra safety check in Jobs Pinia store function 2022-05-11 15:00:25 +02:00
Sybren A. Stüvel
d2f5fa5ca7 Web: simplify handling of job updates 2022-05-11 15:00:05 +02:00
Sybren A. Stüvel
03b880fdb3 Web: also subscribe to current job when webapp starts 2022-05-11 14:59:14 +02:00
Sybren A. Stüvel
e9e1cd8be8 Web: show SocketIO connection status 2022-05-11 13:09:01 +02:00
Sybren A. Stüvel
63ac728732 Web: remove concept of "selected jobs" and replace with "active job"
The selection mechanism of Tabulator was getting in the way of having nice
navigation, as it would deselect (i.e. nav to "/") before selecting the
next job (i.e. nav to "/jobs/{job-id}").

The active job is now determined by the URL and thus handled by Vue Router.
Clicking on a job simply navigates to its URL, which causes the reactive
system to load & display it.

It is still intended to get job selection for "mass actions", but that's
only possible after normal navigation is working well.
2022-05-11 12:13:25 +02:00
Sybren A. Stüvel
af39414a06 Web: introduce VueRouter and split up into more components
Most of the code moved from `App.vue` to `views/JobsView.vue`.
Notification bar has its own component, and there are placeholder
"views" for Workers and Settings pages.

There is still some clunky handling of updates via SocketIO, as those
are a mix of job-specific and global (like SocketIO reconnection
events). The advantage of the current approach is that SocketIO
connections are closed when you leave the Jobs page, and reopened when
you enter the Workers page. My gut feeling says this is nice because it
ensures that all SocketIO connection-specific things are cleaned up when
you navigate.
2022-05-06 16:50:20 +02:00
Sybren A. Stüvel
d673da7a0c Manager: check for stuck jobs at startup
Check for jobs in 'cancel-requested' or 'requeued' statuses, and ensure
they transition to the right status. This happens at startup, before
even starting the web interface, so that a consistent state is presented.
2022-05-06 16:07:27 +02:00
Sybren A. Stüvel
cbf07f1142 Web: remove CSS for removed job statuses 2022-05-06 15:10:40 +02:00
Sybren A. Stüvel
24755e32b6 OAPI: regenerate code 2022-05-06 15:10:31 +02:00
Sybren A. Stüvel
c847929469 OAPI: remove job statuses fail-requested and waiting-for-files
These statuses are no longer needed, as they are a relic from Flamenco 2.
2022-05-06 15:09:01 +02:00
Sybren A. Stüvel
d008991bf4 Revert "Manager: broadcast job/task updates in a separate goroutine"
This reverts commit cd28ef552e2476dda68ba671436b805d7b32a655, as it
breaks the unit tests and I don't want to spend the time to fix those.
2022-05-06 14:48:16 +02:00
Sybren A. Stüvel
98da20f1a9 Manager: vacuum the database at startup 2022-05-06 14:35:34 +02:00
Sybren A. Stüvel
1fc71ccf92 Manager: reduce log level 2022-05-06 14:35:27 +02:00
Sybren A. Stüvel
72f1131e1c Manager: set HTTP Read Header timeout
Only the `ReadHeaderTimeout` is set. `ReadTimeout` is not set, as this is
quite specific per request. Shaman file uploads and websocket connections
should be allowed to run quite long, whereas other queries should be
relatively short.
2022-05-06 14:30:16 +02:00
Sybren A. Stüvel
cd28ef552e Manager: broadcast job/task updates in a separate goroutine 2022-05-06 12:27:10 +02:00
Sybren A. Stüvel
cfab4f5b76 Web: use title case in status indicator 2022-05-06 12:14:28 +02:00
Sybren A. Stüvel
1cf3cb3344 Web: remove check on tabulator.initialized
This check is no longer necessary now we wait with fetching the tasks/jobs
until the `tableBuilt` event is received.
2022-05-06 11:59:52 +02:00
Sybren A. Stüvel
7bda798009 Web: remove debug console.log() calls 2022-05-06 11:59:11 +02:00