51 Commits

Author SHA1 Message Date
Sybren A. Stüvel
f508bd7e92 FEATURES.md: add idea for versioning API 2022-06-28 12:59:26 +02:00
Sybren A. Stüvel
e358862db7 FEATURES.md: some bookkeeping 2022-06-28 12:10:45 +02:00
Sybren A. Stüvel
4e4f67d20a FEATURES.md: some bookkeeping 2022-06-28 11:42:03 +02:00
Sybren A. Stüvel
e687c95e5d Manager: add "last rendered image" processing pipeline
Add a handler for the OpenAPI `taskOutputProduced` operation, and an
image thumbnailing goroutine.

The queue of images to process + the function to handle queued images
is managed by `last_rendered.LastRenderedProcessor`. This queue currently
simply allows 3 requests; this should be improved such that it keeps
track of the job IDs as well, as with the current approach a spammy job
can starve the updates from a more calm job.
2022-06-24 16:51:11 +02:00
Sybren A. Stüvel
27a6dde708 Manager: add local_storage package for managing storage locations
Add a `local_storage` package that finds a suitable place to put files.
Currently it just looks at the location of the currently running
executable; it can later do other things. It can be queried for directory
to put job-specific files.

It is intended to be used by the under-development "last rendered output"
processing system, to store an image file per job. Later we should also
refactor the task log handling system to use this.
2022-06-23 16:45:38 +02:00
Sybren A. Stüvel
b441f3f3de Manager: load job compiler scripts from disk as well
If there is a `scripts` directory next to the current executable, load
scripts from that directory as well.

It is still required to restart the Manager in order to pick up changes
to those scripts (including new/removed files), PLUS a refresh in the
add-on.
2022-06-21 17:59:20 +02:00
Sybren A. Stüvel
21a294a267 FEATURES.md: mark task as done 2022-06-17 16:40:19 +02:00
Sybren A. Stüvel
ec55cf6ce1 FEATURES.md: more features 2022-06-17 16:37:06 +02:00
Sybren A. Stüvel
0396919229 FEATURES: add new way in which jobs can get stuck 2022-06-17 14:59:26 +02:00
Sybren A. Stüvel
50e795c595 FEATURES.md: mark 'clear task failure list' as done 2022-06-17 11:39:57 +02:00
Sybren A. Stüvel
6e12a2fb25 Manager: keep track of which worker failed which task
When a Worker indicates a task failed, mark it as `soft-failed` until
enough workers have tried & failed at the same task.

This is the first step in a blocklisting system, where tasks of an
often-failing worker will be requeued to be retried by others.

NOTE: currently the failure list of a task is NOT reset whenever it is
requeued! This will be implemented in a future commit, and is tracked in
`FEATURES.md`.
2022-06-13 18:41:38 +02:00
Sybren A. Stüvel
b922722614 Manager: broadcast worker timeouts over SocketIO
This way the web interface will also show timed-out workers.
2022-06-13 13:05:20 +02:00
Sybren A. Stüvel
bf831aa0fd FEATURES: mark task timeout monitoring as done 2022-06-10 16:14:38 +02:00
Sybren A. Stüvel
b186ea1828 Manager: write to task log when assigning it to a worker 2022-06-09 10:59:44 +02:00
Sybren A. Stüvel
bba5c6020d FEATURES.md: mark some features as done, add some others 2022-06-03 15:52:07 +02:00
Sybren A. Stüvel
85e24d2b1c FEATURES.md: addition of some more details about the worker management 2022-05-31 17:24:11 +02:00
Sybren A. Stüvel
424e80163b FEATURES.md: reprioritise and some new tasks 2022-05-31 12:28:50 +02:00
Sybren A. Stüvel
131298f5e8 Update FEATURES.md 2022-05-30 16:01:55 +02:00
Sybren A. Stüvel
247903444f FEATURES: two updates of work done 2022-05-19 17:26:02 +02:00
Sybren A. Stüvel
7b664475ca Rename job status requeued to requeueing 2022-05-19 17:25:53 +02:00
Sybren A. Stüvel
d71aa0e5c3 FEATURES.md: various updates 2022-05-17 15:37:08 +02:00
Sybren A. Stüvel
ecbd0acb30 FEATURES.md: update for current state of the project 2022-05-12 15:11:14 +02:00
Sybren A. Stüvel
41d3c27c40 FEATURES: more ideas 2022-05-12 12:35:22 +02:00
Sybren A. Stüvel
b908ac75f5 Features: Add removal of doneChan from worker 2022-05-11 18:09:22 +02:00
Sybren A. Stüvel
f05cf0e9f8 Add another "feature" idea 2022-05-03 16:15:03 +02:00
Sybren A. Stüvel
f533ccce43 Add extra "feature" 2022-05-03 16:03:18 +02:00
Sybren A. Stüvel
cbc8600d90 FEATURES: reorder by priority and add some new things 2022-04-29 11:16:47 +02:00
Sybren A. Stüvel
5466f65225 OAPI: add setJobStatus operation
Add API endpoint for updating the job status.
2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
456a35dfc7 Update FEATURES.md
Some things are done, and added one item.
2022-04-21 10:48:59 +02:00
Sybren A. Stüvel
5f019044ce Add caching of job compilers to FEATURES.md 2022-04-15 15:29:14 +02:00
Sybren A. Stüvel
2830d1eda8 Update FEATRURES.md 2022-04-08 14:34:38 +02:00
Sybren A. Stüvel
27e954090c FEATURES: add 'clean requeue of task when shutting down' to worker list 2022-04-05 16:27:51 +02:00
Sybren A. Stüvel
b2288e7f28 Manager: More work on Shaman support
This introduces some more conceptual changes to Shaman. The most important
one is that there is no longer a "checkout ID", but a "checkout path".
The Shaman client can request any subpath of the checkout directory,
so that it can handle things like project- or scene-specific prefixes.
2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
b6a7a15582 Implement frames-to-video command
This finishes the "Simple Blender Render" job type, as it can now be run
fully by a Worker!
2022-03-17 16:18:10 +01:00
Sybren A. Stüvel
517b007ae5 Manager: improve logging of created tasks 2022-03-17 11:45:56 +01:00
Sybren A. Stüvel
5728884fed FEATURES: add issue with "add task" log entry 2022-03-17 11:25:58 +01:00
Sybren A. Stüvel
e7a5e0565d FEATURES: add new feature idea 2022-03-17 10:53:15 +01:00
Sybren A. Stüvel
d3a3543fdc Features: add more things to do 2022-03-15 19:21:40 +01:00
Sybren A. Stüvel
abe46c264e Add idea to FEATURES.md 2022-03-15 17:48:54 +01:00
Sybren A. Stüvel
04ac003ed6 Features: add new feature to prevent job submission with outdated job type 2022-03-15 10:34:27 +01:00
Sybren A. Stüvel
4196460c29 Automatically include input blend file path in job settings 2022-03-14 18:18:10 +01:00
Sybren A. Stüvel
c8ae856131 Features: add reminder to hide custom RNA options 2022-03-14 14:40:27 +01:00
Sybren A. Stüvel
a2c2f0ccc0 Add idea to FEATURES.md 2022-03-09 23:26:06 +01:00
Sybren A. Stüvel
a19aa86c4e FEATURES.md: mark UPnP/SSDP as 'done' 2022-03-08 15:04:03 +01:00
Sybren A. Stüvel
43a9e1d439 Add some nice-to-have new features to FEATURES.md 2022-03-07 11:21:31 +01:00
Sybren A. Stüvel
648a220098 Update FEATURES.md 2022-02-28 15:30:50 +01:00
Sybren A. Stüvel
41168ff68b Update FEATURES.md 2022-02-28 12:09:07 +01:00
Sybren A. Stüvel
df77d09aa6 Start of a task/job state machine
The task status change → job status change code is a direct port of the
Flamenco Server v2 code written in Python.

There is no job status change → task status changes logic yet, and the
tests are also far from complete.
2022-02-24 16:42:38 +01:00
Sybren A. Stüvel
2a4db143e6 Update FEATURES.md 2022-02-22 14:39:53 +01:00
Sybren A. Stüvel
edfcbb1e40 Add a TODO to the features list 2022-02-14 16:31:23 +01:00