blender/flamenco
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
194 Commits 1 Branch 0 Tags
Commit Graph

132 Commits

Author SHA1 Message Date
Sybren A. Stüvel
e70a44a146 Manager: switch from PostgreSQL to SQLite 
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
 2022-03-01 18:50:31 +01:00
Sybren A. Stüvel
bbc6a3f69e Tweak some logging 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
0235ffcb4a Manager: avoid "no record found" error in task scheduler 
It's fine when there is no task for a worker, so having Gorm log an error
was just causing noise.
 2022-03-01 11:52:28 +01:00
Sybren A. Stüvel
7e5a631f33 Cleanup: refactor updateJobAfterTaskStatusChange() 
Break up a complex function into smaller functions.
 2022-02-28 12:50:34 +01:00
Sybren A. Stüvel
fab988295d Manager: remove task scheduler SQL debug logs 2022-02-28 12:07:23 +01:00
Sybren A. Stüvel
b5b7b228ed Worker: capture panics in 'active' state and cleanly sign off before dying 2022-02-28 12:07:10 +01:00
Sybren A. Stüvel
7689a988b1 Manager: re-queue tasks of worker when signing off 2022-02-28 12:06:50 +01:00
Sybren A. Stüvel
32af1ffaef Manager: actually pass context to Gorm queries 2022-02-28 11:53:31 +01:00
Sybren A. Stüvel
3d854078ba Manager: integrate task state machine into API implementation 2022-02-25 16:30:27 +01:00
Sybren A. Stüvel
17e622ebc3 Some build & filename tweaks to allow building from scratch again 
The build chain got a bit confused when doing things from scratch, as
`test_support.go` was used in the non-test builds. Renaming it to
`support_test.go` was the easiest way to avoid that.
 2022-02-25 16:17:49 +01:00
Sybren A. Stüvel
9a5bbb4131 Manager: implement persistence layer interface for task status machine 
Implement the functions used by the task status machine in the DB
persistence layer.
 2022-02-25 14:34:29 +01:00
Sybren A. Stüvel
7279f2e35f Manager: task state machine, handle job status -> task status changes 
Direct copy of the Flamenco Server Python code, for handling the change
of a job's status to trigger status changes on its tasks.

Not yet connected to the rest of the Manager logic.
 2022-02-25 12:30:40 +01:00
Sybren A. Stüvel
e8707171b4 Manager: add test for saving task status change with error on job save 2022-02-25 11:07:06 +01:00
Sybren A. Stüvel
d2f1cf3614 Cleanup: move test helper functions down in the file 
This way it starts with the actual tests.
 2022-02-25 11:02:47 +01:00
Sybren A. Stüvel
a3a0d97222 Manager: more task state change tests 2022-02-24 17:58:02 +01:00
Sybren A. Stüvel
df77d09aa6 Start of a task/job state machine 
The task status change → job status change code is a direct port of the
Flamenco Server v2 code written in Python.

There is no job status change → task status changes logic yet, and the
tests are also far from complete.
 2022-02-24 16:42:38 +01:00
Sybren A. Stüvel
64db518f15 Cleanup: remove unused code 2022-02-24 12:36:29 +01:00
Sybren A. Stüvel
7420177209 Manager: use api.JobStatus in persistence layer as well 2022-02-24 11:54:35 +01:00
Sybren A. Stüvel
7e776167bb Manager: use api.TaskStatus in persistence layer as well 2022-02-24 11:53:05 +01:00
Sybren A. Stüvel
2a1f5a0df4 Worker: buffer task updates when Manager offline 
Queue task updates in an SQLite database when the Manager is
unreachable, periodically checking whether they can be flushed.
 2022-02-22 19:05:33 +01:00
Sybren A. Stüvel
e948c5d936 Worker: avoid prematurely exiting 'awake' state 2022-02-22 19:01:01 +01:00
Sybren A. Stüvel
5c3a5124f3 Cleanup: add missing comment 2022-02-22 19:00:44 +01:00
Sybren A. Stüvel
b462e8db1a Worker: wrap api.ClientWithResponsesInterface in a local interface 
This makes it possible to locally create a mock object for the API client.
 2022-02-22 17:13:04 +01:00
Sybren A. Stüvel
a650aa4cbd Worker: Blender command, don't put current time in log lines 
The time has to be put in at a different abstraction level.
 2022-02-22 15:36:20 +01:00
Sybren A. Stüvel
adf7113b34 Worker: log chunker, also flush log after certain time 
This flushes the log when the previous `Append()` call was too long ago.
Note that this doesn't flush after X seconds of silence; a call to
`Append()` or `Flush()` still has to happen in order to do the flushing.
 2022-02-22 15:34:37 +01:00
Sybren A. Stüvel
45a95ca4c2 Worker: use time service to log current time 2022-02-22 15:33:30 +01:00
Sybren A. Stüvel
66186e460e Worker: upload logs in chunks to Manager 2022-02-22 15:17:35 +01:00
Sybren A. Stüvel
9a5047a94d Worker: move command_exe.go interface definitions 2022-02-22 14:42:57 +01:00
Sybren A. Stüvel
0525bd726c Worker: actually execute blender and log its output 2022-02-22 13:55:28 +01:00
Sybren A. Stüvel
7d67a1dc7a Manager: default paths for executables are now relative 
This assumes that `blender` and `ffmpeg` are available on the commandline.
 2022-02-22 13:55:03 +01:00
Sybren A. Stüvel
32c7a1fd84 Worker: Blender command, parse CLI arguments from 'exe' parameter 2022-02-22 12:52:16 +01:00
Sybren A. Stüvel
db4505e5fd Cleanup: move Blender command param parsing to its own function 2022-02-22 12:38:25 +01:00
Sybren A. Stüvel
20965691d0 Add test for Blender command and better command parameter parsing 2022-02-22 12:36:37 +01:00
Sybren A. Stüvel
e9a94eecae Worker: add CLI runner interface and move test/mock code around a bit 2022-02-22 12:35:43 +01:00
Sybren A. Stüvel
80df8fa6e4 DB Initialisation: try named parameters 
Should be tested on Windows, as that's where this code will be used most
often. As of now, untested.
 2022-02-22 12:06:54 +01:00
Sybren A. Stüvel
77f1e02c75 Worker: add CommandLineRunner interface for executing CLIs 
Not yet used, but interface is there + mocked for testing.
 2022-02-22 11:48:29 +01:00
Sybren A. Stüvel
6b5b3eacdd Fix some warnings 2022-02-22 10:36:17 +01:00
Sybren A. Stüvel
ad6cbcbae2 Remove SSDP GUID from settings 2022-02-22 10:33:28 +01:00
Sybren A. Stüvel
f0215e20b8 Cleanup: move license block 2022-02-21 20:11:52 +01:00
Sybren A. Stüvel
b3b132fd44 Fix task update unit test 2022-02-21 20:11:19 +01:00
Sybren A. Stüvel
7111e2613e Remove CLI args from default {blender} variable 
Shell args parsing on Go is a bit tricky, need to find the right library
to do this before putting things into the default config.
 2022-02-21 20:07:50 +01:00
Sybren A. Stüvel
ba4b8274c9 Manager: Store default config in its own file 2022-02-21 20:06:57 +01:00
Sybren A. Stüvel
d198e228b7 Manager: perform variable replacement on scheduled tasks 2022-02-21 19:58:13 +01:00
Sybren A. Stüvel
90a2140b8c Manager: store task logs to disk 2022-02-21 19:47:07 +01:00
Sybren A. Stüvel
5b0e11acdc Cleanup: some smaller renames & small function return type change 2022-02-21 19:32:12 +01:00
Sybren A. Stüvel
2e0f44b947 Manager: add missing job type name in job types list 2022-02-21 19:31:37 +01:00
Sybren A. Stüvel
b986fa49d8 Worker: clean up some logging 2022-02-21 19:07:20 +01:00
Sybren A. Stüvel
b1b73de4ee Manager: Actually load config, and use in a few places 2022-02-21 19:02:07 +01:00
Sybren A. Stüvel
ae5846b3d9 Manager config: remove 'mode' and change 'variable audience' to custom type 
The 'variable audience' indicates the audience of a 'task variable'.
 2022-02-21 18:50:24 +01:00
Sybren A. Stüvel
12481a47e7 Start of configuration/settings framework, including variable replacement 2022-02-21 18:09:45 +01:00