blender/flamenco
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
210 Commits 1 Branch 0 Tags
Commit Graph

98 Commits

Author SHA1 Message Date
Sybren A. Stüvel
641ed7ace9 Manager: make Gorm use Zerolog for logging 
A wrapper for Zerolog implements the Gorm logger interface. This gives
us coloured output on Windows, and uniform-looking logs in production.
 2022-03-03 13:52:50 +01:00
Sybren A. Stüvel
8824489980 Manager: use in-memory SQLite database for testing 
The on-disk database that was used before caused issues with tests running
in parallel. Not only is there the theoretical issue of tests seeing each
other's data (this didn't happen, but could), there was also the practical
issue of one test running while the other tried to erase the database file
(which fails on Windows due to file locking).
 2022-03-03 13:51:55 +01:00
Sybren A. Stüvel
9b9c6bffff Replace self-hacked SQLite Gorm driver with 3rd party one 
The new Gorm driver is made by the creators of the pure-Go SQLite library
we were already using.
 2022-03-03 13:48:14 +01:00
Sybren A. Stüvel
2b04623e00 Manager: fix DB transaction isolation issue in task scheduler 
The created transaction wasn't actually used for the should-be-in-the-
transaction queries. That's now resolved.
 2022-03-03 13:46:27 +01:00
Sybren A. Stüvel
9643bf768e Manager: Fix DB migration error of not-null columns 
Where the PostgreSQL DB migration code could handle `NOT NULL` columns just
fine, SQLite has less table-altering functionality. As a result, migrations
have to copy entire database tables, which doesn't play well with
not-nullable columns.
 2022-03-03 12:10:13 +01:00
Sybren A. Stüvel
47e36c927c Change package URL to the blender.org repository 2022-03-01 20:45:09 +01:00
Sybren A. Stüvel
e70a44a146 Manager: switch from PostgreSQL to SQLite 
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
 2022-03-01 18:50:31 +01:00
Sybren A. Stüvel
bbc6a3f69e Tweak some logging 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
0235ffcb4a Manager: avoid "no record found" error in task scheduler 
It's fine when there is no task for a worker, so having Gorm log an error
was just causing noise.
 2022-03-01 11:52:28 +01:00
Sybren A. Stüvel
7e5a631f33 Cleanup: refactor updateJobAfterTaskStatusChange() 
Break up a complex function into smaller functions.
 2022-02-28 12:50:34 +01:00
Sybren A. Stüvel
fab988295d Manager: remove task scheduler SQL debug logs 2022-02-28 12:07:23 +01:00
Sybren A. Stüvel
7689a988b1 Manager: re-queue tasks of worker when signing off 2022-02-28 12:06:50 +01:00
Sybren A. Stüvel
32af1ffaef Manager: actually pass context to Gorm queries 2022-02-28 11:53:31 +01:00
Sybren A. Stüvel
3d854078ba Manager: integrate task state machine into API implementation 2022-02-25 16:30:27 +01:00
Sybren A. Stüvel
17e622ebc3 Some build & filename tweaks to allow building from scratch again 
The build chain got a bit confused when doing things from scratch, as
`test_support.go` was used in the non-test builds. Renaming it to
`support_test.go` was the easiest way to avoid that.
 2022-02-25 16:17:49 +01:00
Sybren A. Stüvel
9a5bbb4131 Manager: implement persistence layer interface for task status machine 
Implement the functions used by the task status machine in the DB
persistence layer.
 2022-02-25 14:34:29 +01:00
Sybren A. Stüvel
7279f2e35f Manager: task state machine, handle job status -> task status changes 
Direct copy of the Flamenco Server Python code, for handling the change
of a job's status to trigger status changes on its tasks.

Not yet connected to the rest of the Manager logic.
 2022-02-25 12:30:40 +01:00
Sybren A. Stüvel
e8707171b4 Manager: add test for saving task status change with error on job save 2022-02-25 11:07:06 +01:00
Sybren A. Stüvel
d2f1cf3614 Cleanup: move test helper functions down in the file 
This way it starts with the actual tests.
 2022-02-25 11:02:47 +01:00
Sybren A. Stüvel
a3a0d97222 Manager: more task state change tests 2022-02-24 17:58:02 +01:00
Sybren A. Stüvel
df77d09aa6 Start of a task/job state machine 
The task status change → job status change code is a direct port of the
Flamenco Server v2 code written in Python.

There is no job status change → task status changes logic yet, and the
tests are also far from complete.
 2022-02-24 16:42:38 +01:00
Sybren A. Stüvel
64db518f15 Cleanup: remove unused code 2022-02-24 12:36:29 +01:00
Sybren A. Stüvel
7420177209 Manager: use api.JobStatus in persistence layer as well 2022-02-24 11:54:35 +01:00
Sybren A. Stüvel
7e776167bb Manager: use api.TaskStatus in persistence layer as well 2022-02-24 11:53:05 +01:00
Sybren A. Stüvel
7d67a1dc7a Manager: default paths for executables are now relative 
This assumes that `blender` and `ffmpeg` are available on the commandline.
 2022-02-22 13:55:03 +01:00
Sybren A. Stüvel
80df8fa6e4 DB Initialisation: try named parameters 
Should be tested on Windows, as that's where this code will be used most
often. As of now, untested.
 2022-02-22 12:06:54 +01:00
Sybren A. Stüvel
6b5b3eacdd Fix some warnings 2022-02-22 10:36:17 +01:00
Sybren A. Stüvel
ad6cbcbae2 Remove SSDP GUID from settings 2022-02-22 10:33:28 +01:00
Sybren A. Stüvel
f0215e20b8 Cleanup: move license block 2022-02-21 20:11:52 +01:00
Sybren A. Stüvel
b3b132fd44 Fix task update unit test 2022-02-21 20:11:19 +01:00
Sybren A. Stüvel
7111e2613e Remove CLI args from default {blender} variable 
Shell args parsing on Go is a bit tricky, need to find the right library
to do this before putting things into the default config.
 2022-02-21 20:07:50 +01:00
Sybren A. Stüvel
ba4b8274c9 Manager: Store default config in its own file 2022-02-21 20:06:57 +01:00
Sybren A. Stüvel
d198e228b7 Manager: perform variable replacement on scheduled tasks 2022-02-21 19:58:13 +01:00
Sybren A. Stüvel
90a2140b8c Manager: store task logs to disk 2022-02-21 19:47:07 +01:00
Sybren A. Stüvel
5b0e11acdc Cleanup: some smaller renames & small function return type change 2022-02-21 19:32:12 +01:00
Sybren A. Stüvel
2e0f44b947 Manager: add missing job type name in job types list 2022-02-21 19:31:37 +01:00
Sybren A. Stüvel
b1b73de4ee Manager: Actually load config, and use in a few places 2022-02-21 19:02:07 +01:00
Sybren A. Stüvel
ae5846b3d9 Manager config: remove 'mode' and change 'variable audience' to custom type 
The 'variable audience' indicates the audience of a 'task variable'.
 2022-02-21 18:50:24 +01:00
Sybren A. Stüvel
12481a47e7 Start of configuration/settings framework, including variable replacement 2022-02-21 18:09:45 +01:00
Sybren A. Stüvel
66ae9b3a64 Job persistence test: create authored job with explicit status 2022-02-21 18:04:52 +01:00
Sybren A. Stüvel
ef2bbd2845 Unified Command field names 
Some parts of Flamenco had a Command consist of "name + settings", and
other parts used "type + parameters" (with the same semantics). This is
now unified to "name + parameters".
 2022-02-21 18:03:51 +01:00
Sybren A. Stüvel
6a7545f334 Check author.Task() parameters 
Check author.Task() parameters, and refuse tasks with empty name or type.
 2022-02-21 15:49:23 +01:00
Sybren A. Stüvel
1756369b70 Cleanup: add comments 2022-02-21 15:41:00 +01:00
Sybren A. Stüvel
6bd1a86337 Start of DB initialisation functions 
Not properly working yet
 2022-02-21 15:25:56 +01:00
Sybren A. Stüvel
bb53cc1e4a Task log storage service 2022-02-18 18:19:35 +01:00
Sybren A. Stüvel
77c5650665 Cleanup: move imports to the right place 2022-02-18 17:25:20 +01:00
Sybren A. Stüvel
5b4625b5f9 Rename logging.go to zerolog.go 2022-02-18 11:58:18 +01:00
Sybren A. Stüvel
399c8af750 Correctly handle workers assigned to tasks + simple task updates 2022-02-17 17:30:52 +01:00
Sybren A. Stüvel
31e39e2137 Cleanup: remove fmt.Sprintf from sendAPIError calls 2022-02-17 17:30:27 +01:00
Sybren A. Stüvel
0ab8151a92 Move code out of main.go and add better check for OpenAPI paths 2022-02-17 13:58:10 +01:00