blender/flamenco
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,202 Commits 1 Branch 0 Tags
Commit Graph

245 Commits

Author SHA1 Message Date
Sybren A. Stüvel
9643bf768e Manager: Fix DB migration error of not-null columns 
Where the PostgreSQL DB migration code could handle `NOT NULL` columns just
fine, SQLite has less table-altering functionality. As a result, migrations
have to copy entire database tables, which doesn't play well with
not-nullable columns.
 2022-03-03 12:10:13 +01:00
Sybren A. Stüvel
47e36c927c Change package URL to the blender.org repository 2022-03-01 20:45:09 +01:00
Sybren A. Stüvel
e70a44a146 Manager: switch from PostgreSQL to SQLite 
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
 2022-03-01 18:50:31 +01:00
Sybren A. Stüvel
0235ffcb4a Manager: avoid "no record found" error in task scheduler 
It's fine when there is no task for a worker, so having Gorm log an error
was just causing noise.
 2022-03-01 11:52:28 +01:00
Sybren A. Stüvel
fab988295d Manager: remove task scheduler SQL debug logs 2022-02-28 12:07:23 +01:00
Sybren A. Stüvel
7689a988b1 Manager: re-queue tasks of worker when signing off 2022-02-28 12:06:50 +01:00
Sybren A. Stüvel
32af1ffaef Manager: actually pass context to Gorm queries 2022-02-28 11:53:31 +01:00
Sybren A. Stüvel
3d854078ba Manager: integrate task state machine into API implementation 2022-02-25 16:30:27 +01:00
Sybren A. Stüvel
9a5bbb4131 Manager: implement persistence layer interface for task status machine 
Implement the functions used by the task status machine in the DB
persistence layer.
 2022-02-25 14:34:29 +01:00
Sybren A. Stüvel
7420177209 Manager: use api.JobStatus in persistence layer as well 2022-02-24 11:54:35 +01:00
Sybren A. Stüvel
7e776167bb Manager: use api.TaskStatus in persistence layer as well 2022-02-24 11:53:05 +01:00
Sybren A. Stüvel
80df8fa6e4 DB Initialisation: try named parameters 
Should be tested on Windows, as that's where this code will be used most
often. As of now, untested.
 2022-02-22 12:06:54 +01:00
Sybren A. Stüvel
90a2140b8c Manager: store task logs to disk 2022-02-21 19:47:07 +01:00
Sybren A. Stüvel
b1b73de4ee Manager: Actually load config, and use in a few places 2022-02-21 19:02:07 +01:00
Sybren A. Stüvel
66ae9b3a64 Job persistence test: create authored job with explicit status 2022-02-21 18:04:52 +01:00
Sybren A. Stüvel
ef2bbd2845 Unified Command field names 
Some parts of Flamenco had a Command consist of "name + settings", and
other parts used "type + parameters" (with the same semantics). This is
now unified to "name + parameters".
 2022-02-21 18:03:51 +01:00
Sybren A. Stüvel
6bd1a86337 Start of DB initialisation functions 
Not properly working yet
 2022-02-21 15:25:56 +01:00
Sybren A. Stüvel
399c8af750 Correctly handle workers assigned to tasks + simple task updates 2022-02-17 17:30:52 +01:00
Sybren A. Stüvel
270c54fdb7 More status change acks & checks to get stable flow between worker states 2022-02-15 17:46:37 +01:00
Sybren A. Stüvel
c4df62d5d4 Start of sending task updates to Manager 
This includes a mocking framework for unittests.
 2022-02-15 15:58:24 +01:00
Sybren A. Stüvel
9543d6221b Wrap error with %w 2022-02-15 15:56:54 +01:00
Sybren A. Stüvel
50088b4c94 Save worker info on sign-on (not just on registration) 2022-02-15 10:57:29 +01:00
Sybren A. Stüvel
0457809641 Scheduler: filter on supported task types & assigned worker ID 2022-02-14 18:00:43 +01:00
Sybren A. Stüvel
4aafb782ac Scheduler: Assign task to worker 2022-02-14 17:47:26 +01:00
Sybren A. Stüvel
eea219c3e2 Move task scheduler into persistence layer 2022-02-14 16:53:21 +01:00
Sybren A. Stüvel
8e01c069d1 DB: task dependencies should be cascade-deleted with their tasks 2022-02-14 15:07:14 +01:00
Sybren A. Stüvel
97ab93d996 Initial task scheduler implementation 2022-02-01 17:17:19 +01:00
Sybren A. Stüvel
fad2dc3042 Job authoring: handling of task dependencies + some bugfixes 2022-02-01 17:17:19 +01:00
Sybren A. Stüvel
e598397ba4 Persistence: move some DB to API struct conversion to API implementation 2022-02-01 16:22:10 +01:00
Sybren A. Stüvel
862ed96af7 Make persistence.CreateTestDB public so it can be used from other packages 2022-02-01 10:57:33 +01:00
Sybren A. Stüvel
2ca8858c28 Only update status field in DB when worker changes status 2022-02-01 10:16:10 +01:00
Sybren A. Stüvel
be89349632 Very basic non-functional framework for a task runner 
Also has some login/logout functionality for storing stuff in the DB.
 2022-01-31 16:05:27 +01:00
Sybren A. Stüvel
c501899185 Ported lots of stuff from gitlab.com/dr.sybren/flamenco-worker-go 
Much isn't working though.
 2022-01-28 17:02:50 +01:00
Sybren A. Stüvel
28a56f3d91 Store workers in database when registering 2022-01-28 15:31:39 +01:00
Sybren A. Stüvel
931fd1a24c Move db/jobs tests to persistence/jobs.go 2022-01-28 14:53:02 +01:00
Sybren A. Stüvel
3b1cb3c187 Remove sqlite library & run go mod tidy 2022-01-28 14:52:46 +01:00
Sybren A. Stüvel
0b3311b0a7 Change parameter format of blender-render command 2022-01-28 14:48:19 +01:00
Sybren A. Stüvel
30518ca3af Store tasks & commands in the database 2022-01-25 18:25:26 +01:00
Sybren A. Stüvel
edda8f21cf Make tests work on PostgreSQL 2022-01-25 18:12:01 +01:00
Sybren A. Stüvel
0f8bacd2e5 Store compiled jobs with GORM in PostgreSQL 2022-01-24 18:07:12 +01:00
Sybren A. Stüvel
b446b02739 Change package URL 
Change package URL from `gitlab.com/blender/flamenco-goja-test` to
`gitlab.com/blender/flamenco-ng-poc` so that it matches the actual URL
on Gitlab.
 2022-01-24 14:28:00 +01:00
Sybren A. Stüvel
0629728ce9 Add frame chunker and make unit test for simple blender render succeed 2022-01-13 16:32:08 +01:00
Sybren A. Stüvel
54933d0262 API: add Fetch Job endpoint 
This is made with plain sqlite. It's probably a good idea to move to
something like GORM instead.
 2022-01-11 18:38:14 +01:00
Sybren A. Stüvel
f0ae7d33e9 Connect API, job compiler, and persistence layer 2022-01-11 18:07:34 +01:00
Sybren A. Stüvel
0c9c99806b Start of database interface 
Contains a dummy migration because the migrate package will error out if
it cannot find any migration files.
 2022-01-11 18:07:34 +01:00