blender/flamenco
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
413 Commits 1 Branch 0 Tags
Commit Graph

413 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sybren A. Stüvel
580b3e5726 Go.mod: mark github.com/glebarez/sqlite as directly used 2022-03-03 15:26:48 +01:00
Sybren A. Stüvel
c91e7b1cac Manager: solve failing unittests by implementing some filepath functions 
Both Go's standard `path` and `path/filepath` packages are too limiting to
work well for Flamenco. The former assumes Linux/POSIX paths, the latter
only works with platform-native paths. Neither can work with Windows paths
on Linux, or Linux paths on Windows.
 2022-03-03 15:22:43 +01:00
Sybren A. Stüvel
b9609f8866 Cleanup: remove unused code 2022-03-03 13:52:57 +01:00
Sybren A. Stüvel
641ed7ace9 Manager: make Gorm use Zerolog for logging 
A wrapper for Zerolog implements the Gorm logger interface. This gives
us coloured output on Windows, and uniform-looking logs in production.
 2022-03-03 13:52:50 +01:00
Sybren A. Stüvel
8824489980 Manager: use in-memory SQLite database for testing 
The on-disk database that was used before caused issues with tests running
in parallel. Not only is there the theoretical issue of tests seeing each
other's data (this didn't happen, but could), there was also the practical
issue of one test running while the other tried to erase the database file
(which fails on Windows due to file locking).
 2022-03-03 13:51:55 +01:00
Sybren A. Stüvel
9b9c6bffff Replace self-hacked SQLite Gorm driver with 3rd party one 
The new Gorm driver is made by the creators of the pure-Go SQLite library
we were already using.
 2022-03-03 13:48:14 +01:00
Sybren A. Stüvel
2b04623e00 Manager: fix DB transaction isolation issue in task scheduler 
The created transaction wasn't actually used for the should-be-in-the-
transaction queries. That's now resolved.
 2022-03-03 13:46:27 +01:00
Sybren A. Stüvel
a563f36b98 README: document licenses better 2022-03-03 12:10:24 +01:00
Sybren A. Stüvel
9643bf768e Manager: Fix DB migration error of not-null columns 
Where the PostgreSQL DB migration code could handle `NOT NULL` columns just
fine, SQLite has less table-altering functionality. As a result, migrations
have to copy entire database tables, which doesn't play well with
not-nullable columns.
 2022-03-03 12:10:13 +01:00
Sybren A. Stüvel
bf4cc9b056 Add notice to our SQLite Gorm driver 
The Gorm driver README now links to the original code, which was modified
to use a different SQLite library.
 2022-03-03 11:52:43 +01:00
Sybren A. Stüvel
a7654394b4 Worker: update default task types to match what's currently in use 
Flamenco v2 had a different classification of task types than Flamenco v3,
and the defaults still referenced the v2 classifications.
 2022-03-03 11:51:53 +01:00
Sybren A. Stüvel
2610836861 Makefile: don't generate code when running make 
Generated code is to be committed to Git anyway, so there is no need to
regenerate it on every build.

The code can be regenerated explicitly by running `make generate`.
 2022-03-03 11:51:15 +01:00
Sybren A. Stüvel
efbbb8c7d8 Makefile: remove some unused parts 
Some parts were still leftovers from other projects this Makefile was
copied from.
 2022-03-03 11:50:03 +01:00
Sybren A. Stüvel
42daa87839 Makefile: convert generated files to native line-ends on Windows 
Git wants to see native line-ends in source files, but the code generators
we use always write UNIX line-ends. `make generate` on Windows now passes
generated files through `unix2dos`. This allows regenerating files without
Git listing them as modified.
 2022-03-03 11:49:26 +01:00
Sybren A. Stüvel
eaeea38e3f Worker: remove -manager URL CLI argument 
This CLI arg isn't actually used by the subsequent code, so better to not
have it for now.
 2022-03-03 11:33:22 +01:00
Sybren A. Stüvel
20fd3228ba Worker: better logging configuration 
Always show startup notification at INFO level, regardless of configured
level. Replace `-verbose` with `-quiet`, add `-trace`.
 2022-03-03 11:32:57 +01:00
Sybren A. Stüvel
954babd914 Manager: redirect web root to Swagger UI 
Temporarily redirect the index page to the Swagger UI, so that at least
you can see something other than a "404 Not Found" error.
 2022-03-01 20:49:11 +01:00
Sybren A. Stüvel
ddd0c7602b Change application name to "Flamenco 3" 
"Flamenco NG PoC" was the name of the proof of concept, and it's no longer
in use.
 2022-03-01 20:45:13 +01:00
Sybren A. Stüvel
47e36c927c Change package URL to the blender.org repository 2022-03-01 20:45:09 +01:00
Sybren A. Stüvel
e70a44a146 Manager: switch from PostgreSQL to SQLite 
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
 2022-03-01 18:50:31 +01:00
Sybren A. Stüvel
60d1710020 Addon: job type form seems to be working 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
5e7e2adef0 Addon: Fixed mypy errors 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
616784df0a Addon: start of framework for API communication 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
b678b90932 Proof of concept to test a generated Python OpenAPI client 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
bbc6a3f69e Tweak some logging 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
0235ffcb4a Manager: avoid "no record found" error in task scheduler 
It's fine when there is no task for a worker, so having Gorm log an error
was just causing noise.
 2022-03-01 11:52:28 +01:00
Sybren A. Stüvel
648a220098 Update FEATURES.md 2022-02-28 15:30:50 +01:00
Sybren A. Stüvel
6067a0ffb0 Remove 'make static' target 
Having `CGO_ENABLED=0` already produces a static binary.
 2022-02-28 13:02:43 +01:00
Sybren A. Stüvel
ba5585461b Cleanup: fix 'vet' warning from socketio-poc 2022-02-28 13:01:52 +01:00
Sybren A. Stüvel
194422d9fa Remove -poc suffix from Manager and Worker 
The code is mature enough to no longer be a proof-of-concept.
 2022-02-28 13:01:13 +01:00
Sybren A. Stüvel
4627169106 Remove unused Makefile targets 
The Makefile started as a standard file I use for multiple Go projects,
but it was aimed at having only one executable to build. I've removed
everything that assumes a single executable, and kept the parts that are
actually used now.
 2022-02-28 12:59:30 +01:00
Sybren A. Stüvel
7e5a631f33 Cleanup: refactor updateJobAfterTaskStatusChange() 
Break up a complex function into smaller functions.
 2022-02-28 12:50:34 +01:00
Sybren A. Stüvel
41168ff68b Update FEATURES.md 2022-02-28 12:09:07 +01:00
Sybren A. Stüvel
fab988295d Manager: remove task scheduler SQL debug logs 2022-02-28 12:07:23 +01:00
Sybren A. Stüvel
b5b7b228ed Worker: capture panics in 'active' state and cleanly sign off before dying 2022-02-28 12:07:10 +01:00
Sybren A. Stüvel
7689a988b1 Manager: re-queue tasks of worker when signing off 2022-02-28 12:06:50 +01:00
Sybren A. Stüvel
32af1ffaef Manager: actually pass context to Gorm queries 2022-02-28 11:53:31 +01:00
Sybren A. Stüvel
3d854078ba Manager: integrate task state machine into API implementation 2022-02-25 16:30:27 +01:00
Sybren A. Stüvel
17e622ebc3 Some build & filename tweaks to allow building from scratch again 
The build chain got a bit confused when doing things from scratch, as
`test_support.go` was used in the non-test builds. Renaming it to
`support_test.go` was the easiest way to avoid that.
 2022-02-25 16:17:49 +01:00
Sybren A. Stüvel
9a5bbb4131 Manager: implement persistence layer interface for task status machine 
Implement the functions used by the task status machine in the DB
persistence layer.
 2022-02-25 14:34:29 +01:00
Sybren A. Stüvel
7279f2e35f Manager: task state machine, handle job status -> task status changes 
Direct copy of the Flamenco Server Python code, for handling the change
of a job's status to trigger status changes on its tasks.

Not yet connected to the rest of the Manager logic.
 2022-02-25 12:30:40 +01:00
Sybren A. Stüvel
e8707171b4 Manager: add test for saving task status change with error on job save 2022-02-25 11:07:06 +01:00
Sybren A. Stüvel
d2f1cf3614 Cleanup: move test helper functions down in the file 
This way it starts with the actual tests.
 2022-02-25 11:02:47 +01:00
Sybren A. Stüvel
a3a0d97222 Manager: more task state change tests 2022-02-24 17:58:02 +01:00
Sybren A. Stüvel
df77d09aa6 Start of a task/job state machine 
The task status change → job status change code is a direct port of the
Flamenco Server v2 code written in Python.

There is no job status change → task status changes logic yet, and the
tests are also far from complete.
 2022-02-24 16:42:38 +01:00
Sybren A. Stüvel
64db518f15 Cleanup: remove unused code 2022-02-24 12:36:29 +01:00
Sybren A. Stüvel
7420177209 Manager: use api.JobStatus in persistence layer as well 2022-02-24 11:54:35 +01:00
Sybren A. Stüvel
7e776167bb Manager: use api.TaskStatus in persistence layer as well 2022-02-24 11:53:05 +01:00
Sybren A. Stüvel
4cd10e57b2 Run all Go commands with CGO_ENABLED=0 
This ensures that nothing will require a C compiler.
 2022-02-24 11:37:02 +01:00
Sybren A. Stüvel
f0aea6646f Run go generate in the right order 
The `pkg/api` code needs to be generated before the interface mocks that
use it.
 2022-02-24 11:35:54 +01:00