blender/flamenco
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,210 Commits 1 Branch 0 Tags
Commit Graph

182 Commits

Author SHA1 Message Date
Sybren A. Stüvel
1176d85496 Worker: add -flush CLI option to just flush the buffer and quit 
Add `-flush` CLI option to just flush the upstream queue and then quit.
 2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
55d264632f Worker: completely flush upstream buffer at startup 
Completely flush the upstream buffer at startup, before attempting to
fetch a new task. These updates could impact any task on the Manager side,
and first flushing the buffer before appending new updates also seems
like a good idea.
 2022-04-21 19:06:18 +02:00
Sybren A. Stüvel
4ea6f99c3e Manager: dynamically generate allowed CORS origins 
Remove the hard-coded list of allowed CORS origins, and build it
dynamically from the list of "own URLs", i.e. the URLs at which the
Manager expects to be available.

This list of "own URLs" is constructed from the available network
interfaces.
 2022-04-15 14:27:52 +02:00
Sybren A. Stüvel
c0b2c6c234 Manager: CORS origins should be without trailing slash 2022-04-12 12:34:10 +02:00
Sybren A. Stüvel
959a235c59 Worker: don't log error when rolling back already-committed transaction 2022-04-08 16:48:28 +02:00
Sybren A. Stüvel
a715b3bfbe Manager: connect SocketIO broadcaster with job creation 2022-04-05 16:19:33 +02:00
Sybren A. Stüvel
0c0df41f5d Job status change system for SocketIO broadcasts 
Not fully tested yet.
 2022-04-05 15:52:55 +02:00
Sybren A. Stüvel
0f2477370f Remove SocketIO PoC 
Remove SocketIO proof of concept, as its code has been integrated into
Flamenco Manager.
 2022-04-04 16:27:15 +02:00
Sybren A. Stüvel
e990603311 OAPI: Add generated JavaScript API client 
This adds a JS client for the OAPI interface, and introduces the SocketIO
stuff into Flamenco Manager itself.

To build & run:
- in `web/manager-api` run `npm install`
- in `web/manager-api` run `npm link`
- in `web/app` run `npm install`
- in `web/app` run `npm link flamenco-manager`
- in `web/app` run `yarn serve`

This may not be a complete list, but at least some of those steps are
necessary.
 2022-04-01 16:40:54 +02:00
Sybren A. Stüvel
2f0b77f45b Manager: disable GZip middleware, as it causes some issues 
Apparently a "204 No Content" response will still get GZip-compressed.
 2022-03-25 16:09:31 +01:00
Sybren A. Stüvel
b2288e7f28 Manager: More work on Shaman support 
This introduces some more conceptual changes to Shaman. The most important
one is that there is no longer a "checkout ID", but a "checkout path".
The Shaman client can request any subpath of the checkout directory,
so that it can handle things like project- or scene-specific prefixes.
 2022-03-25 14:10:26 +01:00
Sybren A. Stüvel
c9dbb2620b Manager: add GZip middleware 
Add support for GZip-compressed streams. This becomes increasingly
important when Shaman support gets added, as the blend files typically
used in studios are uncompressed to support binary diffing.
 2022-03-21 13:24:07 +01:00
Sybren A. Stüvel
e6f224feca Worker: change database filename to flamenco-worker.sqlite 
This makes the Worker and Manager use similarly-named SQLite databases.
 2022-03-17 17:13:12 +01:00
Sybren A. Stüvel
9096ee0533 Manager: disable periodic VACUUM 2022-03-17 15:38:14 +01:00
Sybren A. Stüvel
50cc1e64d0 Manager: add -write-config CLI option to create config file and quit 2022-03-17 11:18:26 +01:00
Sybren A. Stüvel
22ea599554 Manager: periodically run the SQL VACUUM command 2022-03-17 11:03:29 +01:00
Sybren A. Stüvel
2e78e00a0b Manager: don't log an error when the config file doesn't exist 
The configuration file is expected to not exist on many systems, and
thus logging an error (even when it's a very innocent one) will cause
confusion.
 2022-03-14 13:00:27 +01:00
Sybren A. Stüvel
afc6b8e1c5 Worker: add -find-manager CLI arg 
With `-find-manager` the Worker will use UPnP/SSDP autodiscovery, report
which Manager it found, and quit.
 2022-03-11 10:52:03 +01:00
Sybren A. Stüvel
b33db33d17 Manager: show error when there was an error loading configuration 2022-03-11 10:51:32 +01:00
Sybren A. Stüvel
fe91d5e59e Remove UPnP/SSDP proof of concept code 
The code is no longer necessary.
 2022-03-08 17:26:39 +01:00
Sybren A. Stüvel
842255a065 UPnP/SSDP: actually serve a description.xml 
The `description.xml` file is part of the UPnP Service Discovery protocol,
see https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-rxad/9225d145-6b2c-40d5-8ee8-7d837379fc25
 2022-03-08 17:25:49 +01:00
Sybren A. Stüvel
fabb79e583 Worker: move autodiscovery code into the 'worker' package 
Move the UPnP/SSDP Manager autodiscovery code into from `main.go` into the
`worker` package. This also means changing the error handling a bit, as
only the `main.go` file is allowed to do `log.Fatal()`.
 2022-03-08 14:41:30 +01:00
Sybren A. Stüvel
8e05024267 UPnP/SSDP: Worker, ping Manager URLs in parallel 
Ping all Manager URLs at once, and wait until they've all responded (or
caused an error), instead of pinging them one by one sequentially.
 2022-03-08 14:25:44 +01:00
Sybren A. Stüvel
ae892ff712 Worker: test all discovered Manager URLs 
Ping each discovered Manager URL to see whether it is reachable or not.
If there are multiple usable URLs, it'll just pick the first.
 2022-03-08 13:56:45 +01:00
Sybren A. Stüvel
d6a60c73d0 UPnP/SSDP Server: allow advertising slice of URLs 
This makes it possible for the Manager to expose multiple URLs. This way
the Worker can try them out and see which ones work.
 2022-03-08 13:56:45 +01:00
Sybren A. Stüvel
c0cd3ca5ad UPnP/SSDP: Cleaner, easier to test (and actually tested) code 
Convert "get own URLs" code into nicer chunks, and test those.

This minimises the code that actually depends on the available network
interfaces, and increases test coverage. Found a few bugs too.
 2022-03-08 12:11:47 +01:00
Sybren A. Stüvel
fca628b325 Merge remote-tracking branch 'origin/main' into upnp-ssdp 2022-03-07 15:27:12 +01:00
Sybren A. Stüvel
9f5e4cc0cc License: license all code under "GPL-3.0-or-later" 
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
 2022-03-07 15:26:46 +01:00
Sybren A. Stüvel
4062f024b7 Cleanup: move code to a nicer place 2022-03-04 17:46:44 +01:00
Sybren A. Stüvel
d153db4280 Work in progress on using UPnP/SSDP to make the Worker find its Manager 
Due to the way SSDP works, Flamenco Manager needs to know its own URL,
where the Workers can reach it. These URLs are now found, and since there
can be multiple (like IPv6 + IPv4) they are all sent in a SSDP
notification as ;-separated strings.
 2022-03-04 17:44:04 +01:00
Sybren A. Stüvel
656a495652 Manager: clean shutdown on Ctrl+C 2022-03-04 14:34:40 +01:00
Sybren A. Stüvel
eaeea38e3f Worker: remove -manager URL CLI argument 
This CLI arg isn't actually used by the subsequent code, so better to not
have it for now.
 2022-03-03 11:33:22 +01:00
Sybren A. Stüvel
20fd3228ba Worker: better logging configuration 
Always show startup notification at INFO level, regardless of configured
level. Replace `-verbose` with `-quiet`, add `-trace`.
 2022-03-03 11:32:57 +01:00
Sybren A. Stüvel
954babd914 Manager: redirect web root to Swagger UI 
Temporarily redirect the index page to the Swagger UI, so that at least
you can see something other than a "404 Not Found" error.
 2022-03-01 20:49:11 +01:00
Sybren A. Stüvel
47e36c927c Change package URL to the blender.org repository 2022-03-01 20:45:09 +01:00
Sybren A. Stüvel
e70a44a146 Manager: switch from PostgreSQL to SQLite 
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
 2022-03-01 18:50:31 +01:00
Sybren A. Stüvel
ba5585461b Cleanup: fix 'vet' warning from socketio-poc 2022-02-28 13:01:52 +01:00
Sybren A. Stüvel
194422d9fa Remove -poc suffix from Manager and Worker 
The code is mature enough to no longer be a proof-of-concept.
 2022-02-28 13:01:13 +01:00
Sybren A. Stüvel
3d854078ba Manager: integrate task state machine into API implementation 2022-02-25 16:30:27 +01:00
Sybren A. Stüvel
2a1f5a0df4 Worker: buffer task updates when Manager offline 
Queue task updates in an SQLite database when the Manager is
unreachable, periodically checking whether they can be flushed.
 2022-02-22 19:05:33 +01:00
Sybren A. Stüvel
e9a94eecae Worker: add CLI runner interface and move test/mock code around a bit 2022-02-22 12:35:43 +01:00
Sybren A. Stüvel
b1b73de4ee Manager: Actually load config, and use in a few places 2022-02-21 19:02:07 +01:00
Sybren A. Stüvel
30fd4e52b0 Log GOOS and GOARCH on startup 2022-02-21 19:01:43 +01:00
Sybren A. Stüvel
12481a47e7 Start of configuration/settings framework, including variable replacement 2022-02-21 18:09:45 +01:00
Sybren A. Stüvel
6bd1a86337 Start of DB initialisation functions 
Not properly working yet
 2022-02-21 15:25:56 +01:00
Sybren A. Stüvel
bb53cc1e4a Task log storage service 2022-02-18 18:19:35 +01:00
Sybren A. Stüvel
0ab8151a92 Move code out of main.go and add better check for OpenAPI paths 2022-02-17 13:58:10 +01:00
Sybren A. Stüvel
66c052d9fd Move worker config, sign-on, and registration code into worker package 
This makes the `main.go` file simpler, and not depend on any other files
in the `main` package. For some reason, the debugger really likes this.
 2022-02-17 13:58:10 +01:00
Sybren A. Stüvel
dfd55914b2 Worker: keep trying to sign on until we have an answer from the Manager 2022-02-15 11:46:00 +01:00
Sybren A. Stüvel
beda64d7c0 Clean up authentication routine 2022-02-15 11:29:48 +01:00