1636 Commits

Author SHA1 Message Date
Sybren A. Stüvel
d6a60c73d0 UPnP/SSDP Server: allow advertising slice of URLs
This makes it possible for the Manager to expose multiple URLs. This way
the Worker can try them out and see which ones work.
2022-03-08 13:56:45 +01:00
Sybren A. Stüvel
13a74e61d4 UPnP/SSDP client: allow clean shutdowns
The SSDP library will log a warning on shutdown, as it doesn't expect the
UDP socket to be closed. Copy the hack on the Server implementation, which
avoids this warning getting logged, to the Client.
2022-03-08 13:56:45 +01:00
Sybren A. Stüvel
d0b677ffe3 Add API endpoint /api/version to get the Flamenco Manager version
This will be used in the auto-discovery, to check that a discovered URL can
actually be reached & points to a Flamenco Manager.
2022-03-08 13:56:43 +01:00
Sybren A. Stüvel
42407865eb Worker: change how configuration & credentials are loaded/saved
The Worker config/credential management was a bit of a mess. It's now
better structured, and also allows runtime overrides of the Manager URL,
without writing that override to the config file.
2022-03-08 13:51:03 +01:00
Sybren A. Stüvel
c0cd3ca5ad UPnP/SSDP: Cleaner, easier to test (and actually tested) code
Convert "get own URLs" code into nicer chunks, and test those.

This minimises the code that actually depends on the available network
interfaces, and increases test coverage. Found a few bugs too.
2022-03-08 12:11:47 +01:00
Sybren A. Stüvel
bb91c2e6d6 UPnP/SSDP: prefer link-local addresses if available 2022-03-08 11:03:48 +01:00
Sybren A. Stüvel
fca628b325 Merge remote-tracking branch 'origin/main' into upnp-ssdp 2022-03-07 15:27:12 +01:00
Sybren A. Stüvel
9f5e4cc0cc License: license all code under "GPL-3.0-or-later"
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
2022-03-07 15:26:46 +01:00
Sybren A. Stüvel
43a9e1d439 Add some nice-to-have new features to FEATURES.md 2022-03-07 11:21:31 +01:00
Sybren A. Stüvel
4062f024b7 Cleanup: move code to a nicer place 2022-03-04 17:46:44 +01:00
Sybren A. Stüvel
d153db4280 Work in progress on using UPnP/SSDP to make the Worker find its Manager
Due to the way SSDP works, Flamenco Manager needs to know its own URL,
where the Workers can reach it. These URLs are now found, and since there
can be multiple (like IPv6 + IPv4) they are all sent in a SSDP
notification as ;-separated strings.
2022-03-04 17:44:04 +01:00
Sybren A. Stüvel
656a495652 Manager: clean shutdown on Ctrl+C 2022-03-04 14:34:40 +01:00
Sybren A. Stüvel
1e784452f3 README: no longer a "proof of concept" 2022-03-04 13:24:57 +01:00
Sybren A. Stüvel
183bb23d2a Tests: sleep test using bigger time steps
Since every mocked clock time step also waits for 1ms to give other
goroutines a chance to run, it took too much wallclock time to mock-sleep
for 47 seconds with 100ms increments.

Stepping the mocked clock with 1s increments makes the test 10x faster.
2022-03-04 12:35:01 +01:00
Sybren A. Stüvel
dbb9c71df8 Tests: more unified way to do database tests 2022-03-04 12:33:45 +01:00
Sybren A. Stüvel
f497ac8536 Cleanup: add and remove some comments 2022-03-04 12:19:19 +01:00
Sybren A. Stüvel
3bfd5a339f Fix database tests getting interrupted
The root cause was a 2nd `context.Context()` that was used in
`constructTestJob()`, which cancelled when that function returned.

The cancellation of the context caused an interrupt in the SQLite
driver, which got into a race condition and could cause an interrupt on
a subsequent database query.
2022-03-04 12:19:19 +01:00
Sybren A. Stüvel
de150567b0 Manager: avoid double error message 2022-03-04 11:37:29 +01:00
Sybren A. Stüvel
cd2fe8170e Errors: remove "error" prefix from message
Instead of returning an error "error doing X", just return "doing X". The
fact that it's returned as an error object says enough about that it's
an error.

This also makes it easier to chain error messages, without seeing the
word "error" in every part of the chain.
2022-03-04 11:30:31 +01:00
Sybren A. Stüvel
02ce95070f Add VSCode extension recommendations 2022-03-04 11:13:17 +01:00
Sybren A. Stüvel
3c6d7773ef Crosspath: add ToNative() function
Add `crosspath.ToNative(path)`, which returns the path with platform-
native path separators. This is meant for use in the Worker, to convert
paths before attempting to use them.
2022-03-04 11:10:10 +01:00
Sybren A. Stüvel
70f8140df5 Crosspath: extend tests with "long UNC" paths
See https://en.wikipedia.org/wiki/Path_(computing)#Universal_Naming_Convention
2022-03-04 11:09:14 +01:00
Sybren A. Stüvel
0a97d04741 Cleanup: rewrite crosspath.Stem unit test
Use the same style of testing as the other test functions in the file.

No functional changes.
2022-03-04 10:27:31 +01:00
Sybren A. Stüvel
580b3e5726 Go.mod: mark github.com/glebarez/sqlite as directly used 2022-03-03 15:26:48 +01:00
Sybren A. Stüvel
c91e7b1cac Manager: solve failing unittests by implementing some filepath functions
Both Go's standard `path` and `path/filepath` packages are too limiting to
work well for Flamenco. The former assumes Linux/POSIX paths, the latter
only works with platform-native paths. Neither can work with Windows paths
on Linux, or Linux paths on Windows.
2022-03-03 15:22:43 +01:00
Sybren A. Stüvel
b9609f8866 Cleanup: remove unused code 2022-03-03 13:52:57 +01:00
Sybren A. Stüvel
641ed7ace9 Manager: make Gorm use Zerolog for logging
A wrapper for Zerolog implements the Gorm logger interface. This gives
us coloured output on Windows, and uniform-looking logs in production.
2022-03-03 13:52:50 +01:00
Sybren A. Stüvel
8824489980 Manager: use in-memory SQLite database for testing
The on-disk database that was used before caused issues with tests running
in parallel. Not only is there the theoretical issue of tests seeing each
other's data (this didn't happen, but could), there was also the practical
issue of one test running while the other tried to erase the database file
(which fails on Windows due to file locking).
2022-03-03 13:51:55 +01:00
Sybren A. Stüvel
9b9c6bffff Replace self-hacked SQLite Gorm driver with 3rd party one
The new Gorm driver is made by the creators of the pure-Go SQLite library
we were already using.
2022-03-03 13:48:14 +01:00
Sybren A. Stüvel
2b04623e00 Manager: fix DB transaction isolation issue in task scheduler
The created transaction wasn't actually used for the should-be-in-the-
transaction queries. That's now resolved.
2022-03-03 13:46:27 +01:00
Sybren A. Stüvel
a563f36b98 README: document licenses better 2022-03-03 12:10:24 +01:00
Sybren A. Stüvel
9643bf768e Manager: Fix DB migration error of not-null columns
Where the PostgreSQL DB migration code could handle `NOT NULL` columns just
fine, SQLite has less table-altering functionality. As a result, migrations
have to copy entire database tables, which doesn't play well with
not-nullable columns.
2022-03-03 12:10:13 +01:00
Sybren A. Stüvel
bf4cc9b056 Add notice to our SQLite Gorm driver
The Gorm driver README now links to the original code, which was modified
to use a different SQLite library.
2022-03-03 11:52:43 +01:00
Sybren A. Stüvel
a7654394b4 Worker: update default task types to match what's currently in use
Flamenco v2 had a different classification of task types than Flamenco v3,
and the defaults still referenced the v2 classifications.
2022-03-03 11:51:53 +01:00
Sybren A. Stüvel
2610836861 Makefile: don't generate code when running make
Generated code is to be committed to Git anyway, so there is no need to
regenerate it on every build.

The code can be regenerated explicitly by running `make generate`.
2022-03-03 11:51:15 +01:00
Sybren A. Stüvel
efbbb8c7d8 Makefile: remove some unused parts
Some parts were still leftovers from other projects this Makefile was
copied from.
2022-03-03 11:50:03 +01:00
Sybren A. Stüvel
42daa87839 Makefile: convert generated files to native line-ends on Windows
Git wants to see native line-ends in source files, but the code generators
we use always write UNIX line-ends. `make generate` on Windows now passes
generated files through `unix2dos`. This allows regenerating files without
Git listing them as modified.
2022-03-03 11:49:26 +01:00
Sybren A. Stüvel
eaeea38e3f Worker: remove -manager URL CLI argument
This CLI arg isn't actually used by the subsequent code, so better to not
have it for now.
2022-03-03 11:33:22 +01:00
Sybren A. Stüvel
20fd3228ba Worker: better logging configuration
Always show startup notification at INFO level, regardless of configured
level. Replace `-verbose` with `-quiet`, add `-trace`.
2022-03-03 11:32:57 +01:00
Sybren A. Stüvel
954babd914 Manager: redirect web root to Swagger UI
Temporarily redirect the index page to the Swagger UI, so that at least
you can see something other than a "404 Not Found" error.
2022-03-01 20:49:11 +01:00
Sybren A. Stüvel
ddd0c7602b Change application name to "Flamenco 3"
"Flamenco NG PoC" was the name of the proof of concept, and it's no longer
in use.
2022-03-01 20:45:13 +01:00
Sybren A. Stüvel
47e36c927c Change package URL to the blender.org repository 2022-03-01 20:45:09 +01:00
Sybren A. Stüvel
e70a44a146 Manager: switch from PostgreSQL to SQLite
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
2022-03-01 18:50:31 +01:00
Sybren A. Stüvel
60d1710020 Addon: job type form seems to be working 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
5e7e2adef0 Addon: Fixed mypy errors 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
616784df0a Addon: start of framework for API communication 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
b678b90932 Proof of concept to test a generated Python OpenAPI client 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
bbc6a3f69e Tweak some logging 2022-03-01 17:28:24 +01:00
Sybren A. Stüvel
0235ffcb4a Manager: avoid "no record found" error in task scheduler
It's fine when there is no task for a worker, so having Gorm log an error
was just causing noise.
2022-03-01 11:52:28 +01:00
Sybren A. Stüvel
648a220098 Update FEATURES.md 2022-02-28 15:30:50 +01:00