192 Commits

Author SHA1 Message Date
Sybren A. Stüvel
09946c0894 Worker: use bundled FFmpeg if available
Worker will now try one of the following paths, relative to the flamenco-worker
executable, in order to find FFmpeg. If they cannot be found, `$PATH` is
searched for FFmpeg.

- `tools/ffmpeg-$GOOS-$GOARCH`
- `tools/ffmpeg-$GOOS`
- `tools/ffmpeg`

On Windows these paths will have a `.exe` suffix appended. `$GOOS` is the
operating system, like "linux", "darwin", "windows", etc. `$GOARCH` is the
architecture, like "amd64", "386", etc.
2022-07-22 16:37:14 +02:00
Sybren A. Stüvel
a5940a24f0 Worker: load flamenco-worker.yaml from current directory
By accident I made the worker load `flamenco-worker.yaml` from the "local
files" directory (~/.local/share/flamenco on Linux) instead of the current
directory. This was incorrect, as that file is meant to contain
configuration that's shared between workers.
2022-07-22 16:37:14 +02:00
Sybren A. Stüvel
d553ca5ab9 Worker: pass input frame rate to FFmpeg when converting frames to video
FFmpeg needs the input frame rate as well, otherwise it'll default to 25
FPS, and mysteriously drop frames when rendering a 24 FPS shot.
2022-07-19 18:43:06 +02:00
Sybren A. Stüvel
2f76df437b T99415: Worker: change default location for writing local files
Change the location where the Worker writes its local files so that it
follows the XDG specification (instead of writing to the current working
directory).

- Linux:   `$HOME/.local/share/flamenco`
- Windows: `C:\Users\UserName\AppData\Local\Flamenco`
- macOS:   `$HOME/Library/Application Support/Flamenco`

NOTE: The old files will not be loaded any more. This means that if
nothing is done and the new worker is run as-is, it will reregister as a
brand new worker. Move `flamenco-worker-credentials.yaml` and
`flamenco-worker.sqlite` to the new location to avoid this.
2022-07-19 12:08:41 +02:00
Sybren A. Stüvel
fa600d6fc9 Cleanup: rename mustHostname() to workerName()
The function determines the worker's name. The fact that it can use the
hostname for this isn't that relevant.
2022-07-19 12:03:08 +02:00
Sybren A. Stüvel
47e517a3a5 Worker: cleanly sign off after flushing buffer
When running the Worker with the `-flush` CLI argument, actually sign off
from the Manager before shutting down.
2022-07-18 16:36:45 +02:00
Sybren A. Stüvel
e91623557a Worker: log which URLs were tried when auto-discovery failed
When the Worker cannot find any Manager, log which URLs were tried.
2022-07-18 14:14:02 +02:00
Sybren A. Stüvel
0e92004f2a OAPI: regenerate code 2022-07-16 15:59:48 +02:00
Sybren A. Stüvel
726129446d T99730: Allow access to full task log
The web interface has a button that opens the task log in a new window.
This might need some restyling ;-)
2022-07-16 12:55:41 +02:00
Sybren A. Stüvel
e2434b44f2 OAPI: regenerate code 2022-07-16 11:11:34 +02:00
Sybren A. Stüvel
35fe0146d3 Add stress tester for task updates
Build with `make stresser`. Run with:

  ./stresser -worker UUID -secret ABCXYZ

The worker ID and secret can be obtained from
`flamenco-worker-credentials.yaml`. If left empty, the stresser will
register as a new worker, and log the credentials to be used on the next
invocation.
2022-07-15 15:08:00 +02:00
Sybren A. Stüvel
f9a3d3864a OAPI: regenerate code 2022-07-14 17:26:26 +02:00
Sybren A. Stüvel
aec5ee49e0 First-Time Wizard: allow selecting Blender executables
The wizard now finds Blender in various ways, and lets the user select
which one to use.

Doesn't save anything yet, though.
2022-07-14 12:22:56 +02:00
Sybren A. Stüvel
20f13257f7 Move "blender finder" from Worker-specific to common location
Manager's first-time wizard will have to be able to find Blender as well.
2022-07-14 11:17:03 +02:00
Sybren A. Stüvel
e4a38f071c OAPI: regenerate code 2022-07-14 11:16:59 +02:00
Sybren A. Stüvel
ce250a611e Windows: fix error handling of syscall to AssocQueryStringW
syscall.SyscallN returns a `uintptr` type alias, and thus has to be
compared to `0`, not `nil`. Yeah, it's a bit weird.
2022-07-13 11:48:26 +02:00
Sybren A. Stüvel
ae2cb281b4 OAPI: regenerate code 2022-07-08 11:35:57 +02:00
Sybren A. Stüvel
2965856aa3 Worker: add test flag to enable Blender-dependent test
Add a `-withBlender` CLI argument for a unit test, to aid in debugging
T99438.

Run the test with `go test ./internal/worker/find_blender/ -args -withBlender`
to actually fail when the file association with `.blend` files cannot be
found.

Note that this doesn't rely on Blender being runnable, but it does rely
on _something_ being associated with .blend files.
2022-07-05 10:01:10 +02:00
Sybren A. Stüvel
60971722fc Windows: add missing imports
A recent refactor (rFfb89658530da25a77dc03fb329c394198bf6358f) performed
on Linux didn't properly update a Windows-only file.
2022-07-05 10:01:10 +02:00
Sybren A. Stüvel
f2f8357df7 Bump thumbnail JPEG quality from 80 to 85
80 was a bit too low. 85 might still be too low, we'll have to see.
2022-07-01 17:44:26 +02:00
Sybren A. Stüvel
801fa20f12 OAPI: regenerate code 2022-07-01 12:32:42 +02:00
Sybren A. Stüvel
668e25fe95 OAPI: regenerate code 2022-06-28 17:07:08 +02:00
Sybren A. Stüvel
24344e9632 Cleanup: worker, simplify setting the manager URL
The return value of `FileConfigWrangler.SetManagerURL()` was never used,
so now the function doesn't return anything any more.
2022-06-28 11:42:47 +02:00
Sybren A. Stüvel
d6cfff4031 Worker: treat empty config file the same as a missing one
EOF while parsing the config file is now handled as an indication that
the default config should be used, rather than a fatal error.
2022-06-28 10:24:46 +02:00
Sybren A. Stüvel
fb89658530 Refactor: replace os.IsNotExist() with errors.Is(err, fs.ErrNotExist()
`os.IsNotExist()` is from before `errors.Is()` existed. The latter is the
recommended approach, as it also recognised wrapped errors.

No functional changes, except for recognising more cases of "does not
exist" errors as such.
2022-06-28 10:24:46 +02:00
Sybren A. Stüvel
1353d1df0f OAPI: regenerate code 2022-06-27 11:32:12 +02:00
Sybren A. Stüvel
2d6c11e98b Worker: send produced output to Manager
Workers now send output produced by Blender (limited to PNG and JPEG
images, currently) to Manager. This is done by converting to JPEG first,
then sending the bytes via the Flamenco API to the Manager.
2022-06-27 11:30:37 +02:00
Sybren A. Stüvel
34f1cc076c Cleanup: Worker, simplify Listerer.Run() function
No functional changes, except that now the "listener shutting down"
message will also be logged in case of a panic.
2022-06-27 11:30:37 +02:00
Sybren A. Stüvel
f244355328 Worker: parse stdout of Blender to recognise saved files
Prepare the Worker for submission of last-rendered images to Manager, by
parsing `stdout` of Blender to see which files were saved.

This needs more work, as now just an error "not implemented" is logged.
2022-06-27 11:30:37 +02:00
Sybren A. Stüvel
1f8c2df919 Worker: skip sometimes-hanging unit test
The test can hang occasionally, and needs some love & attention. For now
I've done some patching to make it slightly better, but still disabled it
and added a `FIXME` note to it.
2022-06-27 11:30:35 +02:00
Sybren A. Stüvel
167b2eaf45 OAPI: regenerate code 2022-06-24 16:39:50 +02:00
Sybren A. Stüvel
2a4c9b2c13 Worker: enable SQLite foreign keys
They're not used now, but enabling them is good default behaviour anyway.
2022-06-20 13:00:49 +02:00
Sybren A. Stüvel
7327896db9 Worker: allow overriding worker name from environment
Allow overriding the worker name by setting the `FLAMENCO_WORKER_NAME`
environment variable. This makes it easy to do from Docker configs, and,
more importantly, from the scripts I use to run multiple workers on the
same machine while developing Flamenco.
2022-06-17 16:24:03 +02:00
Sybren A. Stüvel
ec10128f85 Worker: Sleep command, return error when sleep time is negative
I need a way to reliably generate task errors, and having a more thorough
check on the sleep duration parameter seemed a nice way to create those.
2022-06-16 15:46:03 +02:00
Sybren A. Stüvel
d5d0893b05 Worker: use explicit types for command parameter errors
Introduce `ParameterMissingError` and `ParameterInvalidError` structs, to
be returned from command executors. These replace free-form `fmt.Errorf()`
style errors.
2022-06-16 15:45:09 +02:00
Sybren A. Stüvel
8af1b9d976 Worker: fix sync issue in TestUpstreamBufferManagerUnavailable unit test
Fix synchronisation/goroutine issue in the "upstream buffer" test,
where very occasionally the queue size was checked at the wrong time.
2022-06-16 15:43:20 +02:00
Sybren A. Stüvel
da1b42f9fa Worker: fix sqlite connection issue in unit tests
Fix sqlite issues in the "upstream buffer" test. The test used
`:memory:` to have an in-memory DB to separate from other tests. The
"flush at shutdown" code runs in a different goroutine, though, and
creates a new DB connection. The SQLite separation was too strong,
making that function not find any tables. This is now solved by having
an in-memory database that's shared between all connections made from
the same unit test.
2022-06-16 15:42:52 +02:00
Sybren A. Stüvel
7e28cfa69c Worker: add task failures to the task log as well
Task failures were only placed in the task's activity field, and are now
added to the log as well.
2022-06-16 12:22:05 +02:00
Sybren A. Stüvel
e1309ad8fc Worker: flush upstream buffer when shutting down
When shutting down, the worker now tries to flush any buffered task updates
before closing.
2022-06-16 12:21:17 +02:00
Sybren A. Stüvel
9ddf72fa37 Worker: sign off as last step of shutdown
Within the shutdown procedure, signing off is now the last thing the
worker does. This makes things more consistent from the Manager's point
of view (like receiving last-second log entries while the Worker is still
online).
2022-06-16 12:19:03 +02:00
Sybren A. Stüvel
5bc94101e8 Worker: Avoid sleep at shutdown
Make the sleep between fetching tasks interruptable, so that a shutdown
doesn't have to wait a few seconds.
2022-06-16 12:08:13 +02:00
Sybren A. Stüvel
9ab41984ac Adjust Go code for Nickname -> Name change
This fixes a bug where 'Worker undefined changed status' was logged in
the web interface, as that was (back then incorrectly) `workerupdate.name`.
Now that code is correct.
2022-06-16 11:03:18 +02:00
Sybren A. Stüvel
25d5b01b3c Cleanup: test errors with assert.NoError() instead of assert.Nil()
No functional changes, just nicer way to test.
2022-06-13 18:40:42 +02:00
Sybren A. Stüvel
132ce8f2ec Merge 'shutdown' and 'offline' states
Move the 'shutdown' state code to the 'offline' state, to match the
removal of the 'shutdown' state from the OpenAPI definition.
2022-06-02 16:35:07 +02:00
Sybren A. Stüvel
fdb0b82664 OAPI: regenerate code 2022-05-31 17:19:06 +02:00
Sybren A. Stüvel
f2f6726ec0 OAPI: regenerate code 2022-05-31 11:21:41 +02:00
Sybren A. Stüvel
eab4fa2ca7 OAPI: regenerate code 2022-05-30 18:52:00 +02:00
Sybren A. Stüvel
bb7ac8319f OAPI: regenerate code 2022-05-20 16:30:42 +02:00
Sybren A. Stüvel
c4cda79ec0 Worker: chunk logs at 10kB instead of 1kB
Send logs in bigger chunks, otherwise a Blender render can cause too many
individual requests.
2022-05-20 13:36:16 +02:00
Sybren A. Stüvel
d9a955beee Worker: only call may-I-keep-running endpoint every 10 seconds
For debugging it was nice to have this called every second, but for
production use that's a bit too frequent.
2022-05-20 12:57:27 +02:00