Don't log the git commands in the version updater Go code. They are already
shown by the Makefile (`update-version` target), and we should only have
one set of those commands to maintain.
Include `RELEASE_CYCLE` in the Makefile. This is mentioned at startup of
Manager and Worker, and reflects in the software version they report.
If `RELEASE_CYCLE == "release"`, Manager and Worker report their version
as `ApplicationVersion`. If it's any other string, the Git hash will get
appended.
Blender and FFmpeg were run in the same way, using copy-pasted code. This
is now abstracted away into the CLI runner, which in turn is moved into
its own subpackage.
No functional changes.
This commit does not introduce functional changes, besides renaming
every mention of 'wizard' with 'setup assistant'. In order to run the
manager setup assistant use:
./flamenco-manager -setup-assistant
The change was introduced to favor more neutral and descriptive working
for this functionality. Thanks to Sybren for helping to get this done!
Logging the URLs at which the Manager can be reached as the last thing
when starting up, in the hope that this makes them more noticable and
inviting to actually visit.
Flamenco now no longer uses the Git tags + hash for the application
version, but an explicit `VERSION` variable in the `Makefile`.
After changing the `VERSION` variable in the `Makefile`, run
`make update-version`.
Not every part of Flamenco looks at this variable, though. Most
importantly: the Blender add-on needs special handling, because that
doesn't just take a version string but a tuple of integers. Running
`make update-version` updates the add-on's `bl_info` dict with the new
version. If the version has any `-blabla` suffix (like `3.0-beta0`) it
will also set the `warning` field to explain that it's not a stable
release.
Worker will now try one of the following paths, relative to the flamenco-worker
executable, in order to find FFmpeg. If they cannot be found, `$PATH` is
searched for FFmpeg.
- `tools/ffmpeg-$GOOS-$GOARCH`
- `tools/ffmpeg-$GOOS`
- `tools/ffmpeg`
On Windows these paths will have a `.exe` suffix appended. `$GOOS` is the
operating system, like "linux", "darwin", "windows", etc. `$GOARCH` is the
architecture, like "amd64", "386", etc.
Change the location where the Worker writes its local files so that it
follows the XDG specification (instead of writing to the current working
directory).
- Linux: `$HOME/.local/share/flamenco`
- Windows: `C:\Users\UserName\AppData\Local\Flamenco`
- macOS: `$HOME/Library/Application Support/Flamenco`
NOTE: The old files will not be loaded any more. This means that if
nothing is done and the new worker is run as-is, it will reregister as a
brand new worker. Move `flamenco-worker-credentials.yaml` and
`flamenco-worker.sqlite` to the new location to avoid this.
For development of the web interface, to get a less predictable order of
asynchronous requests, the API responses were artificially delayed. This
was supposed to be optional, to be enabled via the `-delay` CLI argument,
but somehow the optionalness either never made it in or was mysteriously
removed.
Add a `-pprof` CLI option to enable the profiler. It will expose profiler
info on the web interface at `/debug/pprof/`.
To have a nice view of this, including flame graphs, run:
```
go tool pprof -http localhost:8082 http://localhost:8080/debug/pprof/profile
```
Build with `make stresser`. Run with:
./stresser -worker UUID -secret ABCXYZ
The worker ID and secret can be obtained from
`flamenco-worker-credentials.yaml`. If left empty, the stresser will
register as a new worker, and log the credentials to be used on the next
invocation.
This adds a `-wizard` CLI option to the Manager, which opens a webbrowser
and shows the First-Time Wizard to aid in configuration of Flamenco.
This is work in progress. The wizard is just one page, and doesn't save
anything yet to the configuration.
Rather than just print the error message ("error creating UPnP/SSDP
server"), it now explains what the effect is of this error (workers
unable to automatically find this Manager) and how to solve it (pass
`-manager URL` to the Worker).
The task logs storage system is refactored to use the `local_storage`
package. Configuration options have also changed:
- `task_logs_path` is renamed to `local_manager_storage_path`, to
emphasise that only the Manager deals with those files, with default
value `./flamenco-manager-storage`.
- `storage_path` is renamed to `shared_storage_path`, to emphasise this
is the storage shared between Manager and Workers, with default value
`./flamenco-shared-storage`.
Task logs are still stored in
`${local_manager_storage_path}/job-{jobUUID[0:4]}/{jobUUID}/task-{taskUUID}.txt`
Manifest task: T99409
When installing, Blender will just unzip directly into the addons dir,
so the ZIP has to contain the `flamenco` package directory.
This also makes things simpler, naming-wise. We can offer the addon from
the Manager web interface as `flamenco3-addon.zip`, and still have it
install into the `addons/flamenco` directory.
This makes it possible to start Flamenco Worker at Blender Studio with
a worker-local current working directory, with the executable in a shared
filesystem.
`os.IsNotExist()` is from before `errors.Is()` existed. The latter is the
recommended approach, as it also recognised wrapped errors.
No functional changes, except for recognising more cases of "does not
exist" errors as such.
Vue Router generates URLs for which there are no static files on the
filesystem (like `/jobs/{job ID}`). To make this work, the webapp's
`index.html` has to be served for such requests. The client-side JavaScript
then figures out how things fit together, and can even render a nice 404
page if necessary.
This shouldn't happen for non-webapp URLs, though. Because of this, the
entire webapp (including the "serve `index.html` if file not found logic)
is moved to a `/app/` base URL.
`make flamenco-manager` now also builds the webapp and embeds the static
files into the binary.
`make flamenco-manager_race` does NOT rebuild the static web files, to
help speed up of debug cycles. Run `make webapp-static` to rebuild the
webapp itself, if necessary, or run a separate web development server with
`yarn --cwd web/app run dev --host`.
Add a handler for the OpenAPI `taskOutputProduced` operation, and an
image thumbnailing goroutine.
The queue of images to process + the function to handle queued images
is managed by `last_rendered.LastRenderedProcessor`. This queue currently
simply allows 3 requests; this should be improved such that it keeps
track of the job IDs as well, as with the current approach a spammy job
can starve the updates from a more calm job.
The OpenAPI library we use for request validation needs to know per mime
type how to handle the contents. The same function for
`application/octet-stream` is now used for `image/png` and `image/jpeg`
as well.
Change "accepted CORS origins" to "acceptable CORS origins", as the former
is too ambiguous (it can mean "I just accepted these" or "These are the
acceptable ones").
Within the shutdown procedure, signing off is now the last thing the
worker does. This makes things more consistent from the Manager's point
of view (like receiving last-second log entries while the Worker is still
online).
Requeueing the tasks of a specific worker is now done in the
`TaskStateMachine`, such that it can be called from other services as
well in future commits.
This also makes the `LogStorage` service a dependency of the
`TaskStateMachine`, as it needs to write "this task was requeued" kind
of messages to the task logs.
Tasks that are in state `active` but haven't been 'touched' by a Worker
for 10 minutes or longer will transition to state `failed`.
In the future, it might be better to move the decision about which state
is suitable to the Task State Machine service, so that it can be smarter
and take the history of the task into account. Going to `soft-failed`
first might be a nice touch.
In the future different services will write to the task log, and thus
it makes sense to move the responsibility of prepending the timestamps
to the log storage service.
add `-delay` CLI argument, which adds a random delay of around 250ms to
all HTTP responses.
The web interface is quite asynchronous in nature, and having more
randomness and more visible delays in there will help development.
This CLI argument should not be used in production.
