Log any CLI command that's run in a way that can be easily copy-pasted
from the task log. This can help a lot in determining whether an issue
is caused by Flamenco or by the CLI program itself.
Rename the file containing the command executor from `command_exe.go` to
`cmd_executor.go`), to distinguish it from the implementation of the
`exec` command (`command_exec.go`).
No functional changes.
Now the Worker and the Manager share the same database initialisation
code (enabling foreign key constraints + write-ahead logging).
The foreign key constraints were already enabled before, but now it's done
with (a copy of) the same code as the Manager.
The tests were already made timezone-independent, but in a way that assumed
the day would still be the same and only the time would be different.
Now that assumption is removed as well.
The 'Simple Blender Render' job compiler script uses a `{timestamp}`
variable to determine the render output path. This variable is now set
to the local time, rather than UTC.
This fixes#104219: Unit tests are timezone-dependent
The solution uses Go `time.Local` timezone to satisfy unit tests
assertions using a Mock clock. The timezone of the local workstation
running the tests.
When the worker is started with `-restart-exit-code 47` or has
`restart_exit_code=47` in `flamenco-worker.yaml`, it's marked as
'restartable'. This will enable two worker actions 'Restart
(immediately)' and 'Restart (after task is finished)' in the Manager web
interface. When a worker is asked to restart, it will exit with exit
code `47`. Of course any positive exit code can be used here.
Change the package base name of the Go code, from
`git.blender.org/flamenco` to `projects.blender.org/studio/flamenco`.
The old location, `git.blender.org`, has no longer been use since the
[migration to Gitea][1]. The new package names now reflect the actual
location where Flamenco is hosted.
[1]: https://code.blender.org/2023/02/new-blender-development-infrastructure/
Initial implementation with some basic tests.
The API command only accept one source and destination path, which must
be absolute. If the destination already exists, it will not be
ovewritten, and an error is generated.
JS job definition code using this new command can easily handle list of
paths and/or relative paths if needed.
Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104220
To allow more build-time configuration:
- `Makefile` will now pick up `LDFLAGS` from environment variables, and
- locations of configuration files can now be overridden with linker
options.
These are not used for regular Flamenco builds, but do allow studios to
customize where configuration files are stored.
Review: https://projects.blender.org/studio/flamenco/pulls/104200
When a command (like `blender` or `ffmpeg`) outputs lines that are longer
than our buffer, they are broken into buffer-sized chunks. Extra code has
been added to ensure those chunks consist of valid UTF-8 characters.
Fix T100757 by reducing the log level to "info" when Blender writes output
to a file format the Worker cannot handle. Such cases are expected, and
now no longer result in an error message.
Split "executable" from "its arguments" in blender & ffmpeg commands.
Use `{blenderArgs}` variable to hold the default Blender arguments,
instead of having both the executable and its arguments in `{blender}`.
The reason for this is to support backslashes in the Blender executable
path. These were interpreted as escape characters by the shell lexer.
The shell lexer based splitting is now only performed on the default
arguments, with the result that `C:\Program Files\Blender
Foundation\3.3\blender.exe` is now a valid value for `{blender}`.
This does mean that this is backward incompatible change, and that it
requires setting up Flamenco Manager again, and that older jobs will not
be able to be rerun.
It is recommended to remove `flamenco-manager.yaml`, restart Flamenco
Manager, and reconfigure via the setup assistant.
There are Flamenco "commands" and CLI "commands", and it's nice to be
explicit about which is which. I'm sure this is needed in some other
areas as well.
Instead of sending the current process an interrupt signal, use a dedicated
channel to signal the wish to shut down. The main function responds to that
channel closing by performing the shutdown.
This solves an issue where the Worker would not cleanly shut down on
Windows when `offline` state was requested by the Manager.
Convert the database interface from the stdlib `database/sql` package to
the GORM object relational mapper.
GORM is also used by the Manager, and thus with this change both Worker
and Manager have a uniform way of accessing their databases.
Include `RELEASE_CYCLE` in the Makefile. This is mentioned at startup of
Manager and Worker, and reflects in the software version they report.
If `RELEASE_CYCLE == "release"`, Manager and Worker report their version
as `ApplicationVersion`. If it's any other string, the Git hash will get
appended.
The Worker now always waits for subprocesses. When faced with multiple
errors (like I/O reading from stdout and a returned error status from
the process) will return the most important one (in this case the exit
status of the process).
Subprocesses need to be waited for, even when they crashed, otherwise
they will linger around as "defunct" processes. This caused
out-of-memory errors, because several defunct Blenders were eating up
the memory.
Blender and FFmpeg were run in the same way, using copy-pasted code. This
is now abstracted away into the CLI runner, which in turn is moved into
its own subpackage.
No functional changes.
Remove the `{ffmpeg}` variable from the default configuration, and its use
from the job compiler scripts. Now that the Worker can find its bundled
FFmpeg, it's no longer needed to configure its location on the Manager.
Worker will now try one of the following paths, relative to the flamenco-worker
executable, in order to find FFmpeg. If they cannot be found, `$PATH` is
searched for FFmpeg.
- `tools/ffmpeg-$GOOS-$GOARCH`
- `tools/ffmpeg-$GOOS`
- `tools/ffmpeg`
On Windows these paths will have a `.exe` suffix appended. `$GOOS` is the
operating system, like "linux", "darwin", "windows", etc. `$GOARCH` is the
architecture, like "amd64", "386", etc.
By accident I made the worker load `flamenco-worker.yaml` from the "local
files" directory (~/.local/share/flamenco on Linux) instead of the current
directory. This was incorrect, as that file is meant to contain
configuration that's shared between workers.
Change the location where the Worker writes its local files so that it
follows the XDG specification (instead of writing to the current working
directory).
- Linux: `$HOME/.local/share/flamenco`
- Windows: `C:\Users\UserName\AppData\Local\Flamenco`
- macOS: `$HOME/Library/Application Support/Flamenco`
NOTE: The old files will not be loaded any more. This means that if
nothing is done and the new worker is run as-is, it will reregister as a
brand new worker. Move `flamenco-worker-credentials.yaml` and
`flamenco-worker.sqlite` to the new location to avoid this.
