Add a function `shellSplit(string)` to the global namespace of job
compiler scripts. It splits a string into an array of strings using
shell/CLI semantics.
For example: `shellSplit("--python-expr 'print(1 + 1)'")` will return
`["--python-expr", "print(1 + 1)"]`.
Back in the days when I wrote the code, I didn't know about the
`require` package yet. Using `require.NoError()` makes the test code
more straight-forward.
No functional changes, except that when tests fail, they now fail
without panicking.
When a more complex list of frames is to be rendered (like `1, 4, 5, 10,
15`), simplify the video filename to `{first}-{last}`.
Before: `somename-1, 4, 5, 10, 15.mp4`
Now: `somename-1-15.mp4`
Fix a bug in the translation from 'human' frame ranges ('A-B,C-D') to
'Blender' frame ranges ('A..B,C..D'). It required calling
`string.replaceAll()` instead of `string.replace()` in the job compiler
script, which in turn required an upgrade of the JavaScript engine Goja.
The 'Simple Blender Render' job compiler script uses a `{timestamp}`
variable to determine the render output path. This variable is now set
to the local time, rather than UTC.
This fixes#104219: Unit tests are timezone-dependent
The solution uses Go `time.Local` timezone to satisfy unit tests
assertions using a Mock clock. The timezone of the local workstation
running the tests.
Change the package base name of the Go code, from
`git.blender.org/flamenco` to `projects.blender.org/studio/flamenco`.
The old location, `git.blender.org`, has no longer been use since the
[migration to Gitea][1]. The new package names now reflect the actual
location where Flamenco is hosted.
[1]: https://code.blender.org/2023/02/new-blender-development-infrastructure/
Adjustment of the job settings property names, making them more widely
usable than just the 'use automatic values' button (also known as
'eval-on-submit').
Improve the usability of the 'eval-on-submit' toggle button:
- Add a placeholder text that can be shown instead of the input field. This
can be used to describe what the evaluated Python code will do. In the
case of the 'Simple Blender Render' job, this is set to 'Scene frame
range'.
- Change the way in which the job type has to declare this, both for
clarity and to add the extra placeholder string
Add a new job setting option `autoevalLockable`. Setting this to `true` in
the job compiler's `JOB_TYPE` settings has the following effect:
- By default, the setting will not be editable in Blender's job submission
interface. Instead, a toggle button with a 'car' icon will be shown.
- When the 'car' button is toggled off, the setting becomes editable again.
In its default, uneditable state, the setting will be auto-evaluated before
submission.
This makes it possible to 'lock in' auto-evaluation. The main use case is
for the frame range of the render job. By default this will be locked to
the scene frame range, but it can still be overridden if a different
range is wanted.
As it was decided that the name "tags" would be better for the clarity
of the feature, all files and code named "cluster" or "worker cluster"
have been removed and replaced with "tag" and "worker tag". This is only
a name change, no other features were touched.
This addresses part of #104204.
Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104223
As a note to anyone who already ran a pre-release version of Flamenco
and configured some worker clusters, with the help of an SQLite client
you can migrate the clusters to tags. First build Flamenco Manager and
start it, to create the new database schema. Then run these SQL queries
via an sqlite commandline client:
```sql
insert into worker_tags
(id, created_at, updated_at, uuid, name, description)
select id, created_at, updated_at, uuid, name, description
from worker_clusters;
insert into worker_tag_membership (worker_tag_id, worker_id)
select worker_cluster_id, worker_id from worker_cluster_membership;
```
If Shaman is used to submit the job files, store the job's checkout ID
(i.e. the path relative to the checkout root) in the database. This will
make it possible in the future to remove the Shaman checkout along with
the job itself.
The preview video task would attempt to use JPEG preview files when the
"Preview" checkbox is checked, even when this checkbox is not shown in
Blender's UI (when the output format is not EXR). Blender still stores
this property, even when it's unused, and that's what tripped up the job
compiler.
The "Simple Blender Render" job type now first checks whether the previews
are necessary at all, and only then uses them.
The `require.XXX` functions are exactly the same as `assert.XXX`
functions + directly failing the test, so this refactor simplifies the
code quite a bit. Can be done in more areas than this.
No functional changes.
Simple Blender Render now no longer renders to an intermediate directory.
This not only simplifies the script, but it also opens the door for
selective re-running of individual tasks.
In the old situation, where the intermediate directory was renamed to
the desired name in the last task, rerunning tasks would fail because the
directory they expect to exist no longer exists. This is now resolved.
The original idea behind this job type was that it would work equally
well for videos as for images, but that was never really well tested.
It's currently broken, so this commit removes video support altogether.
Remove the `blender_cmd` setting, and just hard-code it to `{blender}`.
The Blender add-on was already passing this string, and it's very unlikely
that people are already writing custom add-ons to pass something different.
It provided flexibility that was untested, so it's better to simplify
things.
Split "executable" from "its arguments" in blender & ffmpeg commands.
Use `{blenderArgs}` variable to hold the default Blender arguments,
instead of having both the executable and its arguments in `{blender}`.
The reason for this is to support backslashes in the Blender executable
path. These were interpreted as escape characters by the shell lexer.
The shell lexer based splitting is now only performed on the default
arguments, with the result that `C:\Program Files\Blender
Foundation\3.3\blender.exe` is now a valid value for `{blender}`.
This does mean that this is backward incompatible change, and that it
requires setting up Flamenco Manager again, and that older jobs will not
be able to be rerun.
It is recommended to remove `flamenco-manager.yaml`, restart Flamenco
Manager, and reconfigure via the setup assistant.
Add a new `abspath(path)` function to the add-on, for use in job type
settings. With this, the "simple blender render" job can support relative
paths for the "render output root" setting, and still have an absolute
final "render output path".
The etag prevents job submissions with old settings, when the job
compiler script has been edited. The etag is the SHA1 hash of the
`JOB_TYPE` dictionary (as defined by the JavaScript file). The hash is
computed in a way that's independent of the exact formatting in the
JavaScript file. Also the actual JS code itself is irrelevant, just the
`JOB_TYPE` dictionary is used.
Blender cannot be told to only allow absolute path for an RNA property
(of type `string`, subtype `dir_path`), so as a workaround the final
`render_output_path` is now using `bpy.path.abspath()` to make the path
absolute.
This has as advantage that the render output path can be defined by artists
as a blendfile-relative path, and that it'll be resolved when submitting
the blend file.
Remove the `{ffmpeg}` variable from the default configuration, and its use
from the job compiler scripts. Now that the Worker can find its bundled
FFmpeg, it's no longer needed to configure its location on the Manager.
In the `simple-blender-render` job type settings, hide the `chunk_size`
setting from the web frontend, and show the `blendfile` setting instead.
The actual blend file being rendered is important to know, whereas the
chunk size can be inferred from the task names anyway.
`os.IsNotExist()` is from before `errors.Is()` existed. The latter is the
recommended approach, as it also recognised wrapped errors.
No functional changes, except for recognising more cases of "does not
exist" errors as such.
If there is a `scripts` directory next to the current executable, load
scripts from that directory as well.
It is still required to restart the Manager in order to pick up changes
to those scripts (including new/removed files), PLUS a refresh in the
add-on.
When on-disk job compiler scripts are supported, they will be reloaded
often, and it becomes more important to have the access to the map of
loaded job compilers thread-safe.
The global `scriptFS` variable was too easy to access, which caused an
issue where the mandatory `"scripts"` subdirectory was not passed.
Accessing via a getter function that hides this requirement prevents this.
Take some functions out of the `Service` struct, as they are more or less
standalone anyway. This will also make it easier later to make things
thread-safe, as that'll become important when files can get live-reloaded.
The `Load()` function returns a `*Service`, and it was confusing that the
local variable is named `compiler` instead. Now it's called `service`.
No functional changes.
Refactor the JS script file loading code so that it's tied to the `fs.FS`
interface for longer, and less to the specifics of our `embed.FS` instance.
This should make it possible to use other filesystems, like a real on-disk
one, to load scripts.
Add a small wrapper around github.com/google/uuid. That way it's clearer
which functionality is used by Flamenco, doesn't link most of the code to
any specific UUID library, and allows a bit of customisation.
The only customisation now is that Flamenco is a bit stricter in the
formats it accepts; only the `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx` is
accepted. This makes things a little bit stricter, with the advantage
that we don't need to do any normalisation of received UUID strings.
Replace the Vue v2 webapp with a Vue v3 one, and embed the OpenAPI
client in the webapp itself (instead of being its own npm project).
- Vue v2.x -> v3.x
- Tabulator v4.x -> v5.1
- Moment JS -> replaced with Luxon JS
- Vue CLI/UI -> replaced with Vite
