1779 Commits

Author SHA1 Message Date
Sybren A. Stüvel
e8ae0c3cf2 Website: Fix 'report a bug' link 2023-07-24 14:53:18 +02:00
Sybren A. Stüvel
1dd2e165c8 Webapp: show Manager name in the window title
Show the configured Manager name in the webapp browser window title. This
helps me to distinguish the production farm from my development install.
2023-07-22 13:44:02 +02:00
Sybren A. Stüvel
1ef9a81094 Cleanup: remove TODO
Remove a TODO (the work was done in cdf1cff4).
2023-07-22 13:40:38 +02:00
Sybren A. Stüvel
c7a709bbd3 Manager: change default name to 'Flamenco'
Change the default Manager name from "Flamenco Manager" to just "Flamneco".

The Manager is the central 'thing', and the web interface should just
show "Flamenco". There is no more Flamenco Server (which was a component
in Flamenco v2), and there is no other Flamenco component with a web
interface anyway.
2023-07-22 13:26:40 +02:00
Sybren A. Stüvel
3aeb7f6e48 Update CHANGELOG.md 2023-07-21 17:18:26 +02:00
Eveline Anderson
cdf1cff41b Fix #99410: SocketIO Reconnect Web Interface (#104235)
Fix #99410: Web: fetch version on SocketIO reconnect (and maybe reload)

After losing the SocketIO connection and subsequently reconnecting, the
webapp should re-fetch the Flamenco Manager version and display it in
the top-right corner. If it's different from before, then it will log a
notification about the upgrade and refresh the entire page to ensure the
new version is loaded properly.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104235
2023-07-21 17:16:49 +02:00
Sybren A. Stüvel
ef68f71d54 Manager: actually include manager name in version API call
The API call `GetVersion` should return the Manager name, but it returned
the hard-coded application name `"Flamenco"` instead.
2023-07-21 17:08:10 +02:00
Sybren A. Stüvel
ff2d24274c Webapp: delay useNotifs() call in socket-status.js
Instead of calling `useNotifs()` from the main body of `socket-status.js`,
defer that call until the notifications are actually used.

This decouples the two Pina stores at startup, making it possible to load
`socket-status.js` from `App.vue` without conflicts.
2023-07-21 17:02:48 +02:00
Sybren A. Stüvel
2f3712a077 CHANGELOG: add database integrity checks
Document that the checks are there, and that the periodic checking can
be configured.
2023-07-18 16:35:01 +02:00
Sybren A. Stüvel
63634361ce Manager: make periodic database integrity check configurable
Instead of always performing the periodic integrity check, make it possible
to disable it or run it at different intervals.

Currently for the Blender Studio it's crunch time, so the check should
really only run when there is someone looking at the system (i.e. at
restarts for upgrade purposes).
2023-07-18 16:33:01 +02:00
Sybren A. Stüvel
1a79c19583 Manager: improve logging of database consistency checks
The log messages now all start with `database: `.

No functional changes.
2023-07-18 16:12:26 +02:00
Sybren A. Stüvel
4121c899c3 Manager: perform database integrity check every hour
Perform a database integrity check every hour. This check was already
performed at startup, in the main goroutine.
2023-07-18 16:10:17 +02:00
Sybren A. Stüvel
5eb57427fc Manager: better logging of schedule changes
Log more details of schedule changes, from within the sleep scheduler
(instead of the API implementation).
2023-07-18 15:55:51 +02:00
Sybren A. Stüvel
3def3d2260 Worker: make -version CLI option show the 'extended' version
`flamenco-worker -version` now shows the 'extended' version. This is just
the application version when running a release build. Otherwise it will
be like `3.3-alpha0 (v3.2-nn-ghhhhhh)`, where `nn` is the number of commits
since the last-tagged version (`v3.2` in this example), and `ghhhhhh` is
the Git hash. This can optionally be suffixed by `-dirty` if there were
any uncommitted changes when Flamenco was built.
2023-07-13 15:38:39 +02:00
Sybren A. Stüvel
ebd40dbcd5 Update CHANGELOG 2023-07-13 15:19:44 +02:00
Sybren A. Stüvel
b24d748840 Add-on: use the 'eval' description in the 'eval now' button as well
In the tooltip of the 'evaluate now' button (the one with the Python
icon), include the description provided by the job type in its
`evalInfo.description` field. This makes it explicit what the button
will do when pressed.
2023-07-13 15:14:39 +02:00
Sybren A. Stüvel
6994413ed6 Use new property names for 'use automatic values' button
Adjustment of the job settings property names, making them more widely
usable than just the 'use automatic values' button (also known as
'eval-on-submit').
2023-07-13 15:02:00 +02:00
Sybren A. Stüvel
4b53c06467 OAPI: regenerate code 2023-07-13 15:00:34 +02:00
Sybren A. Stüvel
6686ba3610 OAPI: yet another refactor of 'eval-on-submit' behavior for job settings
The data is still the same, but the names of the properties have changed
a bit so that they're more generic, declarative, instead of specific to
one bit of functionality.

The goal is to have the `evalInfo.description` field usable for the
'evaluate now' button in the add-on as well. That way it should be
clearer what that does.

This commit just updates the OpenAPI definition.
2023-07-13 14:59:22 +02:00
Sybren A. Stüvel
242ed119ab Eval-on-submit job settings: improve GUI in the add-on
Improve the usability of the 'eval-on-submit' toggle button:

- Add a placeholder text that can be shown instead of the input field. This
  can be used to describe what the evaluated Python code will do. In the
  case of the 'Simple Blender Render' job, this is set to 'Scene frame
  range'.
- Change the way in which the job type has to declare this, both for
  clarity and to add the extra placeholder string
2023-07-13 14:44:24 +02:00
Sybren A. Stüvel
ab399594c7 OAPI: regenerate code 2023-07-13 14:42:07 +02:00
Sybren A. Stüvel
886f21e958 OAPI: refactor how 'eval-on-submit' settings are configured
Refactor the job settings. The `autoevalLockable` boolean is now
replaced with an `evalOnSubmit` nested object:

```
  evalOnSubmit: {
    showButton: true,
    placeholder: "Scene frame range",
  },
```

This makes it possible to add a placeholder text, and later maybe some
other parameters. The `showButton: true` part always has to be there, as
the entire feature is disabled with `showButton: false`, in which case
it's better to just remove the entire `evalOnSubmit` sub-object
altogether. Still, I think it's preferred to have that `showButton:
true` in there, as it makes it more explicit what this section of the
settings is for.

This commit just contains the OpenAPI definition.
2023-07-13 14:39:29 +02:00
Sybren A. Stüvel
8f9fddd512 Add-on: silence mypy error
Mypy doesn't know the `bl-rna` property exists.

No functional changes.
2023-07-13 12:56:03 +02:00
Sybren A. Stüvel
33aa44e623 Add-on: remove accidental debug print 2023-07-13 12:55:11 +02:00
Sybren A. Stüvel
3a3e664ae2 Add option to make a job setting auto-evaluatable by the user
Add a new job setting option `autoevalLockable`. Setting this to `true` in
the job compiler's `JOB_TYPE` settings has the following effect:

- By default, the setting will not be editable in Blender's job submission
  interface. Instead, a toggle button with a 'car' icon will be shown.
- When the 'car' button is toggled off, the setting becomes editable again.

In its default, uneditable state, the setting will be auto-evaluated before
submission.

This makes it possible to 'lock in' auto-evaluation. The main use case is
for the frame range of the render job. By default this will be locked to
the scene frame range, but it can still be overridden if a different
range is wanted.
2023-07-13 12:07:02 +02:00
Sybren A. Stüvel
168305f785 OAPI: regenerate code 2023-07-13 12:06:21 +02:00
Sybren A. Stüvel
553fffa21e OAPI: add option to make a job setting auto-evaluatable by the user
Add a new job setting option `autoevalLockable`. Setting this to `true` in
the job compiler's `JOB_TYPE` settings has the following effect:

- By default, the setting will not be editable in Blender's job submission
  interface. Instead, a toggle button with a 'car' icon will be shown.
- When the 'car' button is toggled off, the setting becomes editable again.

In its default, uneditable state, the setting will be auto-evaluated before
submission.

This makes it possible to 'lock in' auto-evaluation. The main use case is
for the frame range of the render job. By default this will be locked to
the scene frame range, but it can still be overridden if a different
range is wanted.

This commit just contains the necessary OpenAPI change.
2023-07-13 12:00:29 +02:00
Sybren A. Stüvel
498a71f00c .gitattributes: explicitly denote specific file extensions as 'text'
Configure auto-clrf for all text files. This should make it simpler to
develop Flamenco on Windows, as it makes Git depend less on its global
settings.
2023-07-13 11:55:41 +02:00
Sybren A. Stüvel
a5e405de16 Web: more SocketIO documentation
Add that the OpenAPI specs also define the structures used for SocketIO
communication.
2023-07-13 10:13:24 +02:00
Sybren A. Stüvel
b83b929e01 Web: add some technical details to the socketIO documentation
Add technical details to explain which part of the SocketIO-based broadcast
system is implemented where in the source code.
2023-07-13 10:08:34 +02:00
Michael Cook
520102627d Worker: Point users to docs if worker cannot find Blender
When the Worker cannot find Blender when it starts, provide a more
helpful message + a link to the relevant documentation.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104230
2023-07-10 15:09:06 +02:00
Sybren A. Stüvel
effac1c159 Git-ignore .vscode/launch.json 2023-07-10 14:10:49 +02:00
Sybren A. Stüvel
b58f1e15f1 Add CLI utility to recreate tasks of jobs
Due to an issue (which has been fixed in the previous commit), all tasks
in the database were deleted when starting Flamenco. This tool attempts
to recompile the job and recreate its tasks.

The statuses of the tasks are set based on the job status. Basically:

- job active → tasks queued
- job completed → tasks completed
- job cancelled / failed → tasks cancelled
- otherwise → tasks queued

To ensure that the tool is only used to create tasks from scratch, it
refuses to work on a job that still has tasks in the database.
2023-07-10 14:10:15 +02:00
Sybren A. Stüvel
06738b8aa4 Manager: disable SQLite foreign key constraints when migrating the database
There is an issue with the GORM auto-migration, in that it doesn't
always disable foreign key constraints when it should. Due to
limitations of SQLite, not all 'alter table' commands you'd want to use
are available. As a workaround, these steps are performed:

1. create a new table with the desired schema,
2. copy the data over,
3. drop the old table,
4. rename the new table to the old name.

Step #3 will wreak havoc with the database when foreign key constraint
checks are active, so no we temporarily deactivate them while performing
database migration.
2023-07-10 14:06:21 +02:00
Sybren A. Stüvel
60d54eabb3 Manager: avoid recreation of Worker table at startup
Mark the default value of `Worker.LazyStatusRequest` as `false`. The
previous default was configured as `0`, which was different enough to
always trigger a database migration of that column. However, since these
values do map to each other, the migration didn't do anything concrete,
and would be triggered again at the next startup.
2023-07-10 13:56:47 +02:00
Sybren A. Stüvel
6a170a3ae4 Deployment: make backup of SQLite database before every deploy
Every time Flamenco Manager is shut down in order to upgrade, a database
copy is made.
2023-07-10 12:42:15 +02:00
Eveline Anderson
830c3fe794 Rename worker 'clusters' to 'tags'
As it was decided that the name "tags" would be better for the clarity
of the feature, all files and code named "cluster" or "worker cluster"
have been removed and replaced with "tag" and "worker tag". This is only
a name change, no other features were touched.

This addresses part of #104204.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104223

As a note to anyone who already ran a pre-release version of Flamenco
and configured some worker clusters, with the help of an SQLite client
you can migrate the clusters to tags. First build Flamenco Manager and
start it, to create the new database schema. Then run these SQL queries
via an sqlite commandline client:

```sql
insert into worker_tags
    (id, created_at, updated_at, uuid, name, description)
  select id, created_at, updated_at, uuid, name, description
  from worker_clusters;

insert into worker_tag_membership (worker_tag_id, worker_id)
  select worker_cluster_id, worker_id from worker_cluster_membership;
```
2023-07-10 11:11:03 +02:00
Eveline Anderson
341dc6c8e2 OAPI: regenerate code 2023-07-10 11:08:44 +02:00
Eveline Anderson
df51561640 OAPI: rename worker 'cluster' to 'tag'
This implements the OpenAPI changes necessary for the rename.

See #104204

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104223
2023-07-10 11:08:39 +02:00
Sybren A. Stüvel
44ec93275d Cleanup: reformat all Go files
Run `go fmt` on all files, to fix their formatting.

No functional changes.
2023-07-10 10:58:14 +02:00
Michael Cook
b20ede97ea Shaman: fail unit test when running as root user
If the mock tests are run by root user then this specific test of
inaccessible path fails because root can write files to anywhere on the
filesystem. It is not clear that Flamenco Manager test
TestCheckSharedStoragePath is checking inaccessible file locations when
it fails and that it should be run by an unprivileged user.

Fix is to fail the permission test if the tests are run as a root user.
2023-07-07 16:05:43 +02:00
Sybren A. Stüvel
7a508c7e6b Manager: perform database integrity check at startup
Perform these two SQL calls & check their result:

- `PRAGMA integrity_check`
- `PRAGMA foreign_key_check`:

See  https: //www.sqlite.org/pragma.html for more info on these.

This also removes the unused `PeriodicMaintenanceLoop()` function.
Periodic checking while Flamenco Manager is running might be introduced
in a future commit, after the startup-time checks have been shown to not
get in the way.
2023-07-07 16:03:06 +02:00
Sybren A. Stüvel
7f588e6dbc Manager: close database connection on startup errors
When there is an error detected at startup, close the database connection.
Before, the connection could be kept open even when an error was returned,
causing the write-ahead log files to be kept around. These are now
properly integrated into the main database file before exiting.
2023-07-07 15:48:08 +02:00
Sybren A. Stüvel
2bc6c77e49 Deploy: move old worker executable before overwriting
Move the old worker out of the way. This should keep the old executable
as-is on disk, hopefully keeping currently-running processes happy.

I've noticed some workers crashing as soon as the executable got
replaced, which made me suspect that it's not 100% loaded to memory
before starting the execution. Let's see if this works better.
2023-07-07 14:09:38 +02:00
Sybren A. Stüvel
988cdf61ff Upgrade GORM & SQLite
Upgrade:
- `gorm.io/gorm` v1.23.8 → 1.25.2
- `github.com/glebarez/go-sqlite` v1.17.3 → v1.8.0
- `github.com/glebarez/sqlite` v1.4.6 → v1.8.0

and also some indirect dependencies.

This is in the hope that some weird cases at Blender Studio get resolved.
It appears that sometimes, for some unknown reason, when deleting a job,
its tasks get reassigned to another job (instead of also getting deleted).

Since there is no code in Flamenco itself to do this task deletion (it's
all depending on SQLite following the foreign keys and cascading to tasks),
I hope it was a bug in either GORM or SQLite that got fixed at some point.
2023-07-06 16:08:57 +02:00
Sybren A. Stüvel
22f4aa09f3 Manager: expand job deletion unit test
Add extra job to the database before deleting one, to ensure that job
deletion doesn't do anything with other jobs (and their tasks).

No functional changes to Flamenco itself.
2023-07-06 16:08:57 +02:00
Michael Cook
5657f5e1fe Manager: Fix logURLs possble typo
Change "possble URL" to "possible URLs".

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104231
2023-07-06 12:43:40 +02:00
Sybren A. Stüvel
6a30f844eb Manager: Better reporting of version via API call
Before: `3.3-alpha0-v3.2-76-gdd34d538-dirty`
After : `3.3-alpha0 (v3.2-76-gdd34d538-dirty)`

Also include the new `git` property to always have the Git hash (the part
between parentheses).
2023-07-06 12:21:47 +02:00
Sybren A. Stüvel
82456424c9 OAPI: regenerate code 2023-07-06 12:19:48 +02:00
Sybren A. Stüvel
c6dcf6d7bd OAPI: better reporting of version info
Make it explicit that the `version` property is for human consumption.
Also add a new `git` property so that all info from `version` is also
included in separate fields for machine consumption.
2023-07-06 12:18:26 +02:00