2111 Commits

Author SHA1 Message Date
William Gardner
ea68faa577 Web: improve URL handling to allow for TLS/SSL
Explicitly use the `--mode` flag for the webapp development server
(`vite`) to make the web frontend choose the appropriate HTTP and
WebSocket port to communicate with the backend. This also makes sure
that when accessing the frontend via `https://`, the websocket
connection uses `wss://`.

As a side-effect, this also makes port `:8081` usable in production
environments; it would assume it was the development server and try to
access the backend on port `:8080`.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104296
Reviewed-by: Sybren A. Stüvel <sybren@blender.org>
2024-04-11 15:00:48 +02:00
Sybren A. Stüvel
9ee9c07e76 Website: link to 3rd party bug tracker for 3rd party scripts
3rd part job compiler scripts should have their own tracker and handle
their own bug reports.
2024-04-11 10:55:00 +02:00
Sybren A. Stüvel
e83db45192 Website: sillicon builds are included, not "will be" included 2024-04-06 10:19:38 +02:00
Sybren A. Stüvel
7b139be605 Website: remove link to old Flamenco v2 documentation
The documentation itself has disappeared from the website, and it already
was obsolete for a long time anyway.
2024-04-04 20:37:48 +02:00
Sybren A. Stüvel
d799372639 Website: bump experimental version to 3.5-beta1 2024-04-04 11:18:03 +02:00
Taylor Wiebe
a0cb8735c9 Manager: add optional description to job types
This description will be shown as a tooltip in the job submission UI.
2024-04-04 11:12:42 +02:00
Taylor Wiebe
2bbb7b48d2 OAPI: regenerate code 2024-04-04 11:12:39 +02:00
Taylor Wiebe
03a889345d OAPI: add optional description to job types
This description will be shown as a tooltip in the job submission UI.
2024-04-04 11:08:55 +02:00
Sybren A. Stüvel
cfad4e73f9 Bumped version to 3.5-beta1 2024-04-04 10:54:53 +02:00
Sybren A. Stüvel
3b21437776 Website: bump experimental version to 3.5-beta0 2024-04-04 10:47:23 +02:00
Sybren A. Stüvel
f757deee6a Security: Bump Go version 1.22.2 and golang.org/x/net package
Fix a bunch of security issues by upgrading to Go 1.22.2 and bumping
a few packages to their secure versions.

- [Incorrect forwarding of sensitive headers and cookies on HTTP redirect in net/http](https://pkg.go.dev/vuln/GO-2024-2600)
- [Memory exhaustion in multipart form parsing in net/textproto and net/http](https://pkg.go.dev/vuln/GO-2024-2599)
- [Verify panics on certificates with an unknown public key algorithm in crypto/x509](https://pkg.go.dev/vuln/GO-2024-2600)
- [HTTP/2 CONTINUATION flood in net/http](https://pkg.go.dev/vuln/GO-2024-2687)
2024-04-04 10:44:57 +02:00
Sybren A. Stüvel
bce84bf175 Bumped version to 3.5-beta0 2024-04-04 10:38:42 +02:00
Mateus Abelli
43a452e940 Website: Update the required Go version to Latest 2024-03-25 15:10:36 +01:00
Sybren A. Stüvel
c3b8707390 Worker: reduce log level of 'Blender could not be found' to info level
There's still some confusion that this is a thing to solve, whereas it can
usually safely be ignored. Reduced the log level from Warn to Info to make
the message look more innocent.
2024-03-18 22:39:55 +01:00
Sybren A. Stüvel
00dfbc10b6 Worker: link to FAQ entry when the worker cannot be found 2024-03-18 22:38:20 +01:00
Sybren A. Stüvel
63a578688e Make make test fail fast
Pass `-failfast` to the `go test` command, so that it immediately stops
on test failure. This prevents the need to scroll back to see the actual
error, at the expense of only seeing one failure at a time.
2024-03-16 12:14:41 +01:00
Sybren A. Stüvel
b219f9b1c2 Manager tests: replace assert.NoError() with require.NoError()
Back in the days when I wrote the code, I didn't know about the
`require` package yet. Using `require.NoError()` makes the test code
more straight-forward.

No functional changes, except that when tests fail, they now fail
without panicking.
2024-03-16 12:14:39 +01:00
Sybren A. Stüvel
3f4a9025fe Manager tests: replace assert.NoError() with require.NoError()
Back in the days when I wrote the code, I didn't know about the
`require` package yet. Using `require.NoError()` makes the test code
more straight-forward.

No functional changes, except that when tests fail, they now fail
without panicking.
2024-03-16 11:09:18 +01:00
Sybren A. Stüvel
1fee086cef Website: add MQTT prefix to table of MQTT topics 2024-03-13 19:16:40 +01:00
Sybren A. Stüvel
bd3dd90303 Update CHANGELOG.md
Add recent add-on improvements.
2024-03-13 19:13:37 +01:00
Sybren A. Stüvel
041f792594 Website: document Flamenco's MQTT support 2024-03-13 19:10:51 +01:00
Sybren A. Stüvel
5c60561ddb Website: allow separate page title for TOC
You can now set a page title and a separate title for the table of
contents with:

```
---
title: "Manager Configuration: MQTT"
titleTOC: MQTT
---
```
2024-03-13 18:42:52 +01:00
Sybren A. Stüvel
a23c95a738 Update CHANGELOG.md 2024-03-13 18:29:35 +01:00
Sybren A. Stüvel
91890541a2 Webapp: change Tabulator layout mode to fitDataFill
Change the Tabulator layout mode from `fitData` to `fitDataFill`. The new
value adjusts the layout when the data has changed.
2024-03-13 18:23:41 +01:00
Sybren A. Stüvel
3aec8b15ef Add-on: avoid console spam about the scene's job type property
Avoid these warnings on the console:

```
WARN (bpy.rna): source/blender/python/intern/bpy_rna.cc:1339
  pyrna_enum_to_py: current value '0' matches no enum in 'Scene', 'Scene',
  'flamenco_job_type'
```

The solution was two-fold:
- Use a non-empty string as the identifier for the 'Select a Job Type'
  choice.
- Give the property a default value.
2024-03-11 14:32:29 +01:00
Sybren A. Stüvel
d1fbe8b9f9 Manager: set default MQTT topic prefix to 'flamenco'
Set the default MQTT topic prefix to 'flamenco'. It can still be overridden
by the config in the YAML file, but it's nice to have a sensible default
when people don't configure this.
2024-03-08 16:44:39 +01:00
Sybren A. Stüvel
b476e31c0c Manager: remove unused configuration defaults
Remove commented-out sections in the configuration defaults. They're a
leftover from Flamenco v2.
2024-03-08 16:41:40 +01:00
Sybren A. Stüvel
cbafacdff6 Manager: don't forward task log updates to MQTT
Task log updates are big and frequent, and should not be sent via MQTT.
At least not until we have a practical reason to do so.
2024-03-07 15:22:44 +01:00
Sybren A. Stüvel
358efe7ae0 Manager: perform a database vacuum after migrations
Just to make sure the DB is properly cleaned up after a big migration
happened.
2024-03-06 11:59:17 +01:00
Sybren A. Stüvel
7277286391 Manager: increase 'database open' timeout to 1 minute
Increase the 'database open' timeout from 5 seconds to 1 minute. This
timeout also covers database migrations, and the recently added one that
adds a bunch of `NOT NULL` clauses could time out with the old 5 sec
limit.

The reason this takes long, is that SQLite doesn't directly support
adding `NOT NULL` clauses to columns. The only way to do this is to
create a new table with the desired schema, copy all data over, then
drop the old table. And with a big enough database, this takes time.
2024-03-06 11:50:14 +01:00
Sybren A. Stüvel
16114ee529 Worker: fix Go scheduling issue in sleep command test
Add a 1ms delay in the test loop, so that other goroutines can be scheduled
as well. This should fix #104288.
2024-03-04 14:18:08 +01:00
Sybren A. Stüvel
27cbb2ed0f Manager: increase timeout for database integrity check
With a fuller database, 2 seconds is apparently not always long enough,
so increase the timeout to 10 seconds.
2024-03-04 14:04:59 +01:00
Sybren A. Stüvel
63a2bce600 run go mod tidy
No functional changes.
2024-03-04 13:53:24 +01:00
Sybren A. Stüvel
3b4da656c9 Add-on: cache manager info in a JSON file
Instead of storing the cached manager info in the Blender preferences,
store the info in a JSON file. The file is located in the user prefs
folder (`~/.config/blender/{version}/config` on Linux).

This also reduces the number of 'refresh' operators to a single one, which
then fetches all necessary info from the Manager.

This fixes an issue (reported via chat) where worker tags were sometimes
not retained across file saves.
2024-03-04 13:08:53 +01:00
Sybren A. Stüvel
a4e5eef83e Manager: fix database migration 0004
Fix the database migration that adds `NOT NULL` clauses. It used
`INSERT INTO temp_x SELECT * from x;`, and the `*` returns the fields in
the order they are defined on the table. Since this might be different from
the order that the `INSERT INTO temp_x` expects, strange problems can
happen where columns get swapped (or constraints can fail on columns that
they should not fail for, because they got fed data from a different
column).
2024-03-04 13:06:09 +01:00
Sybren A. Stüvel
f9bb7afcc6 Add-on: fix mypy warnings
No functional changes.
2024-03-04 12:46:38 +01:00
Sybren A. Stüvel
7b72d0ca43 Refactor: move jobs-related queries to queries_jobs.sql
This makes it easier to later also create `query_workesr.sql`,
`query_meta.sql` etc. so that the sqlc-generated code can follow the
same subdivision as the persistence service code itself.

No functional changes.
2024-03-03 23:27:55 +01:00
Sybren A. Stüvel
b102b73a1f Refactor: convert more job functions to sqlc
No functional changes.
2024-03-03 23:23:51 +01:00
Sybren A. Stüvel
1ac796d0d8 Refactor: Manager: remove unused query from queries.sql
No functional changes.
2024-03-03 22:42:37 +01:00
Sybren A. Stüvel
3fbb3cde34 Manager: SQLC rename Uuid to UUID
No functional changes.
2024-03-03 20:54:43 +01:00
Sybren A. Stüvel
c046094880 Manager: start replacing GORM with SQLC
GORM has certain downsides:

- Code-first approach, where queries have to be translated to the Go code
  required to execute them.
- GORM comes with its own SQLite implementation, which doesn't provide an
  on-connect callback. This means that new connections cannot correctly
  enable foreign key constraints, causing database consistency issues.

[SQLC](https://sqlc.dev/) solves these issues for us.

This commit doesn't fully replace GORM with SQLC, but introduces it for
a few queries. Once all queries have been converted, GORM can be removed
completely.
2024-03-03 20:15:39 +01:00
Sybren A. Stüvel
7b31eba8d7 Webapp: improve positioning of the farm status indicator
Split the header into two or three parts, depending on the number of
columns shown. The farm status indicator will be above the middle column
(in 3 col mode) or at the right edge of the left column (in 2 col mode).

Also I reverted the hiding of the farm status when SocketIO has
disconnected, as that disconnect happens when navigation between tabs.
That created a too 'blinky' interface, so now it just shows the last-known
farm status.
2024-03-02 23:11:33 +01:00
Sybren A. Stüvel
7bf121e93e Webapp: show explanation in farm status tooltip 2024-03-02 22:32:02 +01:00
Sybren A. Stüvel
63e3c8de37 Webapp: make SocketIO reconnect faster
The exponential backoff was getting a bit too long, making the webapp
sometimes very slow to reconnect. This is now limited to max 3 seconds.
2024-03-02 22:20:42 +01:00
Sybren A. Stüvel
c0c70758c6 Webapp: hide farm status when SOcketIO is disconnected
Better to not show the farm status if the connection is lost.
2024-03-02 22:20:01 +01:00
Sybren A. Stüvel
1e7c059d12 Manager: check the farm status quickly after startup
The database is polled every 30 seconds to determine the farm status; at
startup the first poll is done after 1 second to get a faster status.

Note that when jobs and workers change their status, the farm status is
always updated.
2024-03-02 22:09:53 +01:00
Sybren A. Stüvel
7eb5eb68a3 Manager: ensure foreign keys are enabled in periodic integrity check
There are still issues with foreign keys getting disabled, so enable them
in the periodic database consistency check.

A more permanent solution is likely to drop GORM and switch to something
else that gives us an on-connect-callback, which can then be used to
turn on foreign key constraints for every connection made.
2024-03-01 23:42:04 +01:00
Sybren A. Stüvel
f763d12cd8 Update CHANGELOG 2024-03-01 22:40:21 +01:00
Sybren A. Stüvel
59bf389018 Webapp: show farm status in header bar
Show the farm status in the webapp header bar, and respond to farm status
events to update it when necessary.
2024-03-01 22:36:38 +01:00
Sybren A. Stüvel
c1a9b1e877 Manager: force a poll of the farm status when a job/worker changes state
This introduces the concept of 'event listener', which is now used by
the farm status service to respond to events on the event bus.

This makes it possible to reduce the regular poll period from 5 to 30
seconds. That's now only necessary as backup, just in case events are
missed or otherwise things change without the event bus logic noticing.
2024-03-01 22:36:38 +01:00