20 Commits

Author SHA1 Message Date
Sybren A. Stüvel
816046663e Manager: remove GORM from database interface
Remove all calls to GORM from the database interface code.

Ref: #104305
2024-09-26 22:58:11 +02:00
Sybren A. Stüvel
476d4059bf Manager: convert pragma foreign key command to sqlc
No functional changes.

Ref: #104305
2024-09-18 10:17:27 +02:00
Sybren A. Stüvel
70faa4e225 Move URLs to the Flamenco website to constants in a dedicated package
Create a dedicated package `.../pkg/website` to contain constants for the
URLs of documentation, bug reporting, etc. That way it's easier to see
which parts of the website are being referred to from the Flamenco
binaries, and updates can happen in a central spot.

No functional changes.
2024-01-25 12:25:06 +01:00
Sybren A. Stüvel
a65f234bea Manager: replace GORM database migration with Goose
Replace GORM's auto-migration with Goose. The latter uses hand-written
SQL queries to apply database schema changes, which is safer and easier to
understand than what GORM is doing.
2023-12-14 10:13:40 +01:00
Sybren A. Stüvel
06738b8aa4 Manager: disable SQLite foreign key constraints when migrating the database
There is an issue with the GORM auto-migration, in that it doesn't
always disable foreign key constraints when it should. Due to
limitations of SQLite, not all 'alter table' commands you'd want to use
are available. As a workaround, these steps are performed:

1. create a new table with the desired schema,
2. copy the data over,
3. drop the old table,
4. rename the new table to the old name.

Step #3 will wreak havoc with the database when foreign key constraint
checks are active, so no we temporarily deactivate them while performing
database migration.
2023-07-10 14:06:21 +02:00
Eveline Anderson
830c3fe794 Rename worker 'clusters' to 'tags'
As it was decided that the name "tags" would be better for the clarity
of the feature, all files and code named "cluster" or "worker cluster"
have been removed and replaced with "tag" and "worker tag". This is only
a name change, no other features were touched.

This addresses part of #104204.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104223

As a note to anyone who already ran a pre-release version of Flamenco
and configured some worker clusters, with the help of an SQLite client
you can migrate the clusters to tags. First build Flamenco Manager and
start it, to create the new database schema. Then run these SQL queries
via an sqlite commandline client:

```sql
insert into worker_tags
    (id, created_at, updated_at, uuid, name, description)
  select id, created_at, updated_at, uuid, name, description
  from worker_clusters;

insert into worker_tag_membership (worker_tag_id, worker_id)
  select worker_cluster_id, worker_id from worker_cluster_membership;
```
2023-07-10 11:11:03 +02:00
Sybren A. Stüvel
8408d28a6b Manager: add support for worker clusters 2023-04-04 12:18:35 +02:00
Sybren A. Stüvel
f413a40f4e Store Shaman checkout ID when submitting a job
If Shaman is used to submit the job files, store the job's checkout ID
(i.e. the path relative to the checkout root) in the database. This will
make it possible in the future to remove the Shaman checkout along with
the job itself.
2023-01-04 01:18:21 +01:00
Sybren A. Stüvel
627996525e Manager: implement operations for getting & setting worker sleep schedule
This is just the API, no web interface yet.

Manifest Task: T99397
2022-07-16 16:00:25 +02:00
Sybren A. Stüvel
d25151184d Add a "Last Rendered" view
Add a "Last Rendered" view to the webapp.

The Manager now stores (in the database) which job was the last
recipient of a rendered image, and serves that to the appropriate
OpenAPI endpoint.

A new SocketIO subscription + accompanying room makes it possible for
the web interface to receive all rendered images (if they survive the
queue, which discards images when it gets too full).
2022-07-01 12:34:40 +02:00
Sybren A. Stüvel
fd31a85bcd Manager: add blocking of workers when they fail certain tasks too much
When a worker fails too many tasks, of the same task type, on the same job,
it'll get blocked from doing those.
2022-06-17 15:49:16 +02:00
Sybren A. Stüvel
c5debdeb70 Manager: add 'task failure list' to record workers failing tasks
The persistence layer can now store which worker failed which task, as
preparation for a blocklisting system. Such a system should be able to
determine whether there are still any workers left to do the work.
2022-06-13 18:41:30 +02:00
Sybren A. Stüvel
9f5e4cc0cc License: license all code under "GPL-3.0-or-later"
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
2022-03-07 15:26:46 +01:00
Sybren A. Stüvel
641ed7ace9 Manager: make Gorm use Zerolog for logging
A wrapper for Zerolog implements the Gorm logger interface. This gives
us coloured output on Windows, and uniform-looking logs in production.
2022-03-03 13:52:50 +01:00
Sybren A. Stüvel
4aafb782ac Scheduler: Assign task to worker 2022-02-14 17:47:26 +01:00
Sybren A. Stüvel
28a56f3d91 Store workers in database when registering 2022-01-28 15:31:39 +01:00
Sybren A. Stüvel
30518ca3af Store tasks & commands in the database 2022-01-25 18:25:26 +01:00
Sybren A. Stüvel
0f8bacd2e5 Store compiled jobs with GORM in PostgreSQL 2022-01-24 18:07:12 +01:00
Sybren A. Stüvel
f0ae7d33e9 Connect API, job compiler, and persistence layer 2022-01-11 18:07:34 +01:00
Sybren A. Stüvel
0c9c99806b Start of database interface
Contains a dummy migration because the migrate package will error out if
it cannot find any migration files.
2022-01-11 18:07:34 +01:00