2215 Commits

Author SHA1 Message Date
Sybren A. Stüvel
a9bec98fcd Fix linter warnings
Fix most linter warnings reported by 'staticcheck'. This doesn't fix all
of them, some unused functions are still there, and some generated code
also still triggers some warnings. Most issues are fixed, though.

No functional changes, except for the captialisation of some error
messages.
2024-12-01 14:49:25 +01:00
Sybren A. Stüvel
167dd027c1 Move the 'Single Image Render' job type to the website
Move the 'Single Image Render' job type to the 'third party job types'
section on the website. It needs more testing & finessing before it can be
bundled with Flamenco.
2024-12-01 14:08:18 +01:00
Sybren A. Stüvel
dfed899c08 Refactor: ensure test function params are in (expected, actual) order
No functional changes
2024-11-11 19:20:41 +01:00
Sybren A. Stüvel
def44abb64 Mage: forward LDFLAGS environment variable to Go build command
When building Flamenco Manager/Worker, forward the `LDFLAGS` environment
variable value to the build command. This makes it possible to override
certain variables, like the default configuration file location.
2024-11-11 11:55:46 +01:00
Sybren A. Stüvel
ed014ccc2a Worker: log which config paths are used at startup
To aid in debugging configuration loading issues, log the paths to config
files at startup.
2024-11-11 11:49:36 +01:00
Sybren A. Stüvel
b84523c0ae Fix build issue with vendored sources
Fix a build issue when `go mod vendor` was used to vendor all the sources.
This now includes Mage (`github.com/magefile/mage/mage`), so that the
build tool itself can be built too.
2024-11-11 11:00:59 +01:00
Sybren A. Stüvel
30997013b1 Fix: Blender add-on was accidentally built as flamenco3-addon.zip
The Mage-based build tools accidentally built the add-on as
`flamenco3-addon.zip` instead of `flamenco-addon.zip`.
2024-11-11 10:55:48 +01:00
Sybren A. Stüvel
7f37c16a8d Add 'index in job' number to tasks
Number the tasks in a job, indicating their creation order. This gives the
web interface something to sort on that doesn't change on task updates.
2024-11-09 23:07:23 +01:00
Sybren A. Stüvel
b3385394ae OAPI: regenerate code 2024-11-09 23:01:52 +01:00
Sybren A. Stüvel
6efd5144da OAPI: Add 'index in job' field to tasks
Give tasks a new field, explicitly storing the creation order of the tasks
in the job. This makes it possible for the web interface to show the tasks
in a stable order, without jumping around when they update.
2024-11-09 22:59:51 +01:00
Sybren A. Stüvel
e08bdbdf16 Manager: increase database busy timeout from 5 to 20 seconds
When under heavy load, queries can take longer.
2024-10-10 15:08:48 +02:00
Sybren A. Stüvel
479f35b658 Mage: avoid creating web/static/emptyfile with wrong permissions
The empty file is created so that Go has something to embed when building
Flamenco  Manager, even when the static files aren't built yet. On macOS
this was done with the wrong permissions, though.
2024-10-05 10:25:02 +02:00
Sybren A. Stüvel
7d8262e630 Website: document the use of build tool mage 2024-10-04 22:47:29 +02:00
Sybren A. Stüvel
4ae4b892fd Mage: remove duplicated targets
The same functionality was available as two Mage targets.

WebappInstallDeps has been removed, in favour of InstallDepsWebapp.
2024-10-04 22:46:46 +02:00
Sybren A. Stüvel
5f37bcb629 Build with Magefile (#104341)
Convert most of the code in `Makefile` to [Magefile](https://magefile.org/):

This makes it possible to build Flamenco without `make` (and the POSIX environment/commands it expect) by running:

```bash
$ go run mage.go webappInstallDeps  # Only on the first build.
$ go run mage.go build
```

More efficient builds are possible with other commands, and some release-related commands still require `make`. At least the barrier to entry should be considerably lower (compared to having to install Make + Cygwin/MSYS2 on Windows).

Fix: #102633

This does not port the building of release packages, so it doesn't address #102671.

### Main Targets

| Target   | Description                                                                     |
|----------|---------------------------------------------------------------------------------|
| build    | Build Flamenco Manager and Flamenco Worker, including the webapp and the add-on |
| check    | Run unit tests, check for vulnerabilities, and run the linter                   |
| clean    | Remove executables and other build output                                       |
| generate | Generate code (OpenAPI and test mocks)                                          |

### All Targets

Get these via `go run mage.go -l`:

```
Targets:
  build                           Flamenco Manager and Flamenco Worker, including the webapp and the add-on
  check                           Run unit tests, check for vulnerabilities, and run the linter
  clean                           Remove executables and other build output
  flamencoManager                 Build Flamenco Manager with the webapp and add-on ZIP embedded
  flamencoManagerWithoutWebapp    Only build the Flamenco Manager executable, do not rebuild the webapp
  flamencoWorker                  Build the Flamenco Worker executable
  generate                        code (OpenAPI and test mocks)
  generateGo                      Generate Go code for Flamenco Manager and Worker
  generateJS                      Generate JavaScript code for the webapp
  generatePy                      Generate Python code for the add-on
  govulncheck                     Check for known vulnerabilities.
  staticcheck                     Analyse the source code.
  test                            Run unit tests
  version                         Show which version information would be embedded in executables
  vet                             Run `go vet`
  webappInstallDeps               Use Yarn to install the webapp's NodeJS dependencies
  webappStatic                    Build the webapp as static files that can be served
```

Co-authored-by: Mateus Abelli <mateusabelli@gmail.com>
Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104341
2024-10-04 21:59:44 +02:00
Sybren A. Stüvel
d61f6c9e14 Manager: in setup assistant, find Blender in macOS default install dir
On macOS, automatically find Blender when it is installed in the default
installation directory (`/Applications/Blender.app`)
2024-10-03 21:15:56 +02:00
Sybren A. Stüvel
cbf82e4a55 OAPI: regenerate code 2024-10-03 21:15:56 +02:00
Sybren A. Stüvel
8c4ac87ad9 OAPI: add 'system_location' as possible source of Blender locations
When finding Blender, Flamenco should be able to find it in platform-
specific default locations. This is mostly used for macOS, where Blender
is typically installed in `/Applications/Blender.app`.
2024-10-03 21:15:56 +02:00
Sybren A. Stüvel
0031ac31af Website: document "brew link" step in macOS dev docs 2024-10-03 19:12:27 +02:00
Sybren A. Stüvel
453c572916 make update-version: always update all relevant files
Fix an issue where `make update-version` would only update the first set
of files it could, and skip the remaining ones.

Only relevant for developers.
2024-10-01 23:12:52 +02:00
Sybren A. Stüvel
6f38ac75ea 'make update-version': don't update website
Do not update the website via `make update-website`. Which version of
Flamenco is published on the website is managed manually, independently of
the version in the Git repo.
2024-10-01 23:11:21 +02:00
Sybren A. Stüvel
f1afae47d4 Website: document how to run the Flamenco Manager profiler
It's basic, but at least it saves me from finding the info again.
2024-09-30 18:07:17 +02:00
Sybren A. Stüvel
29a0c596cc Bumped version to 3.6-alpha5 2024-09-30 11:46:54 +02:00
Sybren A. Stüvel
5e52e1efa4 Bump experimental version to 3.6-alpha4 2024-09-30 11:31:08 +02:00
Sybren A. Stüvel
2e0e211b26 Fix #104338: Error performing BAT pack
Use RFC 2047 (aka MIME encoding) to send the original filename when
uploading a file to the Shaman server.

HTTP headers should be ASCII-only, and some systems use Latin-1 as
fallback. That's not suitable in general, though, because almost all
characters fall outside the Latin-1 range.
2024-09-30 11:25:49 +02:00
Sybren A. Stüvel
1f562b3cbc OAPI: Regenerate code 2024-09-30 11:24:39 +02:00
Sybren A. Stüvel
8d339b1528 OAPI: document encoding of the X-Shaman-Original-Filename header
Document that the `X-Shaman-Original-Filename` HTTP header (used when
submitting files to the Shaman server) should either be ASCII or encoded
using RFC 2047.

No functional changes.
2024-09-30 11:24:21 +02:00
Sybren A. Stüvel
131b7c719c Update CHANGELOG 2024-09-26 23:54:46 +02:00
Sybren A. Stüvel
fc35745f05 Remove GORM from dependencies
Run `go mod tidy` to remove GORM from the `go.mod` and `go.sum` files.

Fixes: #104305
2024-09-26 23:54:46 +02:00
Sybren A. Stüvel
71ae57977d Worker: add unit test for upstream buffer
No functional changes, just a test.
2024-09-26 23:54:46 +02:00
Sybren A. Stüvel
aa964ac205 Worker: replace GORM with sqlc
Ref: #104305
2024-09-26 23:54:46 +02:00
Sybren A. Stüvel
21cf3c47f9 Manager: remove GORM annotations and last dependencies
Remove GORM struct annotations/tags and references to GORM types.

Ref: #104305
2024-09-26 23:05:58 +02:00
Sybren A. Stüvel
816046663e Manager: remove GORM from database interface
Remove all calls to GORM from the database interface code.

Ref: #104305
2024-09-26 22:58:11 +02:00
Sybren A. Stüvel
f1a72903a0 Manager: rename sql/integrity.go to sql/pragma.go
The file contains pragma queries only, and I'm about to add more pragma
queries that aren't related to database integrity.
2024-09-26 22:43:30 +02:00
Sybren A. Stüvel
8513e2fdc8 Manager: replace GORM 'now' function with our own implementation
GORM implicitly sets 'created at', 'updated at' and 'deleted at' timestamps
to 'now' by calling a 'now function'. This is now implemented by Flamenco
directly, instead of relying on GORM.

Ref: #104305
2024-09-26 22:38:17 +02:00
Sybren A. Stüvel
29419cb30e Manager: convert final sleep schedule queries to sqlc
Ref: #104305
2024-09-26 22:26:30 +02:00
Sybren A. Stüvel
906880ae2c Manager: replace queries in task scheduler test
Ref: #104305
2024-09-26 21:36:00 +02:00
Sybren A. Stüvel
2cc049c52c Manager: convert final 'last rendered' queries to sqlc
Ref #104305
2024-09-26 21:26:38 +02:00
Sybren A. Stüvel
d43947898d Manager: replace final job-related queries with sqlc
Ref: #104305
2024-09-26 21:20:01 +02:00
Sybren A. Stüvel
ede5eb1d46 Manager: bump Go version to 1.23.1 2024-09-26 21:17:40 +02:00
Sybren A. Stüvel
71bbaaeae0 Manager; convert fetching of sleep schedules to sqlc
This also corrects the sleep schedule schema to actually store the
`is_active` field as `boolean` (it was `numeric`, which is the same
underlying field type in SQLite, but produces a different struct field
in the sqlc-generated Go code).

Ref: #104305
2024-09-18 21:11:54 +02:00
Sybren A. Stüvel
777a417cc0 Manager: convert timeout checks to sqlc
Ref: #104305
2024-09-18 20:39:03 +02:00
Sybren A. Stüvel
ebf1693a7c Manager: convert busy-timeout query to sqlc
Ref: #104305
2024-09-18 16:49:01 +02:00
Sybren A. Stüvel
40bfa91018 Manager: improve test stability
In a time-dependent test, wait a little longer to make it more stabler.
2024-09-18 16:40:35 +02:00
Sybren A. Stüvel
4bd6dc64b0 Manager: convert worker tag queries to sqlc
Ref: #104305
2024-09-18 16:07:43 +02:00
Sybren A. Stüvel
35313477a0 Manager; convert QueryJobTaskSummaries and SummarizeJobStatuses to sqlc
Ref: #104305
2024-09-18 14:54:49 +02:00
Sybren A. Stüvel
cda0b916fb Manager: replace queryJobs with fetchJobs operation
See the previous two commits for the motivation.
2024-09-18 14:29:15 +02:00
Sybren A. Stüvel
aebfaf4631 OAPI: regenerate code 2024-09-18 14:28:27 +02:00
Sybren A. Stüvel
c71ae574f1 OAPI: replace queryJobs operation with fetchJobs
Replace the `queryJobs` operation (which takes a query object to do
filtering & pagination) with the new `fetchJobs` operation (which just
returns all jobs without filtering).

The reason for this is that the querying that this operation supported
was never used by the front-end. And now it's getting in the way of other
development work, so it has to go.
2024-09-18 14:27:47 +02:00
Sybren A. Stüvel
3a872370df Manager: convert job blocklist queries from GORM to sqlc
Ref: #104305
2024-09-18 10:40:29 +02:00