20 Commits

Author SHA1 Message Date
Sybren A. Stüvel
a9bec98fcd Fix linter warnings
Fix most linter warnings reported by 'staticcheck'. This doesn't fix all
of them, some unused functions are still there, and some generated code
also still triggers some warnings. Most issues are fixed, though.

No functional changes, except for the captialisation of some error
messages.
2024-12-01 14:49:25 +01:00
Sybren A. Stüvel
b219f9b1c2 Manager tests: replace assert.NoError() with require.NoError()
Back in the days when I wrote the code, I didn't know about the
`require` package yet. Using `require.NoError()` makes the test code
more straight-forward.

No functional changes, except that when tests fail, they now fail
without panicking.
2024-03-16 12:14:39 +01:00
Sybren A. Stüvel
3f4a9025fe Manager tests: replace assert.NoError() with require.NoError()
Back in the days when I wrote the code, I didn't know about the
`require` package yet. Using `require.NoError()` makes the test code
more straight-forward.

No functional changes, except that when tests fail, they now fail
without panicking.
2024-03-16 11:09:18 +01:00
Sybren A. Stüvel
e7c4285ac6 Manager: Adjust code for renaming SocketIO... types to Event...
No functional changes, just adjusting to the OpenAPI renames.
2024-02-05 09:25:43 +01:00
Sybren A. Stüvel
76a24243f0 Manager: Introduce event bus system
Introduce an "event bus"-like system. It's more like a fan-out
broadcaster for certain events. Instead of directly sending events to
SocketIO, they are now sent to the broker, which in turn sends it to any
registered "forwarder". Currently there is ony one forwarder, for
SocketIO.

This opens the door for a proper MQTT client that sends the same events
to an MQTT server.
2024-02-03 22:55:23 +01:00
Sybren A. Stüvel
02fac6a4df Change Go package name from git.blender.org to projects.blender.org
Change the package base name of the Go code, from
`git.blender.org/flamenco` to `projects.blender.org/studio/flamenco`.

The old location, `git.blender.org`, has no longer been use since the
[migration to Gitea][1]. The new package names now reflect the actual
location where Flamenco is hosted.

[1]: https://code.blender.org/2023/02/new-blender-development-infrastructure/
2023-08-01 12:42:31 +02:00
Sybren A. Stüvel
726129446d T99730: Allow access to full task log
The web interface has a button that opens the task log in a new window.
This might need some restyling ;-)
2022-07-16 12:55:41 +02:00
Sybren A. Stüvel
686295090b Manager: implement endpoint for getting the full task log
Previously only the log tail was available, which is fine for many cases,
but for serious debugging the entire log is needed.

Manifest task: T99730
2022-07-16 11:13:31 +02:00
Sybren A. Stüvel
d4429d593c Unify task log storage & manager-local storage
The task logs storage system is refactored to use the `local_storage`
package. Configuration options have also changed:

- `task_logs_path` is renamed to `local_manager_storage_path`, to
  emphasise that only the Manager deals with those files, with default
  value `./flamenco-manager-storage`.
- `storage_path` is renamed to `shared_storage_path`, to emphasise this
  is the storage shared between Manager and Workers, with default value
  `./flamenco-shared-storage`.

Task logs are still stored in
`${local_manager_storage_path}/job-{jobUUID[0:4]}/{jobUUID}/task-{taskUUID}.txt`

Manifest task: T99409
2022-07-05 17:58:58 +02:00
Sybren A. Stüvel
fb89658530 Refactor: replace os.IsNotExist() with errors.Is(err, fs.ErrNotExist()
`os.IsNotExist()` is from before `errors.Is()` existed. The latter is the
recommended approach, as it also recognised wrapped errors.

No functional changes, except for recognising more cases of "does not
exist" errors as such.
2022-06-28 10:24:46 +02:00
Sybren A. Stüvel
27a6dde708 Manager: add local_storage package for managing storage locations
Add a `local_storage` package that finds a suitable place to put files.
Currently it just looks at the location of the currently running
executable; it can later do other things. It can be queried for directory
to put job-specific files.

It is intended to be used by the under-development "last rendered output"
processing system, to store an image file per job. Later we should also
refactor the task log handling system to use this.
2022-06-23 16:45:38 +02:00
Sybren A. Stüvel
24204084c1 Manager: move timestamping of log messages to task_logs package
In the future different services will write to the task log, and thus
it makes sense to move the responsibility of prepending the timestamps
to the log storage service.
2022-06-09 17:00:38 +02:00
Sybren A. Stüvel
819cad1d18 Manager: move broadcasting of task logs via SocketIO to task log service
To ensure all task logs also get broadcast via SocketIO, the responsibility
has moved from the `api_impl` to the `task_logs` package.
2022-06-09 16:49:48 +02:00
Sybren A. Stüvel
04dd479248 Manager: protect task log writing with mutex
A per-task mutex is used to protect the writing of task logs, so that
mutliple goroutines can safely write to the same task log.
2022-06-09 14:44:54 +02:00
Sybren A. Stüvel
ce07a46455 Fix error fetching non-existing log tail
A task can exist in the database but not have any log stored on disk yet.
This is now returned as `204 No Content` instead of an internal server
error.

The web interface is also adjusted to cope with this.
2022-05-30 19:23:10 +02:00
Sybren A. Stüvel
9bb4dd49dd Manager: add endpoint to fetch task log tail
It returns 2048 bytes at most. It'll likely be less than that, as it will
ignore the first bytes until the very first newline (to avoid returning
cut-off lines). If the log file itself is 2048 bytes or smaller, return the
entire file.
2022-05-20 16:34:13 +02:00
Sybren A. Stüvel
9f5e4cc0cc License: license all code under "GPL-3.0-or-later"
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
2022-03-07 15:26:46 +01:00
Sybren A. Stüvel
cd2fe8170e Errors: remove "error" prefix from message
Instead of returning an error "error doing X", just return "doing X". The
fact that it's returned as an error object says enough about that it's
an error.

This also makes it easier to chain error messages, without seeing the
word "error" in every part of the chain.
2022-03-04 11:30:31 +01:00
Sybren A. Stüvel
b1b73de4ee Manager: Actually load config, and use in a few places 2022-02-21 19:02:07 +01:00
Sybren A. Stüvel
bb53cc1e4a Task log storage service 2022-02-18 18:19:35 +01:00