11 Commits

Author SHA1 Message Date
Sybren A. Stüvel
423c5c29d7 Shaman: use relative paths for symlinks (#104332)
Use relative paths for symlinks, so that the Shaman storage can be mounted
on different locations on the Worker without the symlinks breaking.

Fixes studio/flamenco#104328

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104332
2024-09-05 12:12:44 +02:00
Sybren A. Stüvel
3f86364fa4 Shaman tests: colorise log output on some tests
Easier to read logging when tests fail. I've just done this where I need
it, as there is no central place where Flamenco logs are initialised for
unit tests.
2024-08-01 13:52:41 +02:00
Sybren A. Stüvel
5da5edd333 Manager: modernise Shaman unit test code
Replace calls to `assert.NoError()` with `require.NoError()`
2024-08-01 13:51:43 +02:00
Sybren A. Stüvel
d60af809be Shaman: skip certain tests if the platform cannot do symlinking reliably
Windows 10 Home ("Core") only allows symlinking when running as admin,
which is not recommended for Flamenco Manager. Instead of failing unit
tests on this system, simply skip them. This reduces noise when developing
on this platform (i.e. my personal laptop) a lot.
2023-10-15 14:26:32 +02:00
Sybren A. Stüvel
02fac6a4df Change Go package name from git.blender.org to projects.blender.org
Change the package base name of the Go code, from
`git.blender.org/flamenco` to `projects.blender.org/studio/flamenco`.

The old location, `git.blender.org`, has no longer been use since the
[migration to Gitea][1]. The new package names now reflect the actual
location where Flamenco is hosted.

[1]: https://code.blender.org/2023/02/new-blender-development-infrastructure/
2023-08-01 12:42:31 +02:00
Sybren A. Stüvel
791d877ff1 Manager: implement API endpoint for deleting jobs
Implement the `deleteJob` API endpoint. Calling this endpoint will mark
the job as "deletion requested", after which it's queued for actual
deletion. This makes the API response fast, even when there is a lot of
work to do in the background.

A new background service "job deleter" keeps track of the queue of such
jobs, and performs the actual deletion. It removes:

- Shaman checkout for the job (but see below)
- Manager-local files of the job (task logs, last-rendered images)
- The job itself

The removal is done in the above order, so the job is only removed from the
database if the rest of the removal was succesful.

Shaman checkouts are only removed if the job was submitted with Flamenco
version 3.2. Earlier versions did not record enough information to reliably
do this.
2023-01-04 01:18:21 +01:00
Sybren A. Stüvel
f413a40f4e Store Shaman checkout ID when submitting a job
If Shaman is used to submit the job files, store the job's checkout ID
(i.e. the path relative to the checkout root) in the database. This will
make it possible in the future to remove the Shaman checkout along with
the job itself.
2023-01-04 01:18:21 +01:00
Sybren A. Stüvel
44ccc6c3ca Shaman: handle duplicate symlinks gracefully
Instead of erroring out when a symlink already exists, investigate it. If
the linked file is the one that's intended, just use it.

For some reason, BAT and/or the Flamenco add-on include some files twice
in the checkout request to Shaman. This is now handled gracefully.
2022-08-04 16:30:31 +02:00
Sybren A. Stüvel
fe9849b134 Shaman: fix bunch of path issues on Windows
Shaman was made on Linux, using the `path` module, which only supports
forward slashes. This mostly replaces `path` with `path/filepath` to use
native paths and increase Windows compatibility.

The work isn't done yet, though.
2022-03-28 11:02:18 +02:00
Sybren A. Stüvel
3905135465 Tests: replace assert.Nil(t, err) with assert.NoError(t, err)
This is a better way to check errors, both semantically (no usually nil is
bad, but nil error is good, so this avoids a mental negation) and for
the reporting by the assert package.
2022-03-26 14:05:53 +01:00
Sybren A. Stüvel
4e8e71e4e2 Initial checkin of Shaman of Flamenco 2
This is not yet working, it's just a direct copy of the Manager of Flamenco
2, with Logrus replaced by Zerolog. The API has been documented in
flamenco-manager.yaml as a starting point for the integration.
2022-03-25 14:10:26 +01:00