1685 Commits

Author SHA1 Message Date
Sybren A. Stüvel
988cdf61ff Upgrade GORM & SQLite
Upgrade:
- `gorm.io/gorm` v1.23.8 → 1.25.2
- `github.com/glebarez/go-sqlite` v1.17.3 → v1.8.0
- `github.com/glebarez/sqlite` v1.4.6 → v1.8.0

and also some indirect dependencies.

This is in the hope that some weird cases at Blender Studio get resolved.
It appears that sometimes, for some unknown reason, when deleting a job,
its tasks get reassigned to another job (instead of also getting deleted).

Since there is no code in Flamenco itself to do this task deletion (it's
all depending on SQLite following the foreign keys and cascading to tasks),
I hope it was a bug in either GORM or SQLite that got fixed at some point.
2023-07-06 16:08:57 +02:00
Sybren A. Stüvel
22f4aa09f3 Manager: expand job deletion unit test
Add extra job to the database before deleting one, to ensure that job
deletion doesn't do anything with other jobs (and their tasks).

No functional changes to Flamenco itself.
2023-07-06 16:08:57 +02:00
Michael Cook
5657f5e1fe Manager: Fix logURLs possble typo
Change "possble URL" to "possible URLs".

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104231
2023-07-06 12:43:40 +02:00
Sybren A. Stüvel
6a30f844eb Manager: Better reporting of version via API call
Before: `3.3-alpha0-v3.2-76-gdd34d538-dirty`
After : `3.3-alpha0 (v3.2-76-gdd34d538-dirty)`

Also include the new `git` property to always have the Git hash (the part
between parentheses).
2023-07-06 12:21:47 +02:00
Sybren A. Stüvel
82456424c9 OAPI: regenerate code 2023-07-06 12:19:48 +02:00
Sybren A. Stüvel
c6dcf6d7bd OAPI: better reporting of version info
Make it explicit that the `version` property is for human consumption.
Also add a new `git` property so that all info from `version` is also
included in separate fields for machine consumption.
2023-07-06 12:18:26 +02:00
Michael Cook
dd34d538ec fix links in mgr configuration doc section (#104226)
Actually include the link URLs in the documentation.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104226
2023-07-03 11:11:44 +02:00
Sybren A. Stüvel
77db55bb14 Manager: when worker signs off, only remember specific statuses
Limit which worker statuses are remembered (when they go offline) to
those that we want to restore when they come back online. This is now
set to `awake` and `asleep`. This prevents workers from being told to go
to states that they cannot handle, such as `error` or `starting`.
2023-06-23 11:38:37 +02:00
Adi Sage
8a5c099d57 Website: Documenting descriptions for Job and Task Statuses in the documentation webpage
This is a comprehensive list of all possible job and task statuses,
along with a brief description of each status
and a list of possible next statuses.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104214
2023-06-22 19:41:51 +02:00
Bastien Montagne
71f2947c4b Add 'copy-file' command. (#104220)
Initial implementation with some basic tests.

The API command only accept one source and destination path, which must
be absolute. If the destination already exists, it will not be
ovewritten, and an error is generated.

JS job definition code using this new command can easily handle list of
paths and/or relative paths if needed.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104220
2023-06-08 16:20:43 +02:00
Sybren A. Stüvel
5b60a0f550 Update CHANGELOG.md 2023-06-02 22:58:54 +02:00
Sybren A. Stüvel
49aeccf37b Remove vscode settings
This prevents different user preferences fighting each other.

Other VSCode files are still there as I think they're generally useful:
- `extensions.json` for suggesting some VSCode extensions
- `launch.json` to launch Worker and Manager in the debugger
2023-06-02 22:56:20 +02:00
Eveline Anderson
4d2200bb0c Fix #99549: Remember Previous Status (#104217)
Fix #99549: When sending Workers offline, remember their previous status

When the status of a worker goes offline, the Manager will now make the status of the worker to be remembered once it goes back online. So when the Worker makes this status change (so for example `X → offline`), Manager should immediately set `StatusRequested = "X" ` once it goes online.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104217
2023-06-02 22:50:07 +02:00
Sybren A. Stüvel
2c163652c0 Makefile: when deploying the website, avoid setting timestamps & owners
The current setup on our webserver disallows setting timestamps or
permissions on the root (of the website) directory, so `rsync -a`
caused an error.

`-a` is a synonym of `-rlptgoD`; the new options are that minus `-t` for
setting times, `-o` and `-g` for setting file ownership (they should
be owned by the uploading user), and `-p` for setting permissions.
2023-06-02 10:51:49 +02:00
Sybren A. Stüvel
86514b9342 Remove the 'None' project finder, set 'blender project' as default
The 'None' project finder is the old behaviour of the add-on, and it is not
really necessary. If any of the other finders cannot find the directory
they're looking for, they'll return the current blend file's directory
anyway.

The new default is 'blender project'.
2023-06-01 16:34:13 +02:00
Sybren A. Stüvel
f4f61ea593 Add 'project finders' to the add-on
The Flamenco add-on can now find the top-level directory of your Blender
project. It can be configured to find any of these directories:

- `.blender_project`
- `.git`
- `.subversion`
- None, which is the default and the old behaviour of just using the
  directory of the current blend file as the 'project directory'.
2023-06-01 16:07:11 +02:00
Sybren A. Stüvel
fbc7c0cfd9 Fix mypy errors
Mypy doesn't understand Blender property annotations.
2023-06-01 15:51:44 +02:00
Sybren A. Stüvel
87ff1187ef Fix #104218: Shaman unit test TestGarbageCollect is unstable
Copy files instead of hard-linking. The hard-links seemed to have some
weird behaviour, at least on Windows 10 where I tested this.
2023-05-31 13:57:48 +02:00
Sybren A. Stüvel
c1fe3fb740 Shaman GC: better logging for file deletion errors
When the Shaman GC process cannot remove files, log a message that explains
that this happens (and why).
2023-05-31 13:55:04 +02:00
Sybren A. Stüvel
caa2f9ccf1 Shaman: improve logging & checking of timestamps
Add more logging of timestamps in the actual code, and a few sanity
checks in unit tests.

These were useful while trying to find the root cause of #104218 and might
be useful in the future too. The solution to that issue will be committed
later.
2023-05-31 13:54:28 +02:00
Sybren A. Stüvel
16da14479b OAPI: regenerate code 2023-05-26 11:25:51 +02:00
Adi Sage
b56a610f8d OAPI: Remove obsolete job statuses
Remove the following statuses from `flamenco-openapi.yaml`:

- 'construction-failed'
- 'archiving'
- 'archived'

These were a leftover from Flamenco v2 and have never been used in
Flamenco v3.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104215
2023-05-26 11:25:35 +02:00
Sybren A. Stüvel
0086abcc7a Makefile: be less verbose when publishing the website 2023-05-26 11:20:05 +02:00
Sybren A. Stüvel
afde952c10 Fix incompatibility with 32-bit platforms 2023-05-24 21:23:05 +02:00
Adi Sage
0a2d7d3361 Website: Fix a typo in the docs for Job Types
In the first line of description for Job Types the word 'or' is changed to 'for'.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104212
2023-05-23 11:44:38 +02:00
Sybren A. Stüvel
9cbdadcd4b Add 'Flamenco gets stuck after the first chunk' to the FAQ
So far it's always been an add-on that caused this.
2023-05-22 16:21:42 +02:00
Sybren A. Stüvel
77ff6151bd Fix check-environment Makefile rule
Bunch of improvements to the `check-environment` rule:

- it was listed as the first target in `Makefile`, causing it to be run
  when you just use `make`. It's now moved down to where it is first used.
- Call with `$(MAKE) -s` so that there is less logging noise.
- Use `@echo` so that there is less logging noise.
- Change the example to set the right environment variable (it had a
  trailing underscore).
2023-05-19 12:09:29 +02:00
Francesco Siddi
9915951007 Deploy: Update deployment instructions
Move sensitive information into a dedicated .env file, which is
required only for deployment purposes.
2023-05-15 20:09:03 +02:00
Sybren A. Stüvel
ebf4021da2 Fix #104191: Manager build error on ARM64
Reimplement the `touch()` function on Linux to avoid depending on the
`syscall` package, and use the `sys/unix` package instead. This is
slightly higher level, and seems to build on AMD64 and ARM64.
2023-05-15 10:51:01 +02:00
Sybren A. Stüvel
bbeefd4bfa Website: fix developer 'get started' links
Links of the `[text][linkname]` form should not have a space between
the two bracketed parts. And I removed the link to projects.blender.org
because the other two links also link there (indirectly resp. directly).
2023-05-09 15:36:23 +02:00
Francesco Siddi
08c67347db Add blender.org analytics 2023-05-08 17:14:33 +02:00
Francesco Siddi
6642ad38da Website: Column based layout for the Get Involved page 2023-05-08 14:23:08 +02:00
Francesco Siddi
79b563b55a Website: Introduce primary and secondary styling for buttons 2023-05-08 14:22:48 +02:00
Francesco Siddi
8da71a323f Website: Content refactor
- Hide sidebar in pages visible in the top nav, except for Docs
- Create an "About" page, featuring the design principles
- Update components illustration
- Move the "What's new in Flamenco 3" page in the FAQ
- Add TOD to the FAQ
- Tweak homepage layout to use two columns
- Styling: slightly reduce max page width (around 12 words per line)
- Styling: tweak padding for titles and paragraphs
2023-05-06 03:34:54 +02:00
Francesco Siddi
1f8301a575 Website: Enable doc editing links 2023-05-06 03:29:39 +02:00
Francesco Siddi
de9eb6ab08 Website: Fix indentation in config file 2023-05-06 03:29:39 +02:00
Francesco Siddi
a14c926284 Website: Style tweaks in dark mode
Increase contrast in dark mode, and increase line height for more
readability.
2023-05-06 03:29:39 +02:00
Francesco Siddi
5981e0d03a Website: Fix flamenco.io reference
The website is now hosted on flamenco.blender.org.
2023-05-06 03:29:39 +02:00
Sybren A. Stüvel
3c1871cf0b Web: some rewording & markup changes to 'mission' page
Some rewording & reformatting of the 'mission', and extending the
'development/getting started' page.
2023-05-01 10:56:52 +02:00
MichaelC
ded6f35347 Add mission statement to project website (#104207)
Initial draft of a mission statement for Flamenco project based on original blog post announcing Flamenco v3.
Based on contribution suggestion from https://blender.chat/channel/flamenco?msg=uQq4thF8uBu7X5FQq

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104207
2023-05-01 10:39:08 +02:00
Sybren A. Stüvel
692da51e55 Fix #104197: Document design principles
Document the design principles. This is basically
https://studio.blender.org/blog/flamenco-3-a-new-beginning/
plus some more info.
2023-04-24 15:49:23 +02:00
Sybren A. Stüvel
ad3cee6b7b Add task limitation fix to CHANGELOG.md 2023-04-24 15:12:57 +02:00
Anish Bharadwaj (he)
0502498dfa Fix #104201: Task Limit error in Flamenco Manager
Insert tasks in batches so that the required SQL query stays within the limits of SQLite.

No changes to the API, only to the persistence layer.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104205
2023-04-24 15:10:59 +02:00
Sybren A. Stüvel
ef53304b1b Fix #104206: Worker cluster assignment is lost when blender is restarted
The available worker clusters are stored on the addon preferences, to be
available across restarts of Blender. The prefs were not marked as 'dirty'
though, so Blender never actually saved them. That's now resolved.
2023-04-24 12:42:45 +02:00
Nitin-Rawat-1
752597b8e1 Check for number of workers before soft failing the task. (#104195)
Manager: fixed issue #104190 job getting stuck with less workers than soft-failed threshold,
before soft-failing check the number of workers to decide if job should be failed or not.

Reviewed-on: https://projects.blender.org/studio/flamenco/pulls/104195
2023-04-20 11:53:41 +02:00
Sybren A. Stüvel
42878e169f Web: document the need to use drive letters on Windows
Flamenco doesn't support UNC notation. Now that's actually documented.
2023-04-20 10:44:25 +02:00
Sybren A. Stüvel
472b73eb5c Cleanup: run go fmt ./...
No functional changes.
2023-04-17 16:36:52 +02:00
Sybren A. Stüvel
d03bbd49f4 Docs: some rewriting of the Compositor Nodes doc
A few minor adjustments:
- Added a note about the job type being third-party, and what that implies.
- Removed the need to use Git or the mention of symlinks, to simplify
  installation instructions.
- Fixed a few markup issues.
2023-04-14 11:55:34 +02:00
Sybren A. Stüvel
ec678e0e51 Docs: 3rd party job types, some small changes
Just a bit of rewording, mostly avoiding using the word 'job' for 'job
type'.
2023-04-14 11:54:21 +02:00
Sybren A. Stüvel
c119819c13 Docs: ask for bugreport when issue with BAT is found 2023-04-14 11:53:33 +02:00