More work on the new flamenco.io site

This is basically the contents of the `README.md` file, spread over
multiple documentation pages.

web/flamenco-io-site/content/_index.md

@@ -1,15 +1,20 @@
 ---
-title: "Flamenco"
+title: "Flamenco 3"
 geekdocNav: false
 geekdocAlign: center
 geekdocAnchor: false
+geekdocBreadcrumb: false
 ---
 
-Take control of your computing infrastructure and get things done.
+Take control of your computing infrastructure and get things done.<br>
 Flamenco is used in production at Blender Institute.
 
+**Disclaimer:** this site describes Flamenco 3, which is still under heavy
+development. For information on its predecessor, see [Flamenco 2][F2].
+
 {{< button size="large" relref="usage/getting-started/" >}}Getting Started{{< /button >}}
 
+
 ## Feature overview
 
 {{< columns >}}
@@ -43,4 +48,6 @@ software. We use Go and SQLite, and host the sources on
 
 -------------------
 
-Looking for the old [Flamenco v2 documentation](/v2/)?
+Looking for the old [Flamenco v2 documentation][F2]?
+
+[F2]: /v2/

web/flamenco-io-site/content/development/_index.md

@@ -0,0 +1,4 @@
+---
+title: Development
+weight: 100
+---

web/flamenco-io-site/content/development/building/_index.md

@@ -0,0 +1,31 @@
+---
+title: Building Flamenco
+weight: 10
+---
+
+For the steps towards your first build, see [Getting Started][start].
+
+[start]: {{< relref "../getting-started ">}}
+
+These `make` targets are available:
+
+| Target | Description |
+| -- | -- |
+| `application` | Builds Flamenco Manager, Worker, and the development version of the webapp. |
+| `flamenco-manager` | Builds just Flamenco Manager. This includes packing the webapp and the Blender add-on into the executable. |
+| `flamenco-worker` | Builds just Flamenco Worker. |
+| `flamenco-manager_race` | Builds Flamenco Manager with the [data race detector][race] enabled. As this is for development only, this does not include packing the webapp and the Blender add-on into the executable. |
+| `flamenco-worker_race` | Builds Flamenco Worker with the [data race detector][race] enabled. |
+| `webapp` | Installs the webapp dependencies, so that the development server can be run with `yarn --cwd web/app run dev --host` |
+| `webapp-static` | Builds the webapp so that it can be served as static files by Flamenco Manager. |
+| `addon-packer` | Builds the addon packer. This is a little Go tool that creates the Blender add-on ZIP file. Typically this target isn't used directly; the other Makefile targets depend on it. |
+| `generate` | Generate the Go, Python, and JavaScript code. |
+| `generate-go` | Generate the Go code, which includes OpenAPI code, as well as mocks for the unit tests. |
+| `generate-py` | Generate the Python code, containing the OpenAPI client code for the Blender add-on. |
+| `generate-js` | Generate the JavaScript code, containing the OpenAPI client code for the web interface. |
+| `test` | Run the unit tests. |
+| `clean` | Remove build-time files. |
+| `version` | Print some version numbers, mostly for debugging the Makefile itself. |
+| `list-embedded` | List the files embedded into the `flamenco-manager` executable. |
+
+[race]: https://go.dev/doc/articles/race_detector

web/flamenco-io-site/content/development/database/_index.md

@@ -0,0 +1,12 @@
+---
+title: Database
+weight: 50
+---
+
+Flamenco Manager and Worker use SQLite as database, and Gorm as
+object-relational mapper.
+
+Since SQLite has limited support for altering table schemas, migration requires
+copying old data to a temporary table with the new schema, then swap out the
+tables. Because of this, avoid `NOT NULL` columns, as they will be problematic
+in this process.

web/flamenco-io-site/content/development/generating-code/_index.md

@@ -0,0 +1,23 @@
+---
+title: Generating Code
+weight: 20
+---
+
+Some code (Go, Python, JavaScript) is generated from the OpenAPI specs in
+`pkg/api/flamenco-openapi.yaml`. There are also Go files generated to create
+mock implementations of interfaces for unit testing purposes.
+
+**Generated code is committed to Git**, so that after a checkout you shouldn't
+need to re-run the generator to build Flamenco.
+
+The JavaScript & Python generator is made in Java, so it requires a JRE/JDK to
+be installed. On Ubuntu Linux, `sudo apt install default-jre-headless` should be
+enough.
+
+The following files & directories are generated. Generated directories are
+completely erased before regeneration, so do not add any files there manually.
+
+- `addon/flamenco/manager/`: Python API for the Blender add-on.
+- `pkg/api/*.gen.go`: Go API shared by Manager and Worker.
+- `internal/**/mocks/*.gen.go`: Generated mocks for Go unit tests.
+- `web/app/src/manager-api/`: JavaScript API for the web front-end.

web/flamenco-io-site/content/development/getting-started/_index.md

@@ -0,0 +1,68 @@
+---
+title: Getting Started
+weight: 1
+---
+
+To start, get a **Git checkout** with either of these commands. The 1st one is for
+public, read-only access. The 2nd one can be used if you have commit rights to
+the project.
+
+```
+git clone git://git.blender.org/flamenco.git
+git clone git@git.blender.org:flamenco.git
+```
+
+Then follow the steps below to get everything up & running.
+
+## 1. Installing Go
+
+Most of Flamenco is made in Go.
+
+1. Install [Go 1.18 or newer](https://go.dev/).
+2. Optional: set the environment variable `GOPATH` to where you want Go to put its packages. Go will use `$HOME/go` by default.
+3. Ensure `$GOPATH/bin` is included in your `$PATH` environment variable. Run `go env GOPATH` if you're not sure what path to use.
+
+## 2. Installing NodeJS
+
+The web UI is built with Vue, Bootstrap, and Socket.IO for communication with
+the backend. **NodeJS+Yarn** is used to collect all of those and build the
+frontend files.
+
+{{< tabs "installing-nodejs" >}}
+{{< tab "Linux" >}}
+It's recommended to install Node via Snap:
+
+```
+sudo snap install node --classic --channel=16
+```
+
+If you install NodeJS in a different way, it may not be bundled with Yarn. In that case, run:
+
+```
+sudo npm install --global yarn
+```
+{{< /tab >}}
+{{< tab "Windows" >}}
+Install [Node v16 LTS](https://nodejs.org/en/download/) and then install Yarn via:
+
+```
+npm install --global yarn
+```
+{{< /tab >}}
+{{< tab "macOS" >}}
+This still needs documenting.
+{{< /tab >}}
+{{< /tabs >}}
+
+## 3. Your First Build
+
+Run `make with-deps` to install build-time dependencies and build the application.
+Subsequent builds can just run `make` without arguments.
+
+You should now have two executables: `flamenco-manager` and `flamenco-worker`.
+Both can be run with the `-help` CLI argument to see the available options.
+
+See [building][building] for more `make` targets, for example to run unit tests,
+enable the race condition checker, and all other kinds of useful things.
+
+[building]: {{< relref "../building/" >}}

web/flamenco-io-site/content/development/openapi-commit-guidelines/_index.md

@@ -0,0 +1,25 @@
+---
+title: OpenAPI Commit Guidelines
+weight: 30
+---
+
+Typically a change to the OpenAPI definition consists of three steps, namely
+making the change to the OpenAPI file, regenerating code, and then alter
+whatever manually-written code needs altering.
+
+Each of these steps should be **committed independently**, by following these
+steps:
+
+1. Commit the changes to `pkg/api/flamenco-openapi.yaml`, prefixing the commit
+   message with `OAPI:`.
+2. Regenerate code with `make generate`, then commit with message
+   `OAPI: Regenerate code`.
+3. Commit any other code changes to complete the change.
+
+The downside to this approach is that the second commit will likely break the
+project, which is unfortunate. However, this approach does have some advantages:
+
+- The regenerated code has the commit hash of the actual change to the OpenAPI
+  definition.
+- Changes to manually-written and generated code are tracked in separate
+  commits. This makes them easier to comprehend by humans.

web/flamenco-io-site/content/development/socketio/_index.md

@@ -0,0 +1,22 @@
+---
+title: SocketIO
+weight: 50
+---
+
+[SocketIO v2](https://socket.io/docs/v2/) is used for sending updates from
+Flamenco Manager to the web frontend. Version 2 of the protocol was chosen,
+because that has a mature Go server implementation readily available.
+
+SocketIO messages have an *event name* and *room name*.
+
+- **Web interface clients** send messages to the server with just an *event
+  name*. These are received in handlers set up by
+  `internal/manager/webupdates/webupdates.go`, function
+  `registerSIOEventHandlers()`.
+- **Manager** typically sends to all clients in a specific *room*. Which client
+  has joined which room is determined by the Manager as well. By default every
+  client joins the "job updates" and "chat" rooms. This is done in the
+  `OnConnection` handler defined in `registerSIOEventHandlers()`.
+- Received messages (regardless of by whom) are handled based only on their
+  *event name*. The *room name* only determines *which* client receives those
+  messages.

web/flamenco-io-site/content/usage/_index.md

@@ -0,0 +1,4 @@
+---
+title: Usage
+weight: 10
+---

web/flamenco-io-site/content/usage/getting-started/_index.md

@@ -0,0 +1,5 @@
+---
+title: "Getting Started"
+---
+
+This will be written when a release of Flamenco 3 can actually be downloaded.

web/flamenco-io-site/content/usage/jobs-tasks-commands/_index.md

@@ -0,0 +1,18 @@
+---
+title: Jobs, Tasks, and Commands
+---
+
+TODO: write about the pipeline from job submission to command execution.
+
+
+## Task Statuses
+
+The following table shows the meaning of the different task statuses:
+
+| Status        | Meaning | Possible next status |
+| ------------- | ------- | ----------- |
+| `queued`      | Ready to be worked on by a Worker | `active`, `canceled` |
+| `active`      | Assigned to a worker for execution | `completed`, `canceled`, `failed`, `soft-failed` |
+| `soft-failed` | Same as `queued`, but has been failed by a worker in an earlier execution | `completed`, `failed`, `canceled` |
+| `completed`   | Worker executed the task succesfully | `requeued` |
+| `paused`      | Not yet implemented | |