4 Commits

Author	SHA1	Message	Date
Sybren A. Stüvel	7d5aae25b5	Manager: add timeout checks for workers	2022-06-13 12:33:22 +02:00
Sybren A. Stüvel	fe1627dd85	Cleanup: timeout checker, move task-specific code to tasks.go ... Just a cleanup to prepare for the addition of worker timeouts.	2022-06-10 14:58:44 +02:00
Sybren A. Stüvel	13307c5a24	Manager: add canary test to timeout checker unit test ... The `TestTaskTimeout()` unit test assumes specific durations for initial & subsequent sleeps of the timeout checker. The test will fail quite cryptically when that assumption doesn't hold, so just test for it at the start of the unit test.	2022-06-10 14:53:23 +02:00
Sybren A. Stüvel	d90a8b987d	Manager: Task Timeout Checker ... Tasks that are in state `active` but haven't been 'touched' by a Worker for 10 minutes or longer will transition to state `failed`. In the future, it might be better to move the decision about which state is suitable to the Task State Machine service, so that it can be smarter and take the history of the task into account. Going to `soft-failed` first might be a nice touch.	2022-06-10 14:32:02 +02:00