BTU Road Map (Short Term)

[brian-pond]

• Create the concept of "Transient Tasks", that run exactly 1 time. This will offer an alternative to Python function 'frappe.enqueue()', with the benefits of BTU logging, email capability, and UI capability.

  • Transient Tasks will be created through Python code only; not the web UI.
  • Transient Tasks should be deleted periodically. This avoids flooding the BTU Task table with too many rows.
  • By default, do not display them on the BTU Task list page.

• Offer an Advanced Logging option: Create 'In-Progress' Task Logs.

  • Under certain circumstances (timeouts, Linux errors) it's possible for Python RQ code to terminate early. Because of this early termination, there is no record anywhere in BTU that the Task actually started.
  • By writing a Log immediately, and giving it an 'In-Progress' status, it will be possible generate reports/alerts that show unfinished Tasks. For example, any 'In-Progress' Log document where the creation is greater than 1 hour ago.
  • When a Task finishes inside Python RQ, instead of creating a new Log document, it should update the existing one and change the status from 'In-Progress' to either 'Success' or 'Fail'

• Add a button that creates built-in "maintenance" Tasks, that help BTU manage itself.

  • Example: A Task that emails System Managers when a Task dies inside Python RQ, based on 'In-Progress' logs.
  • Example: A Task that periodically deletes old Transient Tasks.
  • Example: Email a Daily Summary report, showing info like Task Schedules, last execution time, last result, next execution time.

[brian-pond]

Here is an example of an API for creating and executing Transient Tasks.

from btu.btu_api import TransientTask

# just some local variables; I could have skipped and set inside the call below.
path_to_my_function = "erpnext.selling.doctype.customer.customer.my_special_function"
description = "My Task Description"

# create and schedule a BTU Task that is transient:
TransientTask.create_new_transient(
    function_path = path_to_my_function,
    description = description,
    max_task_duration='6000s',
    my_function_arg_1 = "foo",
    my_function_arg_2 = "bar",
    my_function_arg_3 = "baz"
).enqueue(queue_name='long')

The code above does something very similar to frappe.enqueue().
The difference is you get almost all the benefits of BTU:

• Logs are viewable via the web pages.
• Capturing stdout, plus your function's return object.
• Email on completion, etc.
• Execution time
• You know exactly what arguments were passed to the function, because they're captured in a BTU Task document.

[brian-pond]

BTU now has Transient Tasks and In-Progress logging. Still need to create some maintenance routines.