[feat] : Common Response Schema (CRS)

[brian-pond]

Scenario

1. Assume a BTU Task runs a Python function, whether on-demand or via a schedule.
2. Thet Python function loops through 100 documents.
3. For each document, it transmits an email via Mandrill (Mailchimp)
   • A try + except wisely prevents a failure in 1 iteration from terminating the entire loop.
4. The function ends after the final loop.

Functionally, what are the possible outcomes?

• All emails transmitted successfully (there was an "ok" response from Mandrill on every iteration)
• Some emails transmitted successfully.
• No emails transmit successfully.

What is the Result in the BTU Task Log?

• It will always be a "Success", because of the try-except inside the Python function.

The Problem

Unless a human examines every single BTU Task Log every day, then failures will go ignored.

If 3 of the 100 emails failed, the root cause might exist in the documents themselves (e.g. typo in the email address).
But if 100% of the emails fail to transmit, the root cause might be a TCP network failure. In that case, the solution might be waiting 5 minutes, and trying again. Whether automatically, or with manual human intervention.

But today, there exists no means to programmatically identify and act on these results. The Task succeeds because of the loop handling, and the true results must be examined with human eyes.

Desired Enhancement

• BTU to somehow capture the inner details about its Task's execution.
• BTU to trigger alerts or warnings, based on those details.

[brian-pond]

Options

Option 1: Never Write Ordinary Loops

One possibility is to never write ordinary loops in Python functions intended for BTU. Instead, use the BTU_AWARE_FUNCTION object to enqueue each iteration separately.

Disadvantages:

• Forces breaking a loop into sub-Tasks, even when that is not desirable.
• Does not solve multiple depths of looping.

class create_payments(BTU_AWARE_FUNCTION):

    @frappe.whitelist()
    def run(self, **kwargs):
        customer_list = automation_get_eligible_customers()
        for index, customer_key in enumerate(customer_list):
            TaskComponent(
                btu_task_id=self.btu_task_id,
                btu_component_id=index+1,
                btu_task_schedule_id=self.btu_task_schedule_id,
                frappe_site_name=frappe.local.site,
                function=automation_pay_orders_by_customer,
                customer_key=customer_key
            ).enqueue()
            print(f"* Enqueued a Task Component job for customer = {customer_key}")

        # end of for loop
        print(f"\u2713 {len(customer_list)} task component jobs enqueued.")

---

Option 2: Common Response Schema (CRS)

When an arbitrary Python function is written, the author can return any object. I could create a new class CommonResponseSchema(). If the author returns that class, the inner data could be parsed and acted upon.

Option 3: Send an HTTP POST to an ERPNext API endpoint

Gather results about the function's execution. Then transmit to an ERPNext endpoint, which will process it accordingly (write to logs, send alerts, etc.)