[V18/V19] splitting module in many more little modules

[legalsylvain]

Hi all,

First thanks for this great module !
I just discovered mis_builder during the last OCA days (in the training courses), and it is very powerful tool. It is super easy, super fast and super simple. As Odoo.

However I have a question, regarding the design. When I discovered the code, I was surprised to see that the base module mis_builder, is depending on account :

• In fact MIS mean "Management Information System", that is not related to account.
• after all, it's totally possible to want the MIS feature out of accounting context. (with sub queries, on any model). In fact, we could imagine some people wanting to use this tools in a framework project, without any product / account module installed.
• in very practical terms, I have an instance, where accounting data are not complete / correct. (account moves are exported in EBP, with other account moves that comes from other softwares). As a result, the "bal" features will not works, as Odoo doesn't have all the accounting data (or sometime, moves are changed in EBP, but not in Odoo). In that context, having a lot of useless code installed is not optimal.

As a result,

• the "main" mis_builder module is quite big.
• Users that don't have accounting access right can not see MIS reports.

I have so the idea to split modules into many ones.

Structure changes

mis_builder changes : split into :

• mis_builder_base, depends = "base"
• mis_builder_board, depends = ["mis_builder_base", "board"], auto_install = True
• mis_builder_report_xlsx, depends = ["mis_builder_base", "report_xlsx"], auto_install = True
• mis_builder_date_range, depends = ["mis_builder_base", "date_range"], auto_install = True
• mis_builder_account, depends = ["mis_builder_base", "account"]

mis_builder_budget changes :

• now depends on ["mis_builder_account"] (before ["mis_builder", "account"]``)

Optionally

• keep an empty module mis_builder, depends = ["mis_builder_base", "mis_builder_board", "mis_builder_report_xlsx", "mis_builder_date_range", "mis_builder_account"], application=True, so for functional people, no changes, when making demo, etc.

Main UX changes :

• create a dedicated main "MIS" menu, with the existing icon : https://github.com/OCA/mis-builder/blob/18.0/mis_builder/static/description/icon.png and put all the existing MIS menu entries into the new dedicated menu. (reports & configuration)

Important note :

• in a previous thread, I faced some difficulties with other contributors, when I was in favor to refactor a module that was deployed in production, changing dependencies. (adding new ones in fact). mass_editing refactoring + discussion about stability
  As a result, my question here is ONLY "in an ideal world, Do you think it's better to split modules, or to keep a single big one ?".
  Once this first question answered, and if refactor is desired by PSC, we could focus on the next question : "on which version we are applying this refactor ? (current V18 version, or new V19 release).

Below the Pro / Cons list :
(I will update this list with your pro / cons items you will eventually mention in this issue)

Pro

• Lighter and smaller modules
• Easier migration of modules for each release
• Earlier migration. (as we don't have to wait for extra OCA migration like report_xlsx or date_range, to begin to work on the migration)
• better modularity, allow new usage of MIS buider as a framework tool, out of the account concept.
• Allow non accounting people to access MIS report (and optionaly MIS template & style configuration)
• Perhaps the module will be easier for new comers (developers) to understand. (For now, mis_builder is still a big thing.)
• Others points ?

Cons

• This will make more difficult to port bug fixes from a "all in one version" (like v16) to "a splitted version" (like 20).
• a less "product" vision. For the time being, mis_builder depends on many modules, and there is only 1 main module. (well 2 with the budget module).
• others points ?

Pending Questions

• I'm not sure how long such refactoring will take. (but if there is validation from mis_builder's PSCs, I'm willing to do a PoC in the coming weeks,at least on one split. )

Thanks for your attention !

CC : @sbidoul.

[sbidoul]

A few quick comments:

• extracting xlsx support: ok
• extracting the date_range dependency: ok
• creating security groups that do not depend on account is something I have long wanted to do, so it is possible to have reports that do not use account.move.line, use queries only, or use alternative move line sources
• moving the menus so MIS Reports can be accessed from outside the accounting menu sounds good
• having a core that does not depend on account: I'm not convinced about it at all as reporting on things that look like journal items is really THE core feature

[pedrobaeza]

Although the excel export requires the extra dependency, we already have it also on account_financial_report, so for me it's not critical to split it, and I will always install both.

Something similar happens to date_range one.

[legalsylvain]

Hi @sbidoul. Thanks for your complete answer.

A few quick comments:

extracting xlsx support: ok
extracting the date_range dependency: ok
creating security groups that do not depend on account is something I have long wanted to do, so it is possible to have reports that do not use account.move.line, use queries only, or use alternative move line sources
moving the menus so MIS Reports can be accessed from outside the accounting menu sounds good

👍 we are agree.

having a core that does not depend on account: I'm not convinced about it at all as reporting on things that look like journal items is really THE core feature

Well, maybe we don't see mis_builder module in the same way.
I just discovered this module two weeks ago and you are the original author of this module (since 2015).
At a first sight, discovering this module, I found it quite complex to contribute and understand. The main module contains 5500 python lines and 1100 XML lines. It's quite hard to grasp for newcomers.
Looking at the statistics, I get the impression that you are still the main contributor to this project today. (With very few PRs on 18.0 I realized recently, I am the third most human active contributor to this repository (over the last two years) !)
At the same time, at the last OCA Days, I heard many integrators say that mis_builder was a must-have module as soon as accounting was installed. Having more contributors and a PSC that includes more than one person could be a great long-term goal.

So, behind my proposal to split mis_builder into mis_builder_base and mis_builder_account there is also a proposal to divide complexity and clearly distinguish between basic features and accounting features, and make the code more accessible for newcomers.

Maybe I'm wrong !

That being said. I propose to try to make the migration from 18 to 19 in the next weeks, and in that occasion, to extract the minor dependencies (xlsx, date_range, ...) so mis_builder will depends only on account. It is a first step. And, we'll maybe talk about bigger split (base / account) in the next OCA days, for the version 20.
Are you agree with that last proposal ?
If yes, could you ping me when you merged all the 18.0 pending PRs so I can try to begin a migration on a up to date codebase.

thanks ! (and thanks again for this great tool)

[sbidoul]

Hi Sylvain, I'm not quite sure where you want to get to with your comment about the contributors. The stats sound misleading, though. Most activity has been on 16 so far with mostly forward ports on 18. These days, I focus on steering and code review, as well as some important fixes. I certainly welcome other people that would want to engage long term with MIS builder maintenance and become maintainers.

Regarding a core module without accounting, I think it would make things even more complex and the code is far from being ready for such split. Also, I worry it would set wrong expectations about the MIS Builder features. It is really designed to address fast accounting reports, and queries on other data is more the cherry on the cake.

If we want to extract xlsx and date range support to other modules, I'm fine with that, but I'd like to have it in 18, as it is will facilitate ports between 18 and 19. Indeed I suspect activity on 16 will slow down but we'll likely see still many contributions to 18.

[legalsylvain]

Hi Sylvain, I'm not quite sure where you want to get to with your comment about the contributors.

Sorry if I didn't express myself clearly.

My thinking behind this was to say that you were the main contributor and that this could eventually become problematic and tiring for you in the long run (but I may not have understood the situation correctly from the statistics, so please disregard this comment).

In any case, there was no criticism.

I think it would make things even more complex and the code is far from being ready for such split.

The objective of my proposal is (maybe!) to make technical onboarding more easy. If you think, it will generate more complex code, I totally agree that it's a bad idea !

If we want to extract xlsx and date range support to other modules, I'm fine with that, but I'd like to have it in 18, as it is will facilitate ports between 18 and 19. Indeed I suspect activity on 16 will slow down but we'll likely see still many contributions to 18.

OK with that roadmap !

[legalsylvain]

so, summary of the refactoring :

• split mis_builder into mis_builder and mis_builder_report_xlsx : [18.0][REF][ADD] mis_builder_report_xlsx #728.
• split mis_builder into mis_builder and mis_builder_date_range : TODO
• split mis_builder into mis_builder and mis_builder_board : TODO
• move menu entries out of accounting menu, into a new main menu "MIS Reports" : ([REF] mis_builder: create dedicated 'MIS Reports' Entry menu for mis_builder application #732
• split mis_builder into mis_builder_base and mis_builder_account : considered to add complexity.

[legalsylvain]

Hi @sbidoul #728 is ready for review and merge. Let me know if there is something to do from my side.
I'm waiting for mis_builder_report_xlsx refactor do be merged to make the refactor of date_range / board.

thanks !