Excessive processing time for larger files

[ctron]

Taking the following example file: https://access.redhat.com/security/data/csaf/v2/advisories/2018/rhsa-2018_3140.json

For some of the tests, it takes quite a while:

Running: test: optionalTest_6_2_1
test: optionalTest_6_2_1: 1:05.581 (m:ss.mmm)

Most importantly the schema test seems to run for hours, consuming 100% while processing.

[tschmidtb51]

Most importantly the schema test seems to run for hours, consuming 100% while processing.

With the same file? 🤔

[tschmidtb51]

IMHO, JavaScript does not have the highest performance... So maybe, we should implement the next tests in RUST 🤔

[ctron]

IMHO, JavaScript does not have the highest performance... So maybe, we should implement the next tests in RUST 🤔

Yes, I think what would be a good approach is to write those tests in Rust. It should then be possible to compile them to WebAssembly and use them the same way from JavaScript. But also have the ability to run them compiled to native code.

[tschmidtb51]

@secvisogram/service-dev: timeboxed approach: Please have a look why this takes so long and make suggestions how to improve, if possible.

[bernhardreiter]

It makes sense to understand the nature of the performance problem before implementing this in rust. (Example illustration: If there would be an exponential runtime growth over the number of entries in the algorithm, then even two magnitudes of speed advantages of an implementation in Rust or Go would not help.)

[rainer-exxcellent]

I did a performance analysis of the test with the rhsa-2018_3140.json file on my Laptop.
Here are the results:

Testname	Time(ms)	Time(min)	Remark
informativeTest_6_3_8	4567166,567	76 : 07	about 35000 calls to Hunspell, each took about 120 ms
csaf_2_0	168202,8742	2 : 48	Ajv JSON schema validator
optionalTest_6_2_20	106131,8331	1 : 46	Ajv JSON schema validator
csaf_2_0_strict	98179,5353	1 : 38	Ajv JSON schema validator
optionalTest_6_2_1	74227,2375	1 : 14	Ajv JSON schema validator
mandatoryTest_6_1_1	52950,7155	0 : 52
mandatoryTest_6_1_3	16349,0646	0 : 16
optionalTest_6_2_19	10627,4618	0 : 10
informativeTest_6_3_7	4513,7357	0 : 04
optionalTest_6_2_13	151,7905	0 : 00
mandatoryTest_6_1_7	102,5601	0 : 00
mandatoryTest_6_1_6	68,4435	0 : 00
mandatoryTest_6_1_2	27,7305	0 : 00
mandatoryTest_6_1_13	26,8758	0 : 00
informativeTest_6_3_9	22,1699	0 : 00
optionalTest_6_2_8	16,9061	0 : 00
optionalTest_6_2_3	16,2779	0 : 00
optionalTest_6_2_9	15,0154	0 : 00
informativeTest_6_3_5	9,6898	0 : 00
informativeTest_6_3_6	9,0975	0 : 00
mandatoryTest_6_1_31	8,2825	0 : 00
optionalTest_6_2_2	7,4811	0 : 00
mandatoryTest_6_1_29	6,6613	0 : 00
optionalTest_6_2_16	6,5195	0 : 00
mandatoryTest_6_1_14	5,2445	0 : 00
informativeTest_6_3_10	3,8843	0 : 00
optionalTest_6_2_5	3,8236	0 : 00
optionalTest_6_2_18	3,7355	0 : 00
optionalTest_6_2_6	3,4973	0 : 00
mandatoryTest_6_1_25	3,3507	0 : 00
mandatoryTest_6_1_9	2,5774	0 : 00
optionalTest_6_2_14	2,1244	0 : 00
informativeTest_6_3_11	1,8043	0 : 00
informativeTest_6_3_2	1,5664	0 : 00
mandatoryTest_6_1_21	1,4876	0 : 00
mandatoryTest_6_1_8	1,4084	0 : 00
mandatoryTest_6_1_16	1,1267	0 : 00
informativeTest_6_3_1	0,9568	0 : 00
mandatoryTest_6_1_12	0,9449	0 : 00
optionalTest_6_2_11	0,928	0 : 00
informativeTest_6_3_3	0,835	0 : 00
mandatoryTest_6_1_30	0,7568	0 : 00
optionalTest_6_2_17	0,7447	0 : 00
mandatoryTest_6_1_10	0,7361	0 : 00
optionalTest_6_2_7	0,7004	0 : 00
optionalTest_6_2_4	0,6932	0 : 00
optionalTest_6_2_10	0,5545	0 : 00
mandatoryTest_6_1_11	0,5102	0 : 00
mandatoryTest_6_1_32	0,4823	0 : 00
informativeTest_6_3_4	0,4517	0 : 00
mandatoryTest_6_1_33	0,4487	0 : 00
mandatoryTest_6_1_23	0,4031	0 : 00
mandatoryTest_6_1_15	0,3983	0 : 00
mandatoryTest_6_1_20	0,3907	0 : 00
mandatoryTest_6_1_17	0,3871	0 : 00
mandatoryTest_6_1_22	0,3852	0 : 00
mandatoryTest_6_1_4	0,3576	0 : 00
mandatoryTest_6_1_5	0,3554	0 : 00
mandatoryTest_6_1_24	0,3198	0 : 00
mandatoryTest_6_1_26	0,2881	0 : 00
mandatoryTest_6_1_27_11	0,273	0 : 00
mandatoryTest_6_1_27_3	0,2421	0 : 00
mandatoryTest_6_1_27_4	0,221	0 : 00
mandatoryTest_6_1_27_1	0,2107	0 : 00
mandatoryTest_6_1_27_10	0,2059	0 : 00
mandatoryTest_6_1_18	0,1939	0 : 00
optionalTest_6_2_15	0,1901	0 : 00
optionalTest_6_2_12	0,1769	0 : 00
mandatoryTest_6_1_19	0,174	0 : 00
mandatoryTest_6_1_27_2	0,1454	0 : 00
mandatoryTest_6_1_27_9	0,1452	0 : 00
mandatoryTest_6_1_27_5	0,1413	0 : 00
mandatoryTest_6_1_27_6	0,1358	0 : 00
mandatoryTest_6_1_27_7	0,133	0 : 00
mandatoryTest_6_1_27_8	0,1309	0 : 00
mandatoryTest_6_1_28	0,1118	0 : 00

[rainer-exxcellent]

I think we have a good chance of optimizing the informativeTest_6_3_8 test.
The tests that use the Ajv JSON schema validator seem to be more difficult to optimize.
We should consider running the tests in parallel.

[tschmidtb51]

I think we have a good chance of optimizing the informativeTest_6_3_8 test.

You can proceed with this.

The tests that use the Ajv JSON schema validator seem to be more difficult to optimize.

This is not a priority right now.

We should consider running the tests in parallel.

Feel free to suggest a concept for the next meeting.