training WLDN model

[YH-88]

Hi, I have tried to run the nntrain_direct_useScores. Py with your training set and the file core_wln_global/model-300-3-direct/train.cbond_detailed generated by core_wln_global, but there is an error. Did you have this problem at that time? Could you help me explain the reason?
Much thanks.

[connorcoley]

It doesn't look like there are any errors here. A small percentage of the reactions (<1% if I recall correctly) are unable to be completely described by the graph edit formalism. This is because the recorded reaction might include changes in protonation state, which are not always described consistently in the database.

[YH-88]

I got it. Thank you a lot.

[YH-88]

Hello, Could you help me with the following question? In running the code python ... /scripts/eval_by_smiles.py --gold ... /data/valid.txt.proc --pred model-core16-500-3-max150-direct-use-Scores/valid.cbond_detailed_2400000 --bonds_as_doubles true, the accuracy result appears, e.g. acc@1: 0.8321. I want to know if this accuracy represents the accuracy of the test model WLDN, with the code file nntest_direct_usescore.py, or the accuracy of the conversion from reaction center to SMILES?Thanks a lot.

[connorcoley]

That accuracy (~83%) would be the accuracy of the whole pipeline from reactants to predicted product SMILES for the validation dataset. The input file to the eval_by_smiles script is the output from nntest_direct_usescore

[YH-88]

Thank you very much.

[YH-88]

Hello Dr Coley,
Could you help me answer a few questions?
(1)My WLN model is quite accurate. However, after training and verifying the WLDN model, the accuracy is 0 when running eval_by_smiles. When running mol graph_ direct_ useScores.py_, printing found_ true, the result is False. Comparing the real results with eval_ by_ Smiles predicted the results, it is found that all the predicted results are correct. Do you have any suggestions to solve this problem?
(2)Are the contents of clean_rxns_presani and clean_rxns_postsani manually defined reaction rules in eval_by_smiles?
Thank you very much.

[connorcoley]

(1) I'm a little confused here. Does the eval_by_smiles script show you that the predicted SMILES is equivalent to the ground truth SMILES? That's how the accuracies (top1, top2, top3, top5) are calculated, so I don't know why they'd disagree.

(2) Yes, these are manually defined "cleanup" rules (developed using the training data) that let applying graph edits recover the ground truth product

[YH-88]

Hello Dr Coley,
(1) The first of all candidate products predicted by the eval_by_smiles script in each reaction is consistent with the real product SMILES.Does the accuracy of eval_by_smiles of 0 have anything to do with the "cleanup" rules? Because I trained the model with my own reaction, but using your reaction rules.
(2）Can I extract the reaction rule using the code generate_reaction_templates.py? Or do you have any suggestions on how to manually extract reflection rules from large amounts of data？
Thank you again.

[connorcoley]

(1) The cleanup rules are unlikely to have a negative effect; I'm still not sure why the top predicted SMILES would appear correct but be recorded as inaccurate.

(2) Retrosynthetic template extraction is the focus of another codebase you might be interested in: https://github.com/connorcoley/rdchiral. Slightly modified versions of it are used in the ASKCOS and AIZynthFinder programs.

[YH-88]

Thank you so much for helping me answer so many questions. Thank you again.

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "connorcoley/rexgen_direct" <notifications@github.com&gt;; 发送时间:&nbsp;2021年2月28日(星期天) 上午10:32 收件人:&nbsp;"connorcoley/rexgen_direct"<rexgen_direct@noreply.github.com&gt;; 抄送:&nbsp;"ю。"<1542643549@qq.com&gt;;"Author"<author@noreply.github.com&gt;; 主题:&nbsp;Re: [connorcoley/rexgen_direct] training WLDN model (#22) (1) The cleanup rules are unlikely to have a negative effect; I'm still not sure why the top predicted SMILES would appear correct but be recorded as inaccurate. (2) Retrosynthetic template extraction is the focus of another codebase you might be interested in: https://github.com/connorcoley/rdchiral. Slightly modified versions of it are used in the ASKCOS and AIZynthFinder programs. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[YH-88]

Hello Dr Coley,
What is the meaning of rk,rk_sani = 11,11 in scripts/eval_by_smiles? And why define rk and rk_sani? Thank a lot.

[connorcoley]

rk is the rank at which the ground truth product was proposed; rk_sani is the same, but using the sanitized SMILES for the comparison. We initialize those to 11 before checking what was proposed so that if the true product was not found, the "rank" is 11 and it doesn't count toward our other top-k metrics

[YH-88]

Thank you very much. I still have a few doubts and need your help.
（1）The unmapped two identical smiles before the number 1 are real or predicted product, such as the first data in your test set [CH2:23]1[O:24][CH2:25][CH2:26][CH2:27]1.[F:1][c:2]1c:3[cH:4]c:5c:6[cH:7]1.[H-:22].[NH2:13][c:14]1[s:15][cH:16][cH:17][c:18]1[C:19]#[N:20].[Na+:21] [c:2]1([NH:13][c:14]2[s:15][cH:16][cH:17][c:18]2[C:19]#[N:20])c:3[cH:4]c:5c:6[cH:7]1 N#Cc1ccsc1Nc1cc(F)c(F)cc1N+[O-] N#Cc1ccsc1Nc1cc(F)c(F)cc1N+[O-] 1 [F-].[H-].N#Cc1ccsc1Nc1cc(F)c(F)cc1N+[O-].[Na+].C1CCOC1 Nc1cc(F)c(F)cc1N+[O-].[Na+].[F-].[H-].N#Cc1ccsc1.C1CCOC1 [F-].[H-].N#Cc1ccsc1Nc1cc(F)c(N+[O-])cc1F.[Na+].C1CCOC1 [F-].N#Cc1ccsc1N.[Na+].[H-].O=N+c1ccc(F)c(F)c1.C1CCOC1 [F-].N#Cc1ccsc1N.[Na+].[H-].O=N+c1cc(F)ccc1F.C1CCOC1 O=N+c1cc(F)c(F)cc1F.[H-].NCc1ccsc1N.C1CCOC1.[Na+] N#Cc1ccsc1N.[Na+].[H-].O.C1CCOC1.[O-].Nc1cc(F)c(F)cc1F
（2）Why are the predicted reaction center scores positive or negative, and some are all negative?
（3）Can this model predict multi-step reactions of reactants and products without considering intermediate processes?
Thank you again.

[connorcoley]

(1) In that file are predicted SMILES strings
(2) The scores don't have a meaning on an absolute scale, just a relative scale. Lower scores are more favorable
(3) The model is designed for single-step reactions; multi-step reactions would be difficult or impossible to predict, depending on what happens in those multiple steps.