How to get original functions of tokenized function codes that are present in data.zip?

[anks12297]

Hi, thanks for sharing your great work!

I have downloaded your dataset (data.zip file). I believe that train.token.code files for java and python have tokenized codes based on AST. Links for original dataset for java and python are provided on Readme page but how could I map tokenized function code present in train.token.code to the original function?

It could be very helpful if you provide some guidelines to get original function and tokenized function code pairs?
THANK YOU!

[gingasan]

Thank you for your attention on my work. I still save the original corpus of Python (link bellow). But Java is missing by accident. I am sorry for that. https://drive.google.com/file/d/1Zbven9w21UMs3SIg_CtYGgiOdF9BSP9H/view?usp=sharing Best, Hongqiu

…

----- 原始邮件 ----- 发件人: "anks12297" ***@***.***> 收件人: "gingasan/sit3" ***@***.***> 抄送: "Subscribed" ***@***.***> 发送时间: 星期三, 2021年 12 月 22日 下午 1:59:07 主题: [gingasan/sit3] How to get original functions of tokenized function codes that are present in data.zip? (Issue #6) Hi, thanks for sharing your great work! I have downloaded your dataset (data.zip file). I believe that train.token.code files for java and python have tokenized codes based on AST. Links for original dataset for java and python are provided on Readme page but how could I map tokenized function code present in train.token.code to the original function? It could be very helpful if you provide some guidelines to get original function and tokenized function code pairs? THANK YOU!

-- Reply to this email directly or view it on GitHub: #6 You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

[Li-Lixuan]

Thank you for your attention on my work. I still save the original corpus of Python (link bellow). But Java is missing by accident. I am sorry for that. https://drive.google.com/file/d/1Zbven9w21UMs3SIg_CtYGgiOdF9BSP9H/view?usp=sharing Best, Hongqiu
…
----- 原始邮件 ----- 发件人: "anks12297" @.> 收件人: "gingasan/sit3" @.> 抄送: "Subscribed" @.> 发送时间: 星期三, 2021年 12 月 22日 下午 1:59:07 主题: [gingasan/sit3] How to get original functions of tokenized function codes that are present in data.zip? (Issue #6) Hi, thanks for sharing your great work! I have downloaded your dataset (data.zip file). I believe that train.token.code files for java and python have tokenized codes based on AST. Links for original dataset for java and python are provided on Readme page but how could I map tokenized function code present in train.token.code to the original function? It could be very helpful if you provide some guidelines to get original function and tokenized function code pairs? THANK YOU!
-- Reply to this email directly or view it on GitHub: #6 You are receiving this because you are subscribed to this thread. Message ID: @.>

It seems not the dataset with original split. Could you provide the original corpus with original split? It would be very helpful if you did. Thanks a lot!

[gingasan]

Yes. I write an issue of this problem in the repo. You can find the original split in https://github.com/GoneZ5/SCRIPT.

----- 原始邮件 ----- 发件人: "pomelo715" ***@***.***> 收件人: "gingasan/sit3" ***@***.***> 抄送: "gingasan" ***@***.***>, "Comment" ***@***.***> 发送时间: 星期二, 2022年 7 月 19日 下午 1:45:16 主题: Re: [gingasan/sit3] How to get original functions of tokenized function codes that are present in data.zip? (Issue #6)

Thank you for your attention on my work. I still save the original corpus of Python (link bellow). But Java is missing by accident. I am sorry for that. https://drive.google.com/file/d/1Zbven9w21UMs3SIg_CtYGgiOdF9BSP9H/view?usp=sharing Best, Hongqiu […](#) ----- 原始邮件 ----- 发件人: "anks12297" ***@***.***> 收件人: "gingasan/sit3" ***@***.***> 抄送: "Subscribed" ***@***.***> 发送时间: 星期三, 2021年 12 月 22日 下午 1:59:07 主题: [gingasan/sit3] How to get original functions of tokenized function codes that are present in data.zip? (Issue #6) Hi, thanks for sharing your great work! I have downloaded your dataset (data.zip file). I believe that train.token.code files for java and python have tokenized codes based on AST. Links for original dataset for java and python are provided on Readme page but how could I map tokenized function code present in train.token.code to the original function? It could be very helpful if you provide some guidelines to get original function and tokenized function code pairs? THANK YOU! -- Reply to this email directly or view it on GitHub: #6 You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

It seems not the dataset with original split. Could you provide the original corpus with original split? It would be very helpful if you did. Thanks a lot!

…

-- Reply to this email directly or view it on GitHub: #6 (comment) You are receiving this because you commented. Message ID: ***@***.***>