Add data

[rhnvrm]

The bot has run out of data to tweet. If any one wants to volunteer and contribute to adding data feel free to ping either me or @kumbhakaran on this thread and we can help you get started and things to take care of while adding data.

[rhnvrm]

Steps to add a file:

• Copy data from CAD archives into a new file
• Run the following regex replace for fixing the spacing for line headers: %s/^3\./\n3./g (3 will need to be changed to appropriate headline)
• Run the following regex to trim spaces from lines at the beginning: %s/^ *//g
• Run the following regex to remove double spaces: %s/\s\s\+/ /g (8e339a3)
• Manually review any issues that are visible. (document them here)
• Append the data in this file to data.txt (in case the chapter is ending, truncate the file and move it to data-2.txt and so on and add the work done above to data.txt)
• In case there are duplicate rows while copying, we can run awk -i inplace '!seen[$0]++' data.txt

I used the above steps to add the following commits:

57924f4, cd3a347, e2aeba5, c4145b5, 5df311f, b268bed, 693705c

[rhnvrm]

curl https://www.constitutionofindia.net/constitution_assembly_debates/volume/7/1948-11-26 | htmlq '.ckeditor-content .row' -p  | htmlq -r .tooltiptext -r .summary-block -r .support-text -r .social-block -r .abt-events -t | sed -e 's/^ *//g' |sed -e 's/\s\s\+/ /g' | sed '/^$/d' | awk -i inplace '!seen[$0]++' | sed 's/^7\./\n7./g' >> data.txt

[rhnvrm]

Added Volume 8 - 3944ede

This commit also automates generating the file with a bash script.