-
Notifications
You must be signed in to change notification settings - Fork 4
Contribution
After release 1.6.7 (excluding release 1.6.7), the program is going to use JSON file for storing websites. This is a template in case someone wants to add a site:
{ "hasmainlink": 0 or 1, "key1": "", "key2": "", "key3": "", "name": "", "plusorspace": 0 or 1, "searchurl": "", "type": "", "url": "" }
For versions and including 1.6.7, websites are stored in a SQLite database file. You can manipulate it via two ways:
- db_adder.py that is in the others folder.
- SQLite Browser https://sqlitebrowser.org/
There are 9 properties needed to add:
- name: the site's name
- url: the site's normal url (this may vary so check its own header below)
- searchurl: the url that the program uses to search.
- key1_id
- key2_id
- key3_id
- type_id
- hasmainlink
- plusorspace
First, check if the site you want to add has a search button. If it does, search something randomly but one that gives results. Then, check the URL if it has in it what you searched, because the program searches and scrapes via the URL.
Here's an example:
I searched "the u" in tokybook.com, the search URL is the following:
https://tokybook.com/?s=the+u
Here is what we get from inspecting a result, this is necessary especially for link and the keys (1,2,3):
As you can see, the URL has the search string in it. So what you do here is, whatever before the search string is the search URL. Therefore, the first three properties are filled as such:
- name: Tokybook
- url:
Right click on a result, inspect it (developer tools, you should get something like the screenshot above), check if the link in the starts with the site's url. Some sites put /torrent/... instead of www.torrent.com/torrent/..., so you have to put whatever there is before /torrent/... so the program would add it. Here's an example of each type:
- If it has the main link, you put the URL normal: https://tokybook.com/
- If it doesn't have main link, and let's say the link of the results start with /torrent/..., you put the URL https://tokybook.com. Without the / at the end because the URL already starts with /, if you put the / it will try to connect to https://tokybook.com/**/**torrent/... which is wrong 90% of the time.
- searchurl: you put the searchurl whatever there is before the+u, in this case: https://tokybook.com/?s= Note: "?s=" may change, as long as the search string is in the url you can take anything before it, maybe it says "search" instead of "s", and so on.
key1_id will be the parent container in the HTML file. All of the keys are grabbed via inspecting one of the site's results. In this case, here's the image as well which will helps us identify the key1:
You see the is highlighted by the browser because that's what I inspected on, and before it there is
This is the parent container! Key1, key2, and key3 are all from this line. Because the program searches for the URL (<a...>) with scraping these keys. In this case, the key1 is "h2"'s id from within the database. Key1 just says the program what the name of the container is.
key2_id is the beginning of an identifier. In the h2 line, you can see in it there is class and itemprop (we won't look what's in these, that's key3_id). itemprop doesn't matter, in key2_id we either look for "id" or "class". In this case we have class, so key2_id will be the id of "class" from within the database.
key3_id is whatever is in key2_id. In this case, the class is "class="entry-title"", therefore key3_id is "entry-title". You find its id in the database and you insert it.
In that case, you have to use the DB Browser. Go to "Execute SQL" and use the following command:
INSERT INTO Keys3 (name) VALUES ("name")
Instead of the second name after the VALUES you insert the string that you want to add and you run the command.
Here you simply put the type's id from the Types table. Is it a software? game? course? You specify it here with its own id accordingly.
In the first header where we discussed the link, I mentioned that if the link retrieved from the site doesn't start with the main site's link, then we add the link accordingly for it to match. So if that is the case, you have to put hasmainlink 1. Here's each number explained:
- 0: the program retrieves the result links from the site with the main site's link, so no need for the program to add the link before the result.
- 1: the program retrieves the result links from the site without the main site's link, so the program adds it on its own to make it a valid link.
This accepts the value 0 or 1. O means: the search string is separated by +, for example: grand+theft+auto 1 means: the search string is separated by space %20, for example: grand theft auto
