Skip to content

Contribution

Serj edited this page Jun 2, 2024 · 3 revisions

After release 1.6.7 (excluding release 1.6.7), the program is going to use JSON file for storing websites. This is a template in case someone wants to add a site:

{ "hasmainlink": 0 or 1, "key1": "", "key2": "", "key3": "", "name": "", "plusorspace": 0 or 1, "searchurl": "", "type": "", "url": "" }

For versions and including 1.6.7, websites are stored in a SQLite database file. You can manipulate it via two ways:

  1. db_adder.py that is in the others folder.
  2. SQLite Browser https://sqlitebrowser.org/

There are 9 properties needed to add:

  1. name: the site's name
  2. url: the site's normal url (this may vary so check its own header below)
  3. searchurl: the url that the program uses to search.
  4. key1_id
  5. key2_id
  6. key3_id
  7. type_id
  8. hasmainlink
  9. plusorspace

General steps

name, url, searchurl Discussion

First, check if the site you want to add has a search button. If it does, search something randomly but one that gives results. Then, check the URL if it has in it what you searched, because the program searches and scrapes via the URL.

Here's an example:
I searched "the u" in tokybook.com, the search URL is the following:
https://tokybook.com/?s=the+u
Here is what we get from inspecting a result, this is necessary especially for link and the keys (1,2,3):

What we get from inspecting a result

As you can see, the URL has the search string in it. So what you do here is, whatever before the search string is the search URL. Therefore, the first three properties are filled as such:

  1. name: Tokybook
  2. url:
    Right click on a result, inspect it (developer tools, you should get something like the screenshot above), check if the link in the starts with the site's url. Some sites put /torrent/... instead of www.torrent.com/torrent/..., so you have to put whatever there is before /torrent/... so the program would add it. Here's an example of each type:
  • If it has the main link, you put the URL normal: https://tokybook.com/
  • If it doesn't have main link, and let's say the link of the results start with /torrent/..., you put the URL https://tokybook.com. Without the / at the end because the URL already starts with /, if you put the / it will try to connect to https://tokybook.com/**/**torrent/... which is wrong 90% of the time.
  1. searchurl: you put the searchurl whatever there is before the+u, in this case: https://tokybook.com/?s= Note: "?s=" may change, as long as the search string is in the url you can take anything before it, maybe it says "search" instead of "s", and so on.

key1_id

key1_id will be the parent container in the HTML file. All of the keys are grabbed via inspecting one of the site's results. In this case, here's the image as well which will helps us identify the key1:

What we get from inspecting a result

You see the is highlighted by the browser because that's what I inspected on, and before it there is

This is the parent container! Key1, key2, and key3 are all from this line. Because the program searches for the URL (<a...>) with scraping these keys. In this case, the key1 is "h2"'s id from within the database. Key1 just says the program what the name of the container is.

key2_id

key2_id is the beginning of an identifier. In the h2 line, you can see in it there is class and itemprop (we won't look what's in these, that's key3_id). itemprop doesn't matter, in key2_id we either look for "id" or "class". In this case we have class, so key2_id will be the id of "class" from within the database.

key3_id

key3_id is whatever is in key2_id. In this case, the class is "class="entry-title"", therefore key3_id is "entry-title". You find its id in the database and you insert it.

What if the key3_id isn't in the database?

In that case, you have to use the DB Browser. Go to "Execute SQL" and use the following command: INSERT INTO Keys3 (name) VALUES ("name")
Instead of the second name after the VALUES you insert the string that you want to add and you run the command.

type_id

Here you simply put the type's id from the Types table. Is it a software? game? course? You specify it here with its own id accordingly.

hasmainlink

In the first header where we discussed the link, I mentioned that if the link retrieved from the site doesn't start with the main site's link, then we add the link accordingly for it to match. So if that is the case, you have to put hasmainlink 1. Here's each number explained:

  1. 0: the program retrieves the result links from the site with the main site's link, so no need for the program to add the link before the result.
  2. 1: the program retrieves the result links from the site without the main site's link, so the program adds it on its own to make it a valid link.

plusorspace

This accepts the value 0 or 1. O means: the search string is separated by +, for example: grand+theft+auto 1 means: the search string is separated by space %20, for example: grand theft auto

Clone this wiki locally