It seems that keyword search functionality relies on an external API rather than the local database. And the external API is not reachable.
The following suggests we can access google.com but not the api.
(base) zj@R7000:~/桌面$ export http_proxy=http://127.0.0.1:10808
(base) zj@R7000:~/桌面$ export https_proxy=http://127.0.0.1:10808
(base) zj@R7000:~/桌面$ curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
(base) zj@R7000:~/桌面$ ping mtab.app
ping: mtab.app: 未知的名称或服务
(base) zj@R7000:~/桌面$ dig mtab.app @8.8.8.8
; <<>> DiG 9.18.30-0ubuntu0.20.04.2-Ubuntu <<>> mtab.app @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 29342
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;mtab.app. IN A
;; AUTHORITY SECTION:
app. 286 IN SOA ns-tld1.charlestonroadregistry.com. cloud-dns-hostmaster.google.com. 101427 21600 3600 259200 900
;; Query time: 279 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Tue Aug 19 15:11:58 CST 2025
;; MSG SIZE rcvd: 135
(base) zj@R7000:~/桌面$ nslookup mtab.app 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
** server can't find mtab.app: NXDOMAIN
more information by llm:
Issue: DNS Resolution Failure for mtab.app API in KeywordSearch Class
Description
The KeywordSearch class in keyword_search.py fails to connect to the mtab.app API (https://mtab.app/api/v1/search) due to a DNS resolution failure. The domain mtab.app returns an NXDOMAIN error, indicating it is either decommissioned, expired, or not registered. This breaks the keyword search functionality, preventing mapping of keywords (e.g., "Belgium") to Wikidata QIDs.
Steps to Reproduce
Run the following Python code in the WikiDB project environment:import requests
response = requests.get("https://mtab.app/api/v1/search", params={"q": "Belgium", "limit": 5, "m": "a", "lang": "en", "info": 1})
print(response.json())
Alternatively, use the KeywordSearch class:from keyword_search import KeywordSearch
ks = KeywordSearch()
results, runtime = ks.get("Belgium", limit=5, mode="a", lang="en", info=1)
print(results)
Observe the error:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='mtab.app', port=443): Max retries exceeded with url: /api/v1/search?q=Belgium&limit=5&m=a&lang=en&info=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Verify DNS failure with command-line tools:nslookup mtab.app 8.8.8.8
Output: server can't find mtab.app: NXDOMAIN
dig mtab.app @8.8.8.8
Output: status: NXDOMAIN
Expected Behavior
The KeywordSearch class should successfully query the mtab.app API and return a list of Wikidata QIDs, scores, labels, and descriptions for the given keyword (e.g., ["Q31", 1.0, "Belgium", "country in Western Europe"]).
Actual Behavior
The API request fails due to DNS resolution failure (mtab.app is not resolvable), rendering the keyword search functionality inoperable.
Environment
OS: Ubuntu 20.04
Python: 3.6 (in a conda environment)
WikiDB Commit: [Specify commit hash if known, e.g., latest from master as of August 2025]
Network: Tested with and without a proxy (http://127.0.0.1:10808); issue persists in both cases
Date Tested: August 19, 2025
Additional Context
The mTab platform’s main website (https://mtab.com) is accessible, suggesting the API may have moved to a new endpoint (e.g., https://api.mtab.com) or been decommissioned.
The WikiDB project’s README.md and code reference mtab.app as of 2022, but the domain is no longer resolvable as of August 2025.
Tested with Google’s DNS (8.8.8.8), confirming the NXDOMAIN error is not due to local network issues or regional restrictions.
Suggested Fixes
Update API Endpoint: If mTab has moved the API to a new domain (e.g., api.mtab.com), update KeywordSearch.URL in keyword_search.py.
Use Alternative API: Replace mtab.app with Wikidata’s wbsearchentities API (https://www.wikidata.org/w/api.php?action=wbsearchentities) for keyword-to-QID mapping. Example implementation:import requests
def search_wikidata(query, limit=5, lang="en"):
response = requests.get(
"https://www.wikidata.org/w/api.php",
params={"action": "wbsearchentities", "search": query, "language": lang, "format": "json", "limit": limit}
)
results = response.json().get("search", [])
return [[r["id"], 1.0, r.get("label", ""), r.get("description", "")] for r in results], 0
Local Index: Build a local full-text index (e.g., using Whoosh) for labels/descriptions in wikidb.lmdb to enable offline keyword search, though this requires significant storage.
Request
Could you confirm if the mtab.app API is still active or if it has been replaced? If the endpoint has changed, please provide the new URL or documentation. Alternatively, consider integrating a fallback API (e.g., Wikidata’s wbsearchentities) to ensure the keyword search functionality remains robust.
Thank you for maintaining this project! Let me know if you need more details or assistance testing potential fixes.
It seems that keyword search functionality relies on an external API rather than the local database. And the external API is not reachable.
The following suggests we can access google.com but not the api.
more information by llm:
Issue: DNS Resolution Failure for mtab.app API in KeywordSearch Class
Description
The KeywordSearch class in keyword_search.py fails to connect to the mtab.app API (https://mtab.app/api/v1/search) due to a DNS resolution failure. The domain mtab.app returns an NXDOMAIN error, indicating it is either decommissioned, expired, or not registered. This breaks the keyword search functionality, preventing mapping of keywords (e.g., "Belgium") to Wikidata QIDs.
Steps to Reproduce
Run the following Python code in the WikiDB project environment:import requests
response = requests.get("https://mtab.app/api/v1/search", params={"q": "Belgium", "limit": 5, "m": "a", "lang": "en", "info": 1})
print(response.json())
Alternatively, use the KeywordSearch class:from keyword_search import KeywordSearch
ks = KeywordSearch()
results, runtime = ks.get("Belgium", limit=5, mode="a", lang="en", info=1)
print(results)
Observe the error:requests.exceptions.ConnectionError: HTTPSConnectionPool(host='mtab.app', port=443): Max retries exceeded with url: /api/v1/search?q=Belgium&limit=5&m=a&lang=en&info=1 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Verify DNS failure with command-line tools:nslookup mtab.app 8.8.8.8
Output: server can't find mtab.app: NXDOMAIN
dig mtab.app @8.8.8.8
Output: status: NXDOMAIN
Expected Behavior
The KeywordSearch class should successfully query the mtab.app API and return a list of Wikidata QIDs, scores, labels, and descriptions for the given keyword (e.g., ["Q31", 1.0, "Belgium", "country in Western Europe"]).
Actual Behavior
The API request fails due to DNS resolution failure (mtab.app is not resolvable), rendering the keyword search functionality inoperable.
Environment
OS: Ubuntu 20.04
Python: 3.6 (in a conda environment)
WikiDB Commit: [Specify commit hash if known, e.g., latest from master as of August 2025]
Network: Tested with and without a proxy (http://127.0.0.1:10808); issue persists in both cases
Date Tested: August 19, 2025
Additional Context
The mTab platform’s main website (https://mtab.com) is accessible, suggesting the API may have moved to a new endpoint (e.g., https://api.mtab.com) or been decommissioned.
The WikiDB project’s README.md and code reference mtab.app as of 2022, but the domain is no longer resolvable as of August 2025.
Tested with Google’s DNS (8.8.8.8), confirming the NXDOMAIN error is not due to local network issues or regional restrictions.
Suggested Fixes
Update API Endpoint: If mTab has moved the API to a new domain (e.g., api.mtab.com), update KeywordSearch.URL in keyword_search.py.
Use Alternative API: Replace mtab.app with Wikidata’s wbsearchentities API (https://www.wikidata.org/w/api.php?action=wbsearchentities) for keyword-to-QID mapping. Example implementation:import requests
def search_wikidata(query, limit=5, lang="en"):
response = requests.get(
"https://www.wikidata.org/w/api.php",
params={"action": "wbsearchentities", "search": query, "language": lang, "format": "json", "limit": limit}
)
results = response.json().get("search", [])
return [[r["id"], 1.0, r.get("label", ""), r.get("description", "")] for r in results], 0
Local Index: Build a local full-text index (e.g., using Whoosh) for labels/descriptions in wikidb.lmdb to enable offline keyword search, though this requires significant storage.
Request
Could you confirm if the mtab.app API is still active or if it has been replaced? If the endpoint has changed, please provide the new URL or documentation. Alternatively, consider integrating a fallback API (e.g., Wikidata’s wbsearchentities) to ensure the keyword search functionality remains robust.
Thank you for maintaining this project! Let me know if you need more details or assistance testing potential fixes.