FetchError: ETIMEDOUT

[YDX-2147483647]

从 2023-10-02 10时（UTC+8）开始，持续访问超时，出问题的通知来源不定。至今（2023-10-09）仍未解决。

file:///…/bulletin-issues-transferred/node_modules/node-fetch/src/index.js:108
                       reject(new FetchError(`request to ${request.url} failed, reason: ${error.message}`, 'system', error)>
                              ^
FetchError: request to https://ice.bit.edu.cn/xxfw/tzgg1/index.htm failed, reason: connect ETIMEDOUT 114.247.56.92:443

在校外本地运行常常也如此，获取十几个通知来源后就超时。在校内本地运行正常。

可能的解决方案

• 改用 sindresorhus/got: 🌐 Human-friendly and powerful HTTP request library for Node.js，它支持指定超时时长、自动重试等。
• 从并行 Promise.all() 改为串行，或者 sindresorhus/p-all: Run promise-returning & async functions concurrently with optional limited concurrency 。
• 程序从 crontab 反复运行改为长期运行，不再同时获取通知。

[YDX-2147483647]

现在 cfc10ad 采用鸵鸟策略。

反正每次超时的来源随机，而每天会获取多次，一天下来有不小的概率完全获取到。

[YDX-2147483647]

现在好像又不超时了。

$ jq 'select(.level == "error").timestamp' output/combined.log --raw-output | sed -E 's/\:[0-9]{2}\.[0-9]{3}Z$//' | uniq | tail
2023-10-26T05:17
2023-10-26T06:17
2023-10-26T07:17
2023-10-26T08:17
2023-10-26T09:17
2023-10-26T10:17
2023-10-26T11:17
2023-10-26T12:17
2023-10-26T13:17
2023-10-27T01:17

$ jq 'select(.level == "error").timestamp' output/combined.log --raw-output | sed -E 's/T.+Z$//' | uniq | tail
2023-10-18
2023-10-19
2023-10-20
2023-10-21
2023-10-22
2023-10-23
2023-10-24
2023-10-25
2023-10-26
2023-10-27

（这些是 UTC 时间）

[YDX-2147483647]

又开始超时了。

2023-10-31T07:17
2023-10-31T08:17
2023-10-31T09:17
2023-10-31T10:17
2023-10-31T11:17
2023-10-31T12:17
2023-10-31T13:17
2023-11-01T01:17
2023-11-01T02:17
2023-11-01T03:17

2023-10-21
2023-10-22
2023-10-23
2023-10-24
2023-10-25
2023-10-26
2023-10-27
2023-10-30
2023-10-31
2023-11-01

[YDX-2147483647]

如果网络环境不好（如连接流量很小的路由器），同时发出单纯大量请求就会卡住，且不抛出超时错误。

import ky, { TimeoutError } from 'ky'

const raw_urls: string[] = JSON.parse(
    await Deno.readTextFile('config/sources_by_selectors.json'),
).sources.map(({ url }: { url: string }) => url)

const urls: string[] = []
for (let m = 0; m < 10; m++) {
    urls.push(...raw_urls)
}

function Timer() {
    const start = Date.now()

    return (): number => {
        return (Date.now() - start) / 1e3
    }
}

for (let round = 0; round < 5; round++) {
    let count = 1
    const t = Timer()

    await Promise.all(
        urls.map(async (u) => {
            try {
                const text = await ky(u).text()
                console.log(
                    `%c${count} ${t().toFixed(3)}s`,
                    'color: green',
                    u,
                    text.length,
                )
            } catch (error) {
                if (error instanceof TimeoutError) {
                    console.warn(
                        `%c${count} Timeout ${t().toFixed(3)}s`,
                        'color: purple',
                    )
                    console.log(u)
                } else {
                    console.error(
                        `%c${count} Error ${t().toFixed(3)}s`,
                        'color: red',
                    )
                    console.log(u)
                    console.error(error)
                }
            }
            count += 1
        }),
    )

    console.log('\n\n\n')
}

Task test deno run --allow-net --allow-read src/examples/test.ts
1 7.134s http://cxcy.bit.edu.cn/pt/pt/Notice 24511
2 7.519s https://www.bit.edu.cn/tzgg17/zxtz/index.htm 12008
3 8.208s https://bit.edu.cn/tzgg17/zcgg/index.htm 12498
4 8.915s https://www.bit.edu.cn/tzgg17/zxtz/index.htm 12008
5 9.813s https://sgg.bit.edu.cn/tzgg/index.htm 14145
6 Timeout 10.113s
https://www.bit.edu.cn/rcpy_sjb/blsy87/qssygb/tzggsy3/index.htm
7 Timeout 10.115s
https://www.bit.edu.cn/rcpy_sjb/blsy87/jgsygb/tzggsy/index.htm
8 Timeout 10.116s
https://mingde.bit.edu.cn/tzgg/index.htm
9 Timeout 10.117s
https://design.bit.edu.cn/zs/tz2/index.htm
10 Timeout 10.118s
https://bj.bit.edu.cn/tzgg/index.htm
11 Timeout 10.120s
https://isc.bit.edu.cn/eventsnotices/index.htm
…
…
313 Timeout 10.343s
https://mse.bit.edu.cn/tzgg/index.htm
314 Timeout 10.344s
https://sfl.bit.edu.cn/tzgg/index.htm
315 Timeout 10.344s
https://cce.bit.edu.cn/tzgg/index.htm
316 Timeout 10.345s
https://bit.edu.cn/tzgg17/zcgg/index.htm
317 Timeout 10.346s
https://zsc.bit.edu.cn/xwgg/tzgg/index.htm
318 10.556s https://xyy.bit.edu.cn/tzgg/index.htm 8697
319 10.724s https://mingde.bit.edu.cn/tzgg/index.htm 19199
320 10.809s http://cxcy.bit.edu.cn/pt/pt/Notice 24511
321 12.202s https://jxzx.bit.edu.cn/tzgg/index.htm 12681
322 13.115s http://cxcy.bit.edu.cn/pt/pt/Notice 24511
323 13.203s https://cs.bit.edu.cn/tzgg/index.htm 16692
324 13.639s http://cxcy.bit.edu.cn/pt/pt/Notice 24511
325 13.917s https://math.bit.edu.cn/tzgg/index.htm 24899
326 13.942s http://cxcy.bit.edu.cn/pt/pt/Notice 24511
327 13.963s https://rw.bit.edu.cn/rwszjyb/tzxw/index.htm 17547
328 13.995s https://century.bit.edu.cn/tz/index.htm 21224
…
…
462 21.260s https://hi.bit.edu.cn/rxbd/dttzyx/index.htm 26181
463 21.734s https://xuteli.bit.edu.cn/tzgg/index.htm 20824
464 22.224s https://rw.bit.edu.cn/tzgg/index.htm 24905
465 23.831s https://sme.bit.edu.cn/gbxwzx/tzgg/index.htm 32201
466 25.461s https://xxgk.bit.edu.cn/tzgg/index.htm 30457
467 Error 38.406s
http://cxcy.bit.edu.cn/pt/pt/Notice
TypeError: request or response body error: error reading a body from connection: 远程主机强迫关闭了一个现有的连接。 (os error 10054)
    at async Object.pull (ext:deno_web/06_streams.js:936:25)

（本来能到 490，但卡住了）

如果m上限小一些，即使round上限很大，也不容易卡住。