metrics: Achievements metrics shows (Unexpected Error)

Yes, you can view the complete changes in https://github.com/TonyCrane/metrics/commit/46f327283ad2221c4d47963a3acd1096a4316d72.

But seems this repo haven’t updated for a few months, and my last pull request is ignored and closed by robot, so I haven’t open a PR for this (also I don’t think my solution is the best).

But I think we should repair this? So what do you think @lowlighter ?

@TonyCrane I tested your patch locally and it works as expected. I think the workaround is good enough for now so I merged it into master 👍

I guess now we could use the Search API directly by passing your query into it and be more reliable and faster, instead of using puppeteer as it could break again in a future redesign

GitHub have added a home page to GitHub Search, so metrics can’t get total repository/user/issue number from https://github.com/search.

And the query string can’t be empty, so maybe we can try searching created:>2007, although this may be not exact, but at least it works.

      for (let i = 0; (i < 100) && ((!total.users) || (!total.repositories)); i++) {
        const page = await browser.newPage()
        await page.goto("https://github.com/search?q=+created%3A%3E2007")
        const totalres = await page.evaluate(() => [...[...document.querySelectorAll("h2")].filter(node => /Filter by/.test(node.innerText)).shift()?.nextSibling?.innerText.trim().matchAll(/(?<type>Repositories|Users|Issues)\n(?<count>.*?)M/g) ?? []]) ?? null
        for (const result of totalres) {
          const type = result[1]?.toLowerCase()
          console.debug(`metrics/compute/plugins > achievements > setup found ${type ?? "(?)"}`)
          const count = result[2] ?? ""
          if ((count !== "") && (!total[type])) {
            total[type] = Number(count) * 10e5
            console.debug(`metrics/compute/plugins > achievements > set total.${type} to ${total[type]}`)
          }
        }
        await page.close()
        await imports.wait(10 * Math.random())
      }

After this pr is merged, there is no new release, so @latest is still an old version. If you want to use the repaired version, you need to use @master.

GitHub have added a home page to GitHub Search, so metrics can’t get total repository/user/issue number from https://github.com/search.

And the query string can’t be empty, so maybe we can try searching created:>2007, although this may be not exact, but at least it works.

      for (let i = 0; (i < 100) && ((!total.users) || (!total.repositories)); i++) {
        const page = await browser.newPage()
        await page.goto("https://github.com/search?q=+created%3A%3E2007")
        const totalres = await page.evaluate(() => [...[...document.querySelectorAll("h2")].filter(node => /Filter by/.test(node.innerText)).shift()?.nextSibling?.innerText.trim().matchAll(/(?<type>Repositories|Users|Issues)\n(?<count>.*?)M/g) ?? []]) ?? null
        for (const result of totalres) {
          const type = result[1]?.toLowerCase()
          console.debug(`metrics/compute/plugins > achievements > setup found ${type ?? "(?)"}`)
          const count = result[2] ?? ""
          if ((count !== "") && (!total[type])) {
            total[type] = Number(count) * 10e5
            console.debug(`metrics/compute/plugins > achievements > set total.${type} to ${total[type]}`)
          }
        }
        await page.close()
        await imports.wait(10 * Math.random())
      }

I have tested and implemented your code, and it works as expected. Thank you for your contribution!

Your question has been answered already, but you have to read the full contents of the issue. I will copy the direct link to make it easier for you to read.

#1479 (comment)

Thanks. But this means the problem is not on my side but with metrics? If yes, thank you for the fix in advance.

GitHub have added a home page to GitHub Search, so metrics can’t get total repository/user/issue number from https://github.com/search. And the query string can’t be empty, so maybe we can try searching created:>2007, although this may be not exact, but at least it works.

      for (let i = 0; (i < 100) && ((!total.users) || (!total.repositories)); i++) {
        const page = await browser.newPage()
        await page.goto("https://github.com/search?q=+created%3A%3E2007")
        const totalres = await page.evaluate(() => [...[...document.querySelectorAll("h2")].filter(node => /Filter by/.test(node.innerText)).shift()?.nextSibling?.innerText.trim().matchAll(/(?<type>Repositories|Users|Issues)\n(?<count>.*?)M/g) ?? []]) ?? null
        for (const result of totalres) {
          const type = result[1]?.toLowerCase()
          console.debug(`metrics/compute/plugins > achievements > setup found ${type ?? "(?)"}`)
          const count = result[2] ?? ""
          if ((count !== "") && (!total[type])) {
            total[type] = Number(count) * 10e5
            console.debug(`metrics/compute/plugins > achievements > set total.${type} to ${total[type]}`)
          }
        }
        await page.close()
        await imports.wait(10 * Math.random())
      }

can you explain please what I have to do to make it work?

@X-itachi-X probably says to edit the http here index.mjs : line 85