Known biases and blind spots
What can SIREN not see?
An index is only as wide as its inputs. Ours are public, free and recomputable, and that choice comes with a tilt: almost everything SIREN counts is published in English, and most of it in the United States. This page says where that bends the reading, and what we are looking at to straighten it.
Where does each input lean?
| Input | Pillar | Lean |
|---|---|---|
| arXiv | Capability | Preprints are overwhelmingly in English, and the habit of posting to arXiv first is strongest in US and European labs. Work that appears first in Chinese-language venues or company blogs is undercounted. |
| GitHub releases, Hugging Face, OpenRouter | Capability (uncalibrated) | Western-hosted platforms. Chinese labs often ship first on ModelScope or domestic mirrors, so a busy week in Beijing can look quiet here. |
| SEC full-text search, US equities, Vast.ai prices | Compute & Capital | US filings and US-listed companies, priced in dollars. Capital spent by state-backed or unlisted companies outside the US barely registers. |
| Hacker News | Attention | An English-language forum with a mostly US and European tech audience. Its idea of "big news" is a Silicon Valley one. |
| English Wikipedia pageviews | Attention | Only the English edition. Interest that shows up as Japanese, German or Spanish readers is invisible. |
| US Federal Register, GOV.UK | Governance | Two English-speaking jurisdictions. EU implementing acts, Chinese administrative rules and other national regulators do not move the pillar. |
| Polymarket, Kalshi, Manifold | Markets (uncalibrated) | English-language questions written mostly by and for US traders. What they choose to list is itself a filter. |
Most news sources behind Which AI stories has SIREN scored in this window? are English-language as well. That newsroom does not feed the score, but it shapes which stories sit next to it.
What does that skew?
- Timing. A launch, rule or funding round outside the English-speaking web reaches our inputs late, when English coverage catches up, or never. The score can lag real events by days.
- Size. Activity is measured against each source's own history, so a source that systematically misses one region also has a quieter baseline. The bias is consistent rather than random, which keeps hour-to-hour moves meaningful but means the level reads "how busy is the English-speaking AI world".
- Governance. With two jurisdictions, the governance pillar reads the US and UK regulatory calendar, not the world's.
- Attention. A story that is huge in Seoul or São Paulo and ignored on Hacker News does not exist as far as this pillar knows.
Which sources are under consideration?
Each candidate has to meet the same bar as today's inputs: public, free to read without a private agreement, machine-readable, with enough history to build a reference year, and licensed so the counts can be republished with receipts. Anyone has to be able to recompute the number.
Would other-language Wikipedia pageviews help?
Attention
Same Wikimedia pageviews API the English series already uses, pointed at the German, French, Spanish, Japanese, Chinese and other editions. The cheapest fix on this list: same method, same receipts.
Would EUR-Lex widen governance?
Governance
The EU's official journal and legal database. Counting AI-related acts, implementing decisions and consultations would add the jurisdiction with the most active AI rulebook.
Would China MIIT and CAC notices help?
Governance
Public notices from the Ministry of Industry and Information Technology and the Cyberspace Administration of China, including published batches of registered generative AI services.
Would Japan METI publications help?
Governance
The Ministry of Economy, Trade and Industry's AI guidelines, study groups and policy publications.
Would India MeitY notices help?
Governance
The Ministry of Electronics and Information Technology's advisories, IndiaAI programme notices and consultations.
A new source never goes straight into the score. It is collected and published as an uncalibrated signal first, then run as a shadow score beside the official one, and promoted only under the published rule on How does the SIREN baseline get updated?.
How do I propose a source?
Know a public, non-English or non-Western dataset that counts AI activity? Send it through the How do I report a problem or send an idea to SIREN? form with a link to its API or download, how far back it goes, and its licence. The more of those three you can answer, the faster it can be tested.
What questions do people actually ask?
What can SIREN not see?
An index is only as wide as its inputs. Ours are public, free and recomputable, and that choice comes with a tilt: almost everything SIREN counts is published in English, and most of it in the United States.
Where does each input lean?
Almost everything counted is published in English, and most of it in the United States. The table below names the lean of each input.
What does that skew?
Timing (events outside the English-speaking web reach inputs late or never), size (the level reads how busy the English-speaking AI world is), governance (US and UK calendar, not the world's), and attention (stories ignored on Hacker News do not exist to that pillar).
Which sources are under consideration?
Each candidate has to be public, free to read without a private agreement, machine-readable, with enough history to build a reference year, and licensed so the counts can be republished with receipts.
How do I propose a source?
Know a public, non-English or non-Western dataset that counts AI activity? Send it through the feedback form with a link to its API or download, how far back it goes, and its licence.
What does that skew?
Timing (events outside the English-speaking web reach inputs late or never), size (the level reads how busy the English-speaking AI world is), governance (US and UK calendar, not the world's), and attention (stories ignored on Hacker News do not exist to that pillar).
How do new sources get into the score?
A new source never goes straight into the score. It is collected and published as an uncalibrated signal first, then run as a shadow score beside the official one, and promoted only under the published rule on how the baseline gets updated.
Is the biased reading a doom probability?
No. SIREN measures activity tempo. The level reads how busy the English-speaking AI world is — not danger, and not a probability of harm.
Would other-language Wikipedia pageviews help?
Same Wikimedia pageviews API the English series already uses, pointed at the German, French, Spanish, Japanese, Chinese and other editions. The cheapest fix on this list: same method, same receipts.
Would EUR-Lex widen governance?
The EU's official journal and legal database. Counting AI-related acts, implementing decisions and consultations would add the jurisdiction with the most active AI rulebook.
Would China MIIT and CAC notices help?
Public notices from the Ministry of Industry and Information Technology and the Cyberspace Administration of China, including published batches of registered generative AI services.
Would Japan METI publications help?
The Ministry of Economy, Trade and Industry's AI guidelines, study groups and policy publications.
Would India MeitY notices help?
The Ministry of Electronics and Information Technology's advisories, IndiaAI programme notices and consultations.
Search-shaped sibling: How the SIREN AI activity index is computed · methodology · How do I report a problem or send an idea to SIREN?.
How the SIREN AI activity index is computed · methodology · How do I report a problem or send an idea to SIREN?