Hey guys, first-time poster – and I have a head scratcher for you
I’m indexing Shopify Polaris docs. Cursor is indexing pages that no longer exist, and not including some recent additions. It’s almost as if it’s using a cached version. Shopify Polaris v12 was released early October, but the indexed docs all refer to Polaris v11 (and Polaris v12 beta).
Some of the scraped/indexed docs no longer exist in the sitemap, or on the website (I looked through the HTML). Yet they appear as pages indexed by Cursor. And some new sections/pages added since the release of v12 isn’t among the indexed docs.
If I ask the docs “what’s the latest version of Polaris?” I get the answer:
The latest version of Polaris is version 12, which is currently in beta and scheduled for a stable release at the end of September.
Similarly, if I ask “What stack component exists?” I get:
Both 1 & 2 above were replaced in Polaris 12, and are no longer in the sitemap or on the website. The list of indexed docs include links to components/layout-and-structure/horizontal-stack and …/vertical-stack, which now are 308 redirects to inline-stack and block-stack.
Where does the scraper/indexer find these? Can I do something to work around this?
Steps to reproduce:
- @add Components — Shopify Polaris /
- Name your docs, use URLs: same as above + without /components/
Try to find e.g. inline-stack or block-stack components, or anything about
VSCode Version: 1.83.1
OS: Darwin arm64 22.4.0