Chrome extension webscraper.io - how does pagination work with selecting "next"
Following the advanced documentation here, the problem is solved by making the "pagination" link a parent of its own. Then, the scraping software will recursively go through all pages and their "next" page. In their words,
To extract items from all of the pagination links including the ones that are not visible at the beginning you need to create another Link selector that selects the pagination links. Figure 2 shows how the link selector should be created in the sitemap. When the scraper opens a category link it will extract items that are available in the page. After that it will find the pagination links and also visit those. If the pagination link selector is made a child to itself it will recursively discover all pagination pages.