I'm providing SEO advice to a company that does web development for a large news agency. They publish around 700 articles daily and have more than 10m URLs in total. Their website has thousands, maybe even hundreads of thousands up to a million of topic URLs that have only IDs and are non indexable. They serve like a topical page, but dont have anything besides the list of URLs towards articles. I aim to help them improve their crawl budget and I'm confused whether disallowing these URLs will be helpful, or could it prevent some pages from being crawled and discovered. Furthermore, the website has authors pages thag provide basically no value. These pages are non indexable, dont havs bios, images, or anything. I told them to disallow them but Im not sure whether this was the right move. Any advice? submitted by /u/DukeVeljko [link] [comments] from Search Engine Optimization: The Latest SEO News https://ift.tt/CJ29nWS