Skip to main content

What to do with 6 MIL + pages

I’m in-house SEO working for a really old company that’s been around for decades and has lots of different facets to it.

There’s a real legacy issue with the website. Hundreds of people have had access and autonomy over the site, and there’s so much crap. They’ve also used the site as an intranet and - though they’re pretty hard to find - there’s notes from meetings, HR docs and so much more live on the site.

I’m running a crawl now, after I noticed I’ve got www pages linking to non www pages. So I need to get everything on the same domain. I can’t do that until I know the extent of the issue, historically I’ve not been able to crawl the full site just because of time restraints.

So I’ve always crawled specific subfolders and tackled deadweight in stages. Now I want to bite the bullet and do a full crawl because I want the full picture. But it’s onto 6 million pages now (and counting) excl. images obviously.

When this is done, how do I even go about exporting this? Surely excel and google sheets can’t handle that much data? Any advice around this would be amazing.

Thank you!

PS using Screaming Frog

submitted by /u/Sick_Turtle
[link] [comments]

from Search Engine Optimization: The Latest SEO News https://ift.tt/3jCi7ce
Labels:

Comments

Post a Comment

Popular posts from this blog

Legit Reviews

The google review thing is insane I have gotten 2 legit verified reviews that i ask client to screenshot and observe goolge not post them, pretend like they never happened and when i sent them the image a policy message was sent . As I was browsing through the google forum a lot of ppl are getting hit with legit reviews being removed. While all this is going on i have observe a company go from 12 reviews to 65 reviews in a two month period with some stuff that dont make sense on some of them. Seems like a new business or profile is being limited while aged accounts can have a free for fall but who really knows. Still waiting for a resolution on a if any. submitted by /u/Ok-Bowl-6167 [link] [comments] from Search Engine Optimization: The Latest SEO News https://ift.tt/zkbfgDH
Post a Comment
Read more

Local seo vs. natiowide seo?

I've done SEO for local businesses but I recently got my first client that sells an item nation wide. ​ Any suggestions for doing nationwide SEO? ​ I am used to making geopages for local towns. I was going to do the same with some input from the client about what cities or towns he would like to show up in? submitted by /u/Letmeinterviewyou [link] [comments] from Search Engine Optimization: The Latest SEO News http://bit.ly/2JHy0k0
Post a Comment
Read more

7 SEO Lessons I've Learned in 2025 (So Far) - From Building a Competitor Analysis Tool

Hey r/SEO , I've been spending a lot of time lately building a competitor keyword research tool (more on that below!), and it's really forced me to dig deep into how SEO works today . Here are some of the key lessons I've learned – hopefully, they're helpful to you too: Keyword Gaps Are Gold: It's not enough to just know what keywords your competitors rank for. You need to know what they rank for that you don't . This is where the real opportunities lie. Focusing on these "gap" keywords has been a game-changer for my own site. (This is actually why I built the tool – to make finding these gaps easier). Relevance > Volume (Sometimes): High search volume is great, but relevance is even more important. A keyword with 100 searches per month that's perfectly aligned with your niche and audience is often more valuable than a keyword with 10,000 searches that's only tangentially related. I've seen much better results targeting tho...
Post a Comment
Read more