Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Guys & Gals anyone know if urllist.txt is still used?
-
I'm using a tool which generates urllist.txt and looking on the SEO Forums it seems that Yahoo used to use this. What I'd like to know is is it still used anywhere and should we have it on the site?
-
Thanks for the advice, we already create and submit the XML sitemap to Google, that wasn't the question. Would there be any benefit in creating the urllist.txt file?
-
I would just use a sitemap.xml file instead for Google, Bing and Yahoo. Then you can submit the sitemap.xml file within the Google Webmaster Tools and Bing Webmaster Tools (includes Yahoo). You can easily create an XML sitemap at http://www.xml-sitemaps.com/
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Same H1 & H2 Tags
Is it bad to have the same H1 & H2 tag on one page? I found a similar question here on the moz forum but it didn't exactly answer my question. And will adding "about" on the H2 help, or should we avoid duplicate tags completely? Here is a link to the page in question (which will repeat throughout this site.) Thanks in advance!
Technical SEO | | Mike.Bean0 -
Should I block Map pages with robots.txt?
Hello, I have a website that was started in 1999. On the website I have map pages for each of the offices listed on my site, for which there are about 120. Each of the 120 maps is in a whole separate html page. There is no content in the page other than the map. I know all of the offices love having the map pages so I don't want to remove the pages. So, my question is would these pages with no real content be hurting the rankings of the other pages on our site? Therefore, should I block the pages with my robots.txt? Would I also have to remove these pages (in webmaster tools?) from Google for blocking by robots.txt to really work? I appreciate your feedback, thanks!
Technical SEO | | imaginex0 -
Are robots.txt wildcards still valid? If so, what is the proper syntax for setting this up?
I've got several URL's that I need to disallow in my robots.txt file. For example, I've got several documents that I don't want indexed and filters that are getting flagged as duplicate content. Rather than typing in thousands of URL's I was hoping that wildcards were still valid.
Technical SEO | | mkhGT0 -
Invisible robots.txt?
So here's a weird one... Client comes to me for some simple changes, turns out there are some major issues with the site, one of which is that none of the correct content pages are showing up in Google, just ancillary (outdated) ones. Looks like an issue because even the main homepage isn't showing up with a "site:domain.com" So, I add to Webmaster Tools and, after an hour or so, I get the red bar of doom, "robots.txt is blocking important pages." I check it out in Webmasters and, sure enough, it's a "User agent: * Disallow /" ACK! But wait... there's no robots.txt to be found on the server. I can go to domain.com/robots.txt and see it but nothing via FTP. I upload a new one and, thankfully, that is now showing but I've never seen that before. Question is: can a robots.txt file be stored in a way that can't be seen? Thanks!
Technical SEO | | joshcanhelp0 -
Use of + in url good or bad?
Hi, I am working on a SEO project for a client.
Technical SEO | | MaartenvandenBos
Some of the urls have a + between the keyword.
like www.example.com/make+me+happy/ Is this good or bad for seo?
Or is it maybe better to use - ? Thanks!0 -
Iframes & SEO
I've got a client that wants a site with all content in iFrames. They saw another site they liked & asked if we could do it. Of course we can technically. How big a negative hit would they take with SEO? Is there anything we can do to mitigate it, such as redirects, etc? Thanks for the help!
Technical SEO | | wcksmith0 -
301 Redirect & Cloaking
HEllo~~~~ People. I have a question regarding on cloaking. I will be really greatful if you can help me with question. I have a site www.example.com and it is targeting for multi countries. So I use sub directories for targeting multi countries. e.g. www.example.com/us/ www.example.com/de/ www.example.com/hk/ ....... so on and on. Therefore, when people type www.example.com, I use IP delivery to send users to each coutries. Here is my question. I use 301 redirect for IP delivery, which means when user enter www.example.com, my site read user's IP and send them to right country site by 301 redirect. In this case, is there any possibility that Google considers it as cloaking? Please people.... share me some ideas and thoughs.
Technical SEO | | Artience0 -
Has google panelized us ? If so, why ? How do I know if our website is panelized ?
We were ranked on first page among top 5 position a year ago for most of our pages. On one fine day, google decided to drop us from the results although google keeps indexing our pages. Google index our pages regularly but doesn't show them in its results. All google traffic we receive is for our own site name and its variations. I wanted to know - how do we know if google has panelized us. Why has google panelized us ? If they have panelized us, what can we do to get out of it ? Also I wanted to know if any tool will help me identify such thing. We have not done any link building. Our site page rank is 4 (it was 5 few months ago). All we did was on page optimization. Thanks for your help!
Technical SEO | | seoidea0