How to resolve your problems on robot.txt sitemap broken links

Reliable Web Hosting in Nigeria by DomainKing.NG
this how you will know how to set sitemap, robot.txt, and to resolve crawling issue

This are ways to resolve problems on optimization

GOOD Evening this the long awaited tutorial I would be teaching on SEO programme my name is Elvis Chris first of all I would explain the term seo,


(1) SEO stands for Search engine optimization, I know you have been on search of strategy, techniques and tactics in which to use to increase the amount of visitors to your website, seo is a programme which enables your blog to be effective, visible in search result however there are many tools of strategy used in allowing crawl to your content on robot.txt the no1 is another is


(2) Most bloggers ask about this robot.txt the name is called robot so its a robot not a human its specifically programmed to interact with your blog content to tell your sitemap which you submitted on where to crawl just understand that any broken, duplicate, malicious cant be crawled to your sitemap through robot.txt
This robot can make your site not to function again when mis-used example when you pirate a foreign music and Dcma banned your blog you would notice that all your search result would be lost due to google has disabled your robot.txt at there end this usually occurs when you violate google rule.

I tell you there are three important things that you should know when it comes to the robots.txt file.
Check/run a check if you have a robots.txt file
If you have any,

make sure it is not harming your ranking or blocking content you don’t want blocked
Determine if you need a robots.txt file hope you understand

example to tell you how it works with your site is when you have a new WordPress website without any seo plugin like yoast or All in one seo plugin  to generate your robot.txt In its database then no matter how you submit sitemap there must be an error that your sitemap cant be verified in google console before you verify your sitemap in google for WordPress you must have an seo plugin to enable it to create your sitemap on default then you can now submit to google

NOTE: Remember knowing that robot.txt Is the spider / crawler does not mean you should edit to cheat this can specifically destroy all your search results you cant trick it because google algorithms is like a mainframe to all blogs .

You must have heard about this term sitemap.


(3)  Sitemap is the map engine that contain information of all your located resource on your blog this you must know every website or blog have a sitemap.

The real truth is that without sitemap google wont crawl to index your site 100% not that the wont crawl but when you have 100 post only 5 will be indexed due to you never hosted your content with them so I would tell you all to create a sitemap in google, bing, yandex sitemap helps users, as well as search engines, to easily explore a website.
although you must study to know how your robot.txt is specifically working on daily basis in your search console its important to know that, because it alerts you of error which cant be crawled in search engine example broken links and duplicate content cant be crawled to search engine in process of robot.txt,
in this lesson I would like to know if you understood the meaning of seo, sitemap, robot.txt


(4) the xml extension will enable google spider to crawl you as you post, This format is not understandable by humans but search engines do understand what is written in it and the purpose of writing in it.

XML stands for extensible markup language sitemap on xml extension use it basically to know what your. Contents are all about.

hope you all understands me ?

.Xml sitemap stores information about an article, pages and images with alternate tag remember any image without alternate definition cant be crawled

Because pictures are void the are not text google cant study them so we use alternate tags to define what a particular picture is all about

Look at the picture below will show you how spider interact with images I would like you all to include alt definition to all your images

.html extension is a map that contains information about your website resources and their location.

Html sitemap

Html stand for hypertext markup language .html sitemaps are primarily for users to assist them to find their desired item on your blog also to explore your blog easily.
Hope you understand me?


(6) I have been receiving report of people asking me about there post not crawled in this lesson I would explain to you while your post wasn’t crawled,

Your post wasn’t crawled because it was malicious, broken, 404 page, example the primary part are duplicate links this happens on WordPress mostly example you created a post in WordPress and published maybe the title was “ who wants to be a millionaire

then you published it later run you thought of changing the title you went ahead and changed the title once you change the title and update your article goto the post again on the picture below

how broken links occur in wordpress permalinks

you would see that your title box is on the new title which you changed while the second box known as permalink will be with the old title.

Most articles you might like to read

which you changed you can see that in that post is completely duplicate the title does not match with the permalink
I would advice you when changing any post title also change your permalink title on second box with same topic name.

I will advice you all to download seo frog tool to know about your duplicate and broken links my name is elvis chris i teach bloggers and webmaster am a programmer a standcode official feel free asking us any question we will answer you without hesitation thanks.

About the Author

Elvis Chris
Elvis Chris
My name is Elvis Chris am a Nigerian a certified programmer in html, css, JavaScript, php, jQuery a standcode official I teach bloggers and webmasters I build blogs on monetization for people I teach on search engine optimization.
Question posted in standcode
are answered in a moment

Be the first to comment on "How to resolve your problems on robot.txt sitemap broken links"

Leave a comment

Your email address will not be published.