{"version":"1.0","provider_name":"Pingler Blog","provider_url":"https:\/\/pingler.com\/blog","title":"Accommodating Bots, Crawlers and Spiders On Your Website - Pingler Blog","type":"rich","width":600,"height":338,"html":"<blockquote class=\"wp-embedded-content\" data-secret=\"aZpR2uv8Xm\"><a href=\"https:\/\/pingler.com\/blog\/accommodating-bots-crawlers-and-spiders-on-your-website\/\">Accommodating Bots, Crawlers and Spiders On Your Website<\/a><\/blockquote><iframe sandbox=\"allow-scripts\" security=\"restricted\" src=\"https:\/\/pingler.com\/blog\/accommodating-bots-crawlers-and-spiders-on-your-website\/embed\/#?secret=aZpR2uv8Xm\" width=\"600\" height=\"338\" title=\"&#8220;Accommodating Bots, Crawlers and Spiders On Your Website&#8221; &#8212; Pingler Blog\" data-secret=\"aZpR2uv8Xm\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" class=\"wp-embedded-content\"><\/iframe><script type=\"text\/javascript\">\n\/* <![CDATA[ *\/\n\/*! This file is auto-generated *\/\n!function(d,l){\"use strict\";l.querySelector&&d.addEventListener&&\"undefined\"!=typeof URL&&(d.wp=d.wp||{},d.wp.receiveEmbedMessage||(d.wp.receiveEmbedMessage=function(e){var t=e.data;if((t||t.secret||t.message||t.value)&&!\/[^a-zA-Z0-9]\/.test(t.secret)){for(var s,r,n,a=l.querySelectorAll('iframe[data-secret=\"'+t.secret+'\"]'),o=l.querySelectorAll('blockquote[data-secret=\"'+t.secret+'\"]'),c=new RegExp(\"^https?:$\",\"i\"),i=0;i<o.length;i++)o[i].style.display=\"none\";for(i=0;i<a.length;i++)s=a[i],e.source===s.contentWindow&&(s.removeAttribute(\"style\"),\"height\"===t.message?(1e3<(r=parseInt(t.value,10))?r=1e3:~~r<200&&(r=200),s.height=r):\"link\"===t.message&&(r=new URL(s.getAttribute(\"src\")),n=new URL(t.value),c.test(n.protocol))&&n.host===r.host&&l.activeElement===s&&(d.top.location.href=t.value))}},d.addEventListener(\"message\",d.wp.receiveEmbedMessage,!1),l.addEventListener(\"DOMContentLoaded\",function(){for(var e,t,s=l.querySelectorAll(\"iframe.wp-embedded-content\"),r=0;r<s.length;r++)(t=(e=s[r]).getAttribute(\"data-secret\"))||(t=Math.random().toString(36).substring(2,12),e.src+=\"#?secret=\"+t,e.setAttribute(\"data-secret\",t)),e.contentWindow.postMessage({message:\"ready\",secret:t},\"*\")},!1)))}(window,document);\n\/\/# sourceURL=https:\/\/pingler.com\/blog\/wp-includes\/js\/wp-embed.min.js\n\/* ]]> *\/\n<\/script>\n","thumbnail_url":"https:\/\/pingler.com\/blog\/wp-content\/uploads\/2012\/05\/8710955_s.jpg","thumbnail_width":450,"thumbnail_height":338,"description":"Every single day, there are countless bots, spiders and crawlers perusing the internet on behalf of major search engines that are collecting information about each site and page they come across. \u00a0Your site \u2013 especially if it has many backlinks \u2013 may be indexed several times per day by bots that are looking for any [&hellip;]"}