.Google's Gary Illyes discloses Googlebot gathers links rather than "following" all of them, opposing formal records.Googlebot gathers web links rather than observing all of them in real-time.Gary Illyes discussed this details in Google.com's 'Look Off The Document' podcast.Google.com's records contradicts the true crawler habits.