Practice
Learn
← Quit
0 / 0 correct
System Design
Medium
1 of 174
Web crawler: why cache DNS results inside the crawler?
A
DNS lookups add latency to every fetch and can bottleneck throughput at billions of pages
B
It is required by robots.txt
C
To avoid HTTPS
D
DNS results never change