Practice
Learn
← Quit
0 / 0 correct
System Design
Medium
1 of 10
Web crawler: why cache DNS results inside the crawler?
A
DNS results never change
B
To avoid HTTPS
C
It is required by robots.txt
D
DNS lookups add latency to every fetch and can bottleneck throughput at billions of pages