Practice
Learn
← Quit
0 / 0 correct
System Design
Medium
1 of 20
Web crawler: why cache DNS results inside the crawler?
A
To avoid HTTPS
B
DNS results never change
C
DNS lookups add latency to every fetch and can bottleneck throughput at billions of pages
D
It is required by robots.txt